Drops N’ Roses : Cache app

Hey !

Since last time, I have made a lot of things.
First of all, I worked with Matthieu on cryptography. I worked on scaloid with java spongy-castle library while he worked on the equivalent C library. At first, I programmed key generation, message signature, and message verification. I also programmed a method which allows me to generate the same public key as the one stored in the drop, thanks to an exchange in BLE of some parameters. Then, we wanted to exchange signed messages between my android app, and the drop, and that the drop stored the message only if it has the right signature. We spend a lot of time on it because for each of us, we could sign and check our own messages, but we were not able to check each other messages. So the drop never stored anything. We found that it was because of the hash : it was not done in the same way in the two lib (because of LSB and MSB). So we fixed it, and it worked.

Then, I began a new app : Cache. It is the app for treasure hunt.
I made the first activities. Even if the design is not really awesome yet, it was functionnal.

Because informations concerning the different hunts, and the clues have to be stored into the smartphone, I used SQLite to create two tables, one for the hunts, the other for the clues. I added methods to access the database, get all clues concerning one particular hunt, delete hunt/clue, etc…

In addition to that, I implement the whole behaviour of the app (the fact it has to connect automatically to all drops the app meets to know if there are some new hunts and notify the user if it is so, or check if the drop do not contains the next clue of a current hunt, etc…)

However, when we tested the app, there were some connection problems (connection to a drop was successful one time for ten attempts).

Moreover, I have not test all functions of the database.

I hope we will fix everything concerning this app tomorrow, to prepare a little hunt for you !

Drops N’ Roses : Guestbook and crypto

Hello !

So yersterday, I programmed Cairn guestbook v1.0. It gets the ten most recent messages and displays them on the screen as you can see.
So, at first, I wrote in my flash drop 10 messages with “This is a message for Cairn application. This is the %d comment of its guestbook”. Then, I added two short messages thanks to my application Cairn. Messages had to be short, because we have still some problems to write long messages even if we know how to do to send long messages, we do not have programmed it in Scala yet.



I also discovered cryptography based on elliptic curves. It seems to be a really good solution for signing messages. We discussed a lot (as always) about how programming the different functions we need to distinguish an authenticate person, a signed message, etc… for different use cases of our drop.

Then, we dispatched the work, and I was able to code again in C (Yippee!) to manage public, authenticate, and admin rights for the different “opcodes” that someone can send to the drop, in order to read/edit/write/delete a message.

At least, this evening, I tried to found an equivalent library of the one we already for the nRF… but for java ! We want indeed to be able to verify the signature in our Android application. I found a lib called bouncy castle, but I dit not achieve to sign and verify a message with the keys that Matthieu generated on our drop, with the C lib.

I am also a little bit anxious, because we have so many things to do, and… it is nearly the end !
I keep hackin’!

Drops N’ Roses : more Scaloid

Hey !

I am still working on Cairn application. It is a huge work to do on Scaloid. I have an application able to scan and list the specific cairn devices (it is done by filtering some advertising data). This app also notifies its user when a new cairn device is found. Then, it gives the possibility for the user to connect to any device nearby.
The next step is to list automatically the ten most recent posts in the guestbook.

It is still hard to work on Scaloid (because I am not used to it). But I was much more concerned about the guide lines I will have to adopt. I have some problems implementing the different functions and activities because… I don’t really know what is for the best.
It is quite weird to be confronted to such problems. I really feel lost.

But it is in progress… step by step, I create my first app 🙂

Drops N’ Roses : Scaloid programming

Hey !

I am sorry, I have no amazing photographs or videos to show you. However, I have not been inactive.
Thanks to the work of my coworkers (and my own), we are now able to write/read/edit/delete messages in our flash thanks to our application DropScanner. For example, I modified some functions of DropScanner to match the format that we defined for our BLE message characteristic. I also continued working on Android for creating our Cairn application. I have to automatize the connection to Cairn drops, to get directly important messages, etc…
I am not used to Scaloid yet, even if it is better than a few days ago. But I would like to work on something different because I don’t feel comfortable with Scaloid programming. I am really slow because of design part, and because I am a bit lost : I don’t really know where I should write my functions (API, local activities, or in another file). I am wondering how I will be able to finish the project (and really go on) if I am so slow (because a huge part of the things left to do are linked with Scaloid programming.

That is why I also try to debug our drops with Matthieu and Lau, to clean our repository, etc…
I checked that our drop is robust, by letting it advertise all night long without any crash (that is quite reassuring).
I also reflected on how we will modify advertising fields to add our own data, and I think we will use the “field data”.

I also thought of how we will make the CPU and our memory sleep, but I have to wait until Tuesday to actually make some measurements (we will have some material to do so we don’t have yet).


Take Me Home, Country Roads

I hear her voice
In the mornin’ hour she calls me
The radio reminds me of my home far away
And drivin’ down the road I get a feelin’
That I should have been home yesterday, yesterday

Hello World!

Since Wednesday, I have been exiled in my lovely little village: Pernes les Fontaines. However, this is no excuse to stop working and enjoy the beautiful weather of Provence. So when I arrived, I immediately brought out my computer and all my cables, and now there is an official branch of the A406 in the South of France.

My main task was to implement all the BLE services and characteristics on the drop. For now, there is one service (Drop) and two characteristics (Message Command and Box Command). The Message Command characteristic has already been explained with schematics in a previous post, but since there has been some changes, I will publish an updated version soon. But the code behind the scene is now working thanks the amazing work of Lauriane on DropFS. I heavily tested the message commands (and the called DropFS functions) with gatttool, because I don’t have a phone with BLE. Fortunately, I took some time to develop a web interface to help me writing requests and parsing responses (don’t care about the design, this is just a developer tool !!).

Message Command Generator

This tool helped me a lot to test all the functions, and check if nothing had been broken when I made some changes in the code. And good news: we now have a working version of the Message Command characteristic, which supports the basic operations (create, read, update, delete) and returns potential error codes.

I also started to work on the Box Command characteristic. But there is still a lot of work to do in DropFS, so I keep this task in background and I update the code as soon as new functions are added by Lauriane.Another job was to migrate to SDK 4.4.2. As I explained in my previous post, we had a bad surprise because we received an outdated and very buggy version of the nRF51822. So we had to rollback to a previous version of the SDK and I performed this migration (otherwise, the drop was simply not working, so we couldn’t test anything!).For the next days, I have a lot of items in my todo list! I have to think about a way to handle long messages (for the moment, we can’t write messages longer than 13 characters). I also have to add new commands and characteristics to reflect the functions that Lauriane is adding in DropFS. And above all, I have to find all the easter eggs before my brother tomorrow.

May the force be with you!

Drops N’ Roses : Stressful day 1000

Hey !

As Matthieu and Lau said, yersterday was a stressful day. I spend the day trying to make our PCB send something in BLE… to prove that our antenna and balun were functional. I checked everything : the fact our components were well-welded, the fact we had only the high frequency quartz, the fact that the softdevice 6.0.0 was not working on our chip (we have a N51822QFAAC0). With Alexis and Sam, I even check with an oscilloscope the behaviour of an eval kit with a N51822QFAA which was functional, to compare it with our drop, which was not functional… And it was quite weird, because the behaviour was quite the same for each pin. I also checked our PCB plan, because I was thinking that I made a mistake.
At least, we noticed that the quartz on the eval kit was 16 MHz and not 32Mhz as we thought. Matthieu, who was reading all bugs for SDK4 that we had to use (we have a N51822QFAAC0) told us that one bug was that the microcontroller could not work with 32MHz…
So Alexis welded a 16Mhz quartz instead, and it works ! We were really eased.

This morning, I spend some time correcting some part of my code for programing the memory, or refactoring it. I also wanted to add the hardware protection by controlling the WP pin. I thought I programmed it correctly, but I made some tests, and it does not work… I still don’t know why, because I really do it as the documentation recommends it.It is quite mysterious… so maybe we’ll have no hardware protection on our external flash.

Then, I read all our Scaloid code… again and again, to understand it. It is quite complicate to understand for me, because I am not used with Java anymore, and I have never programmed anything in Scala nor Android.
I began to make some minor changes. I need some information from Matthieu to continue, because it concerns BLE characteristics to exchange data, and we have to agree about what I have to send him.

As he sends me an email when I was writing this post, I think I can go back to work !

Day 51: DropFS and a big fear


Before the week-end and on Monday, I worked on DropFS, our flash memory management system for drop messages.

First, I had to establish precisely the byte (even bit!) mapping for each type of page in memory. I did it with packed structures and bitfields in order to simplify parsing. Consequently, I spent a few days on writing and rewriting lots of typedefs and function prototypes, because our model changed everyday and because I wanted data structures as easy to manipulate as possible. And when we discovered that our flash could be written bit per bit, I once again had to rethink all my structures! But now I think that the result is very near to its final state.

Then on Monday, I was able to begin programming DropFS’s actual functions. Right now the basis (creating, reading and deleting indexes, boxes and messages) is done, I only have to finish the write and edit functions for tonight. Actually, I didn’t have much time to work on DropFS today because we received our first drops and discovered that their radio wasn’t working! Hopefully, we solved all the problems we had (with a big help from Alexis and Sam!) and now each of us has a working drop in their pocket. To summarize, we were unlucky on several points: the nRF we received was a C0 revision instead of G0 (with a lot more bugs and differences for PCB design) and some components weren’t delivered in time, so we had to use similar but inadapted quartz, capacities and resistances for the first prototypes. With this special combination of changes, everything went wrong! We were in a configuration where our SoftDevice and SDK versions weren’t supported, the HFCLK was supported by the chip but not the radio… In the mean time, our git repository went half-crashed… For a few hours, we thought that maybe none of this year’s PCBs could work with BLE. So having them working tonight was quite a relief.

Solving these issues nevertheless took us the whole day and we weren’t able to do anything else. So I can’t wait for testing DropFS on an actual drop!

I just came to say hello

Kinda like this thing but there’s something you should know
I just came to say hello

Hello World!

What a touching moment. Our first drop (Drop65) just said “Hello”.

Hello Drop

It was not an easy journey to make it work. We came through doubts, tears and despair. But we finally got it. We got a sign of life from our dear and beloved drop.

We encountered many problems before achieving this. First, we noticed that the nRF chip was a very old revision, with its bunch of bugs. So we had to downgrade to a previous version of the nRF SDK compatible with this revision. Then we tried to test a simple program using BLE, but the drop was not even advertising (whereas the same program was working on the development board). Next, we decided to test another program directly using the radio (without BLE), but again it did not work on the drops. Finally we tried to replace the 32MHz quartz with a 16MHz quartz (so that we had the same clock than on the development board). And guess what… it worked! We can now see our drop advertising! However, we still don’t know why it was not working with the 32MHz quartz, but it seems that there is a bug with this quartz frequency in our chip revision.

May the force be with you!

Drops N’ Roses : AT45DB321E test

Hey !

This week-end (and yersterday) I made some tests to check if the behaviour of our drop was correct.At the beginning, I created a new file in order to work with a clean repository (because I could not stand to work in the SDK file anymore). Then, I created our own drop_board.h, change the Makefile, and make a really simple test : a blinking led (yes, we have one !). So that is how our dead drop came alive.

After this primary test, I tried to write and read some pages of the flash memory. Write a page, erase one, and read one were succesful tests. But writing a byte was giving strange results. So, I factorize my code to make it easier to read… and I corrected a stupid mistake (a & instead of a | ) for writing the byte address in the same 3 bytes already used for page address. And this time, the function to write a byte was working. Afterwards, I check if it was possible to write a whole page and rewrite some of the bytes of the same page without erasing the page.
To give an example, what I would like to test was :
– I write a whole page with some bytes at 0xFF.
– Am I able to rewrite those bytes at 0xFF ?
The answer is yes. And it is even better ! I can rewrite a bit : if I have 0xFF (1111 1111), I can put 0xFE (1111 1110) and then 0xBE (1011 1110). Then, if I write 0xF9, I will obtain (1011 1000).
This is because the write operation for this memory is just puting “0” instead of “1”. And because we can’t erase a smaller area of the memory than a page (meaning to put every bits of the page at 1), we can only lower the value of a byte when we write the flash. That was a relief (even if we have to change our memory architecture, because we were not sure to have this bit writing possible, and so we plan to use a complete byte for some information that we can now store in one bit.

Then, I would like to test the BLE. But… this time, we had a really bad surprise. I was wondering what version of the SDK I could use with my drop, so I read our nRF number : N51822QFAAC0. The C0 corresponds with an old version of Nordic SDK (4.4).
With our eval kit, we were using SDK 5 because we had N51822QFAAG0. And there are a lot of differences between these versions. That is a bad surprise for everybody, because some functionalities are only working using the new SDK.

After this bad new, I decide to do another thing : merge our branches. And it was quite complicate because we were not working in same branches, nor at the same level (because Lau and Matthieu still worked on the SDK file, whereas I created a new one), but some changes were made in “same” .c or .h.
Sam helped us a lot. We had to rewrite history, to pretend that we had always worked on the “new file” I created, and then merge some files, etc… It was not so easy.
But now, it is done. And I added the SDK 4 in our repository as well.

Yersterday evening, Matthieu was trying to make the BLE working. I hope he succeeds.

Drops N’ Roses : synchronization protocol

Hey !

Be prepared for the longest post ever…
Today, I would like to explain to you our synchronization protocol. As I said yersterday, for some applications such as KTA (speleology application), we would like to be able to both :
– keep some messages in a specific drop, for example for direction indications.
– spread some messages in our drop-network. It would allow anyone to access these messages everywhere. It would not have been possible without spreadable messages, unless the user goes to that specific drop.

But for having such spreadable messages, we need a synchronization protocol.

When we will have it, we would be able to synchronize a smartphone and a drop, and then, if the smartphone meets another drop or another smartphone, they would synchronize themselves, and so… one by one, every network drops would have the same messages.


  • Use Case

In order to explain how we will proceed, I am going to take an example and develop it during all my speech.One of the aims of KTA application is to avoid people to get lost in caves, and to make research easier if someone really get lost.
To do that, we want to store (and spread step by step in all our drop-network) for each user a message, which contains his pseudonym and a list of his last locations (the id of the last drops he met) and a timestamp for each location. Knowing that the size of a message is limited, we would be able to store around 50-60 locations for each user.

  • Problems

This use case raises some problems :
– to be able to spread a message, it needs a unique id in all our network.
– if we want to edit a message, for example to add a pair (position and timestamp), we will have to distinguish different versions of the same message and identify the newest one.
– to avoid that a user A erases user B ‘s messages, or even worst, changes the content of the message with pair of location and timestamp, and replace them with others, we need to protect these informations. the only user that could edit his message should be himself, but other users should be able to spread his messages as well.
– because our memory is quite limited (32Mb is quite big, but… ), we want to forbid someone to flood the memory. We want to allow only one message for the box “location” per person.
– we want also to manage the term date of a message, and to stop its spreading when it is too old. (We want to save some battery !)

  • Our tools

What we are going to use are :
– a web server, with a data base which will contains a limited number N of users. It will also store their public key (RSA), and their pseudonym. This server would be accessible thanks to our mobile-application, out of the caves. Before to go in catacombs, people would have to register on this server if they want to use our system which stores their last locations.
– on our drop, we create different boxes. For this use case, there are two boxes (n°1 and n°2) whose size is N messages, and another box (n°0). Box n°2 contains location messages for each user, certified by each user (with their private key (RSA), corresponding to the public key they have on the server). Box n°1 contains  a set of messages. Each message is certified by the server. Each message contains a server version number, the pseudonym of one user, his rights (I will explain what it is for later) and his public key. These keys will serve to check the certificate of messages in box n°2. Box n°0 contains the public key of the server, to check the certificate of the keys in box n°1.
The advantage of using certified messages is that nobody (except the owner of the message) can change its content, but everybody can store the message, and spread it.

– a very simple synchronization protocol : one for the keys synchronization and another for location messages synchronization.
But before I can explain these protocols, let me explain how the server and its data base works.

  • Server and data base

Each line of this table represents data for a user in the data base : the first item is the server version, the second is the user’s pseudonym, the third his rights, and the last his public key. All this data are certified by the server.

Server data base

1 – User A – Rights – public key

2 – User B – Rights – public key

3 – User C – Rights – public key

50 – User D – Rights – public key

So, imagine N = 50. You can store only 50 users in your data base. And now, it is full. If another user wants to use our system, he can’t. Except if for exemple, user A don’t want to be in the database anymore. He is deleted, and so, another user can register.

Server data base

51 – User E – Rights – public key

2 – User B – Rights – public key

3 – User C – Rights – public key

50 – User D – Rights – public key

The version number is incremented, so it is now 51.

  • Keys synchronization

We want this database to be stored into our box n°1.
Please, consider that each drop has already the public key of the server stored into the box n°0.
Consider that someone (user C) downloads KTA application, and registers as “user C” in our website, and that he is the third person to subscribe. Consider that the drop he would meet has only the first line (corresponding to user A) of the data base stored as a message in its box n°1. The drop will advertise “KTA” and “1”. Thanks to this advertising, the smartphone of user C would know that the drop is not up-to-date. So the smartphone would connect and exchange step by step the (version + pseudonym + rights + user’s public key), all certified by the server. It means that, if the smartphone tries to send something that is not certified by the server, the drop would refuse to store it. In our case, if the smartphone send at first (3 – User C – Rights – public key) certified by the server, the drop would also refuse to store it, because the drop wants the second version, and not the third.

That is because we want to simplify the behaviour of the drop, and make it a bit stupid in order to avoid calculations and save energy. This behaviour is also good, because even if a smartphone has to send 40 versions and get disconnected before it has transmitted all 40 versions, the drop would be on a steady state, and would advertise the last version it has stored. So if another smartphone came, it can get the versions it has not yet without any calcutation required.

But what if now, we take the example of a 51th version again. And the drop is synchronized with the 50th version. If a smartphone with the 51th version meets the drop, it would synchronize : the drop would delete the first version, and replace it with the 51th version.

What if it was not the first version which was replace but the third, because user C want to give up KTA. If someone (user F) registers, the database would change the first version (1 – User A – Rights – public key) into fifty first version (51 – User A -Rights – public key). Same for the second version. And then, for the third, it would create (53 – User F – Rights – public key).
Proceeding like this assures us that we would keep a steady database and synchronization behaviour with the drop.

  • Location messages synchronization

As I explained ahead, the drop would advertise the server version number, so keys would be synchronized.
So now, we would like to synchronize messages which contains the last (locations + timestamps) of this user. These messages are certified, to assure the fact that it is its owner who posts it.

On connection, the drop will send a list of pair with : user ID (for our example, it would be between 1 and 50) and the last timestamp of this user. The last timestamp would be known because it would be the first data of each user message.

The smartphone compares what it receives with the messages it has. It determines which messages it should give to the drop and would message it should ask from the drop.

After these exchanges, the two devices are synchronized !

  • Conclusion

This is the principle of our synchronization protocol.
All smartphones would use this principle to synchronize themselves with drops and other smartphones.
And with this way of doing, we also can send (when someone goes out the caves) these data to the server. Thanks to that, without moving, an administrater could monitor people movements and if someone disappear, he can directly go to the last location where this person has been seen.

Next time, I will complete this protocol with the management of expired messages (too old to be spread in the network).I hope I have been clear. It was quite complicate to explain !
And… congrat’s if you read all this post !