Categories

[Little Brosers] The Delights of Working with Android

Hello everyone,

Last week I resumed working fully on the Android app, the app on which I’ve been working On/Off for about 3 weeks. My major sticking point was moving the scanning to a foreground service, which is a service that the user is aware of, and that keeps running after the app is swiped from the recent apps list (that is as long as the system has enough memory for its other tasks). The problem was that on my device, the Huawei P9, the service was getting killed the moment I swiped the app from the recent apps list. I tried everything I possibly could, until a teammate of mine tried the app on his phone, and it worked like a charm. What a delight it is to work with Android, where every device acts in its own special way. After a long search I was able to find the setting that made my app work on my Huawei.

 

So right now the app scans for BLE devices in a separate service, in a separate process, and when it finds one it sends a notification with the BLE device’s address as a proof of concept at the moment.

 

BLE scan screenshot

 

With that out of the way, we can now focus on refining the scanning, while implementing our filters to wake up the app only when a Little Brosers drop is detected. The global protocol is explained in one of my previous posts.
For now, with the help of the dev kits, we will bring the app closer to reality and make it establish a conversation with (what will act as) a real drop.

[Little Brosers] Stepping into the PCB’s design

Hello every one!

 

Ok, first things first : yes, my week in Madrid was great. Thanks for asking.

You need to see the Royal Palace if you ever go there, it’s beautiful. Seriously.

Let’s get back to our dear dead drops. I had no chance to publish any post last week but still, there is always something happening in the ROSE’s projects!

 

Updates on PSSC

We began this week with a meeting about PSSCs. To be honest, some of our early PSSC were late on schedule and others had not much time left before its deadline. Thus, the first days of the week were dedicated to put the finished ones in “Wait for validation” state and actually finish the others. We had a code review session as well as an android app test among the team’s member’s phones. The goal was to make sure the ongoing notification (the ones you can’t dismiss by swiping) generated by our app could not be dismissed on any of our phones.

We now wait for the ROSES’s guru’s validation.

 

Lucky me!

Ok, so this week I wanted to start our PCB. The thing is… there is not su much to wire, and that’s great!

 

Indeed, our drops will only contain a battery holder, a flash memory, a JTAG connector, a SoC , a quartz and some passive components. We don’t even need to bother about the battery recharging system because… we don’t need one. Remember! Our goal is to be able to hide our Drops in a hard to reach stash, we don’t care about recharging the battery.

Moreover, our dear Nordic engineers have put into their info center a typical NRF52832 schematic for us. I started from it to design our PCB.

Also, by default I chose the DC/DC power mode for the SoC. We’ll discuss it in our next group meeting, but it is very likely the power mode we will use considering its lower consumption compared to LDO power mode.

About our flash memory, we made the simple choice to take the same than the previous Dead Drops-like project (ROSE 2014). Of course, the idea is not to copy their work but reading theirs posts about this particular flash and their PCB helped me determine how we will wire it for our NRF52. In their logbook posts they also explained the few trubles they had with this flash memory. It’s a work we will not have to waist time on. Thanks’ fellows!

 

I guess that is all for this week.

 

See you next week!

 

Antony Lopez

Https backend

Currently, I’m working on the backend, which will mainly register user by signing their certificates.

Technologies

I work with Flask because everybody in our group know Python and they will be able to understand and review my code !

I chose MongoDB because our db has no relation, it just stores usernames, public keys, and ids.

I work with the cryptography python package, it uses openssl and I found it to be really great (I tried pyOpenSSL, and pycrypto before).

Certificate format

I decided to use x509 certificates because they have already everything we need in them, and they give different serialisation options.

Advancement

My server can already generate and sign a certificate based on the username and the public key given in a http post, but it’s still far from being ready.

I still need to work on:

  • Write the API documentation
  • Write more tests
  • Find the best way to handle CA private key storage
  • Handle User ids
  • Deployment, we will use Heroku for development (of course if there is a real production one day, we will not use it)
  • Get a https certificate (Let’s encrypt !)

We will maybe create a github mirror so everyone will be able to see and try our code !

[Little Brosers] Starting The Conversation

Last week we saw the hierarchy of our network, as well as the initial connection stage. Since then I’ve continued to work on the communication protocol over BLE between the drops and the mobile app.

Authentication

Our authentication protocol is there to guarantee two things: First, that the smartphone belongs to the Little Brosers network (and to the specific network in particular), and second that the drop is indeed a real drop of the network. These verifications will be done using a challenge-response type of protocol.

However, over BLE, the most commonly used protocol is Generic Attribute Profile (GATT), which defines services that group characteristics which essentially are value you can read or write over Bluetooth.

The GATT server will be running on the drop after the connection has been initiated by the phone. After that, the authentication service will be activated, and it will be after passing this stage that the rest of the services will become available. To implement this double authentication, I thought about adding four characteristics to the server:

  • User_challenge (read)
  • User_response (write)
  • Drop_challenge (write)
  • Drop_response (read)

Note that the read/write permissions are seen from the smartphone’s side of the interaction.

So after all each party gives a challenge to the other, and waits for their response. If the response is incorrect the connection is terminated. However, if both pass this phase, then they can proceed to the next stage, which could be more verification steps or simply the start of the message exchange.

[Little Brosers] Python script and CI

Hello there, this is Guillaume from the Little Brosers for my weekly post. This one is a little late because I am in Vienna since Friday and I could not find some time to write it down.

 

I have done two major things this week. First is finishing the python script started last week. It now generates a random pool of messages, simulates two devices taking random messages from this pool, build each device’s Merkle tree, and then simulates the tree comparison protocol. The result of this protocol is the listing of messages from the device and messages the device has to send. All my tests have been successful: each device identified correctly what message it is missing and what message it has to send, so it has been a relief.

 

I have not yet done extensive study of the number of transmission to find the most effective tree structure, but I most likely come to that later.

 

The second thing I have done this week is set up the CI with my group. We organized it in five stages:

  • Checking software version on the docker image
  • Linting of our code
  • Compilation
  • Semantic tests
  • Deployment

We have not yet written propoer semantic tests but it is going to come alongside our code for the project.

 

We also received the development kits from nordic, si I started working on it a little bit. Unfortunately, I am going to be quite busy this week so I am not sure how much I will be able to accomplish. For the moment I managed ot connect to the board through a gdb server. Nect will be making a led blink, and using the UART, then we’ll be ready to roll and start to implement our code.

 

I am looking forward to come back to Paris since there is a very active period ahead of us now that we can start the actual work to implement our ideas. I hope I will be able to start properly working on the dev board before I come back.

 

That’s all for now, thank you for reading.

Cheers,

Guillaume

[Little-Brosers] Encryption[#2] and Compression

Encryption

(context: everyone should be able to verify every message)

We changed our encryption, we decided to use RSA only (with 2048 bit keys).

With the padding scheme (OEAP), we have approximately 190 bytes of plaintext.

The total size of a message is 368 bytes.

The overhead seems huge, we will maybe implement an hybrid encryption with AES, but the overhead would still be big: 16 bytes IV, ~8 bytes padding (mean), 32 bytes MAC, 64 bytes signature)

We hash the encryption key and transmit it to let the receiver know which key was used to encrypt (or we would lose the plaintext non-repudiation property).

Because of the small allowed message size, we can try to compress them.

Compression

I tested two compression software: gzip (with default options) and smaz (source). Smaz is optimised for english, and is only 200 lines of c.

Results:

(sentences about BLE are from wikipedia BLE articles)

Sentence 1: “Les modes BLE (bande passante plus limitée et très faible consommation) et Bluetooth standard (niveau d’émission plus élevé et portée plus grande) sont donc des technologies complémentaires.”

Smaz: 76 % (output size on input size, smaller is better)

gzip (default options): 91 %

 

Sentence 2: “Borrowing from the original Bluetooth specification, the Bluetooth SIG defines several profiles — specifications for how a device works in a particular application — for low energy devices.”

Smaz: 65 %

gzip: 84 %

 

Sentence 3: “eiv9TaecCei3tah9quahJe 7aaCha8AinaewaZ1errait0Koizuo9ahQuikooQu6ahy air3saMeee LiPh8aieWae3HoaiThahb4Eer1dooGwu4Rae2aaichoe5F” (high entropy string)

Smaz: 117 %

gzip: 113 %

Conclusion

Smaz looks pretty good, it is better than gzip even in french. We will try to adapt it for the french language.

[Little Brosers] Finalize the Makefile and setup the CI

Hello there!

 

Guess what ? This week i’m writing my article from Madrid ! So let’s sum up what happened this week int the Little Brosers project, or at least, what I worked on.

Makefile

This week was all about designing a compilation environment. First we started from the Nordic’s SDK examples. It gave us a functional Makefile compatible with the design we had in mind.

First idea was to put all make targets (devkit targets, c++ simulation target, custom Little Brosers PCB targets),  in one Makefile. However, it seemed simpler and cleaner to have a Makefile per platform. Thus we now have two Makefiles. One for building a .elf for the DevKit and another one for the C++ simulation.

Those two pick code from three folders:

  • hw : All code here is hardware dependent
  • core : All code here is hardware independent. It’s basically merging Merkle trees algorithms and cryptographic code. No main.c/.cpp should exist here.
  • sim : All code here is used to test the core folder C code. It allows us to get ride of BLE communication constraints to test algorithms.

The C++ simulation Makefile also comes with the linter related targets designed by me and Guillaume Lagrange.
Those are used by ourselves (programmers) before pushing our code and also by the CI to test the code style.

I worked a lot on the C++ Makefile to allow it manage automatically dependencies and put all the .o and .d files in a “build” sub folder. The objective was to never create any file in the src/ folder of the project. Indeed, we have to keep in mind that other Makefile will read files in the src/ folder.

 

Continuous Integration

This week was also about setting up the continuous integration using GitLab and Docker. This job has been shared between all group members. I took care of the CI artifacts and C/C++ compilation stages. Other group members took care of CI for python, linter and Android mobile app.

See you next week from home this time 🙂

Antony Lopez

 

[Little Brosers] Narrowing the components list and pushing my first commit !

SoC final choice

After more digging into SDK features, we finally chose the NRF52832 from Nordic as our SoC thanks to Nordic’s great SDK.

Why this choice ? I’m glad you asked!

Comparing NXP, Cypress and Nordic SDKs, I came to the conclusion that our development process would be simplified the most using Nordic’s SDK. Not only the C libraries from Nordic such as the nRF5 SDK v14.1.0 are impressively well documented but we will also be helped by the Nordic Android BLE SDK and user guide for the mobile app we plan to develop.

Other tools may be considered. For example, Nordic provide the Power Profiler Kit allowing developers to estimate the current consumption of the SoC during our code execution from 1µA to 70mA with a resolution of 0.2µA.

Also, Nordic’s SoC come with OTA firmware update feature. Allowing us to easily setup the firmware update process though BLE.

Choosing the Flash memory

We had less specific needs for the flash memory.

To estimate the needed size, I used the following values:

  • A message with payload +  Little Brosers’s overhead should take less than 400 bytes
  • A drop should be able to store around 9000 messages
  • A worst case scenario 4-ary Merkle tree with this configuration should have a size of 300kB with 128 bits long hashs
  • The total of those estimation gave us a size of around 4MB

Of course those estimations are not precise but 9000 messages is quite a big number and we’ll allow us to store less messages on a drop if needed.

I also knew that a easy to solder package like SOIC-8 would be way more convenient during the soldering process of our PCBs.

In the end I had 5 flash models packaged in a SOIC-8 with similar features and I selected one of average price, well balanced set of feature and allowing several power modes to adapt its consumption. This is the AT45DB321E from Adesto Technologies.

Pushing my first commit

Indeed, for the drops firmware we plan to separate the C source code supposed to be used in the drops and the C++ source code only executed one our PCs to perform quick simulations. This structure should allow us to develop the hardware independent C code for the drops and test it using C++ ease of use. Those simulations will allow us to perform Merkle tree merges without the whole BLE communication process. We will emulate it with C++ objects.

So, i spoke about my first commit… and what a commit !

Indeed, we decided to start from a Nordic’s SDK’s example four our Makefile structure. Thus, I had to put the whole Nordic SDK in the repository (~120MB).

In the following weeks, I’ll work on editing the Nordic’s Makefile to include our C++ simulation environnement.

Eventually, we would like to have a Makefile from which we could run the following kind of targets:

  • make sim : build a x86 binary including hardware independent C code and the C++ simulation code
  • make devkit : build an ARM binary for the NRF52 DevKit including hardware independent C code, SDK and DevKit specific code
  • make firmware : build an ARM binary for the drops including hardware independent C code, BLE code, SDK and Drops specific code
  • make clean : well… clean the project
  • make flashdv : flash the DevKit with its dedicated binary
  • make flashd : flash a Drop with its dedicated binary

See you next week !!

Antony Lopez

[Little Brosers] Still working on the trees

Hello everyone this is Guillaume from the Little Brosers.

 

This week I’ve mostly continued last week’s work imagining the process of Merkle tree comparison.

 

The major progress has been figuring out the idea I think we are going to use behind the whole procedure. The difficulty is that there are two trees, on two devices that communicate via bluetooth, and we want to minimize the amount of transmission. The main idea is to go through the tree in a simili-depth first search. The first device sends its root hash, and if it is not equal, the second device sends the hash of its children. To keep track of the dialog progress, each device will have a local stack of operations representing whether he has to send or receive next and what he is supposed to send or to wait for. This is still a work in progress so I will not go into details.

 

To help us planning the key values of the tree we started working on a small python script to simulate a pool of random messages, put them randomly on two devices, build their trees. Next in line is implementing the protocol itself inside the script. Then we will be able to run quick simulations and keep track of the number of transmission needed to resolve the Merkle tree comparison.

 

This is mostly what my next week will be. I will also be reading resources on how to correctly set up CI for a project, and start to implement the solutions before we start pumping out code ! One thing I would also want to implement this week is a way to clone our wiki to a public location (most likely a github repo). Our project needs a lot of theoretical work, and the result of the work we’ve already done and the work to come is in the wiki, and making it available would be the easiest way to share our work.

 

That’s all for this week, thank you for reading.

Guillaume

[Little Brosers] Being Heard

Hello,

If you’ve read my last post, you’re probably expecting me to have finished the Bluetooth discovery part of the mobile app. However, sometimes in order to go forward, we have to take a different route, which is why today, we are going to delve further into BLE advertising. But before we do that, let’s take a look at the way we imagined the hierarchy of Little Brosers:

Hierarchy

 

  • Company A
  • Company C
    • Network A
    • Network N
      • Drop 1
      • Drop 2
      • Drop 3

Let’s explain more what each level represents:

  • Company represents the company or the entity that is able to create and manage drop networks. We’re still not sure if we want to add this level or not.
  • Network represents a network of drops within a company.
  • Drop represents the individual drops inside a network.

BLE Advertising

 

Now that that’s out of the way, we can take a look at the BLE advertising packet, more specifically the connectable undirected advertising. It’s made out of a normal BLE packet, where the payload contains 2 sections, AdvA, a 6-byte device address, and AdvData, the actual data being advertised, which can take from 0 to 31 bytes. Our focus is on the AdvData section, which in itself can contain multiple advertising data structures that can offer different types of info. We however, will target a specific type called “Manufacturer Specific Data”. 

Let’s not forget that when the drop is not connected to a device, it will be transmitting this advertising packet at a constant interval, and since this is an essential part of the process, it would be a good idea to use it to announce as much information as possible so that the smartphone can tell from the advertising alone if it needs to connect to this drop or not.

Back to manufacturer specific data, the format is the following:

  • Company ID (2 bytes)
  • Manufacturer data

The company ID is an assigned number by the Bluetooth SIG. This will probably force us to use a specific value that we have no control over in order to stay compliant with the specification. However, the manufacturer data part is where will put the info we need.

  • Little Brosers ID (4 bytes)
  • Company ID (2 bytes)
  • Network ID (2 bytes)
  • Drop ID (2 bytes)
  • Additional info to prevent unneeded connections

Company, network and drop ID are IDs for the elements within the hierarchy detailed above. Little Brosers ID is a magic number that will define to identify the drops as part of the Little Brosers project. The scan results filter of the mobile app will act on these fields to determine if the detected BLE advertisements are of interest or not.

Our Own UUIDs

 

In Bluetooth, services are identified by a Universally Unique IDentifier (UUID). Custom UUIDs which are 128-bit long are used for custom services not registered with the Bluetooth SIG. So in order to follow the same method used for the registered services, we decided to give a 16-bit ID to our services, while changing of course the base UUID so that we do not use the default Bluetooth assigned numbers. Our custom base UUID will contain the following elements:

  • Little Brosers ID
  • Company ID
  • Network ID

I’m not sure if we will include the drop ID as well. At this point it doesn’t seem to be useful here.