Categories

The Android application localizes !

On the last day, we’ve rebuilt the Android application completely from scratch and it works ! This screenshot has been made in the foyer of Telecom. In red are the tweeters, in black our position.

The application will be better tomorrow, stay tuned 😉

2013-05-03 00.20.48

Structure of the Android application

Just a little post to describe you the current architecture of our Android application 🙂

As we are not 200% sure on the algorithms to use to detecting the time arrival differences, the localization and the bip planning, I was forced to do a flexible architecture. The idea was to cut the processing path into blocks of different nature, and use Dependency Injection to make replacing each block easy.

 

archi-android

 

  • AudioThread/AudioFifo acquire the audio data from the soundcard at 44.1kHz / 16 bit PCM data, stores it in a FIFO, and redirects it to the next layer. We’ve only analog data here.
  • PeakDetector takes the analog data, and deduces from it the presence of an audio peak. It doesn’t have any knowledge of the number of tweeters, of the planning of the beeps, and so on. It’s only a smart threshold with different technics (basic FFT, Groetzel filter, cross-correlation, …)
  • PeakProcessor takes the information about when we’ve heard a beep, identifies which tweeter has made the noise, and maps it with the planning of the beep to deduce which tweeter has beeped when. If it has a valid vector of emitter 1 => beeped at 20 µs, emitter 2 => beeped at 40µs, …, it gives this vector to the next layer
  • Localizer takes the information of the different times of beeps and uses it to localize the smartphone. After having the data, it gives the information to a custom MapView for displaying it.

In order to fetch the data on the Android smartphone about which tweeter is where, how many tweeter they are, and so on, we needed a simple way to fetch these data.

To do so, we’ve developed a server using Node.js which acts as a XML-RPC server which can give the current state to the smartphone, and store the current state from an external source. It also has a web console on port 9091 so we can monitor how many emitters there is alive from the network, and so on. The server is hosted in a dedicated OpenVZ container at OVH, so we don’t worry about if the server’s port is accessible from the smartphone or not.

nodejs-console

The Android application now fetchs this data so the PeakProcessor / Localizer can work.

On the other side, a Python script will take data given on the USB port of the PCB which sync all the clocks, and then send it via XML-RPC to the server so we can tell to the Android client how many tweeters are alive, which are alive, and their beeping order.

A little success

341

 

That was a bip at 440Hz detected by an Android phone, using cross-correlation 🙂

All the structure of the Android application is there, I’ll come to that in a future post.

As soon as the localization algorithm has been finished to be transcribed from Python/Sympy to Java/Common Math, we should have a basic usable Android application which will be able to localize the smartphone and show the position dot on a map.

The architecture of our PCB

Some days ago, we discussed about the architecture of our PCB. I will be short, and just give you the overall design.

 

archi-pcb

 

 

This PCB will be used as well for the emitters and the card which will emit the master clock (but we can also use the course’s card for this).

It is designed to be relatively generic : some part will be not soldered so we can add them after if we need them. The example would be the audio codec. We don’t think we’ll need him but it can serve as two purposes : once could be measuring the speed of sound (which vary with the humidity for instance), and the second could be using one of the cards as a receiver.

H-bridge

Alexis Polti gave us some tweeters he ordered. My goal was to determine the electrical characteristic of the tweeters in order to design the H-bridge.

As the tweeter acts as a capacitor, we’ll need to add a resistor in series with the tweeter in order to protect against intensity peaks the capacitor will generate when the voltage changes (i=C * du/dt).

So, I need to figure out what is the capacitance of the tweeter at the given frequency, so I can calculate the resistance value (it will act as a low-pass filter for the capacitor). With the tweeter was given a little circuit which generates the ultrasound frequency.

 

Oscillator circuit

 

After soldering it (and figuring out that the short circuit was because I haven’t applied the erratum. Yes, the PCB was faulty), I have been able to measure the frequency output, the voltage, and the intensity of the tweeter. The strange thing is that even if the datasheet says it will generate a 23kHz frequency, I only see a 19kHz frequency with a very crappy signal (you can see the frequency by clicking on the image and zooming — I had only two hands to take the photo and connect the circuit 🙂 ).

 

Oscilloscope

 

I measured 6.7Vpp, 19.01kHz and 202uA. That means the capacity of the tweeter at 19.01kHz is 1.59 nF. So by using a resistor of 10kohm, the cutoff frequency will be over 40kHz which will be really fine.

Some logic analyzer fun

As we want to synchronize our slave clocks using the XBee modules, we wanted to know what could be the difference in the reception dates between two slaves when the master clock broadcasts a “clock sync” message.

So I wrote a little bit code which toggled a GPIO when the message “clock sync” (0xAA) is received. The master firmware would emit 0xAA at 2Hz.

Then, I used a logic analyzer to observe the time difference between the two GPIO blinks.

Logic Analyzer

I found a mean difference of 60µs with a standard deviation of 55µs. I’ll let Issam & JB conclude (because they have far more knowledge in the localization part than me) but from my point of view, it seems to be fine.

Episode 2 : more changes

Last Friday, we discussed with Alexis Polti and Samuel Tardieu of our current view of the project. This was really interesting.

First : our assumption that Ethernet would be great was false. Even in a local Ethernet segment we can’t assume that the jitter would be nearly inexistant if there is other people using the segment. No more lwIP then 🙂

As using Ethernet was mainly used to synchronize the beeps, they proposed a solution : using the emitters timer and just synchronize their clock using RF.

 

ZigBee sync architecture

 

So I wrote a prototype of code for the TP boards : a master would emit a synchro pulse via ZigBee every 4s, and two slaves will reset their timer when this message will be received. The timer interrupt would then be used to generate a pulse on a GPIO.

Using the LED as output, slaves clocks seemed to be well synchronized, but I need to analyze this using a logic analyzer to figure out if the slaves clocks are really synchronized enough for us.

Then, Alexis Polti told us that it was useless to use a class A amplifier with a pure sinus : making a square wave using the PWM output with a class D amplifier will result in a much higher efficiency. The piezzo tweeter will then filter out the square form to make it a square wave. No more DAC used.

In the week-end Issam and Jean-Baptiste made huge progress on the simulation side, but I’ll let them explain their work 🙂

A late tale. Episode 1 : how we reduced the complexity, and the story of a failed lwIP attempt

First, I want to say that I’m sorry. These last weeks I was not really writing posts on this blog for two reasons : the first is that I don’t like to write to say things with no deep content (like saying that I did my TP work). The second is that I was a little bit procrastinating these blog post, to be more focused on using the free time I had to currently work on the project, constantly saying that I’ll write something when I’ve finished what I’m currently doing. As a result, the WAMI project was not very transparent and I’m partly a culprit for that.

So, I’ve decided to fix this and write the tale of WAMI, the tale of our previous weeks of work.

Jean-Baptiste and I met on the 10th of December to merge our views of the project. We agreed that we needed to remove some complexity layers we’ve put then. No more complex modulation, no more complex error-correcting codes, and so on : we’ll only produce beeps. We had too many disavantages in term of complexity for the advantages we would gain (which was having a perfectly scalable system up to 255 emitters) to continue.

Our system is now far more simple for a nearly identical result.

We’ve decided to simplify even more : the emitters would be only a microcontroller with an Ethernet port which would connect itself to a local server using TCP. The local server would be a Python program which would say to the emitters “beep now” at the correct times. The idea behind this was to assume that the local Ethernet segment would have nearly no jitter so the latency would be constant and the latencies would be compensated in the localization algorithm (it computes by relative timestamp differences).

 

Ethernet architecture of WAMI

 

The emitters would use the mirocontroller’s DAC to produce the sinus wave to be emitted : it’s simple, effective and reprogrammable. So we’ve studied a little bit which class of amplifier we would need : as we were considering that we would need an amplifier which will not alter the signal content so we would have less error on the receiver, we were opting for a class A amplifier (like a common-emitter amplifier).

So Jean-Baptiste had just to figure out the formulas for doing the localisation given the beeps timestamps, and Issam had just to write some Python programs so we could test on computers what we’ve found before going deeper in the electronics part.

I have the chance to have access to a STM32F4DIS-BB board with all its accessories (Touchscreen, camera, …) which has an Ethernet port. I decided that I could try to begin the firmware using my development board while I wait Issam and Jean-Baptiste, so we could have a little precious advance for the firmware when we would receive the PCB.

STM32F4DIS-BB with its Ethernet port plugged

So I tried using ChibiOS with lwIP using ChibiOS’s demos. ChibiOS has an embedded version of lwIP (in the ext/) folder, and has a set of tools which integrates it nicely in ChibiOS (like pre-made configuration threads).

After tweaking the STM32F4DISCOVERY ChibiOS’s board configuration to route to the proper alternate pins, I hit a lot of problems.

Firstly, just note that if you want to use RMII, you need to add #define BOARD_PHY_RMII 1 in the board.h.

Secondly, on the STM32F4DIS-BB you must reset the PHY manually using a GPIO, but ChibiOS’s reset method in halInit(); issues a soft reset using RMII (so the PHY would already had its hardware RESET phase done). halInit(); was looping infinitly waiting for the PHY to send him that it has been successfully reseted. So I’ve replaced ChibiOS’s reset method with my own, using #define BOARD_PHY_RESET macro on board.h which will trigger the reset pin using the GPIO (by the way, I wasn’t able to figure out how to use the triggers of the poor Bus Pirate’s logic analyzer mode to figure out if I did the reset part correctly. I wonder if the triggers really work.)

Thirdly, ChibiOS’s uses a strange method for probing the PHY address. It sends a message to all possible PHY having a specified physical address and sees if someone replies. The problem was that the PHY is a SMSC LAN8720A and its PHY ID is not in ChibiOS’s constants so I needed to figure it out. After digging deep in the LAN8720A’s datasheet, I couldn’t manage to have a reply from the PHY. So I looked in the examples given in the STM32F4DIS-BB kit (on Element14 website) which use FreeRTOS and ST’s peripheral library, to see that they were using a PHY address of 0x01. So I skipped the probing method, and used #define BOARD_PHY_ADDRESS 0x01 to do this.

At last, I was locking on a DMA transfer (the micro-controller was looping infinitly for a pending bit). I stopped there, as the next post will tell you some direction changes in our project.

Thinking about modulation

In our presentation, Alexis Polti suggested that we should only listen to raw pulses and not emit frames, using a blank pulse for sync (to detect which is the first  emitter). The problem with this solution is that if you don’t receive the signal from one or several emitters, but you have 4 emitters in range, you will misinterpret the blank due to the out of range emitters and you will have syncing problems.

That means, we might have to send at least the number of the emitter by ultrasound. So I studied a little bit which modulation we could use for sending this data.

I warn you : I’m not really good at modulations, and it’s probable I’m wrong, please correct me.

First, we don’t want a huge data bandwidth, we only want reliable communication, so we want to reduce as much as possible the bit error probability.

A little computation shows that BPSK and ASK have the same bit error probability for the case 1 bit per symbol.

But, QPSK has the interesting property in which you can transmit 2 bits per symbol for the same error-bit probability than BPSK : the only drawback is you need twice the power for emitting the symbol (the energy per bit is constant), so it could be more interesting for us to use 2 bits per symbol. A plot with 4-QAM shows that you have a little bit more bit error probability.

peeb

As I didn’t manage to find data for MSK and FSK, I couldn’t draw them on this plot. Although, I managed to find some informations on how to boost communication range and they seems to go against MSK/FSK for this kind of use-case.

Also, we must consider that maybe, tweaking the transmission by using a more suitable modulation might be pushing the boundaries of our project a little bit too far.

And, as I said, I also can be completely wrong.

Initial work

We’re currently researching on how we can technically achieve the localization, so we can have a better look on how our project will be done.

We were investigating a method where smartphones are listening to ultra-sonic beacons produced by known stations and then deduce their absolute position.

This has some great advantages : first the fact that it’s a software only solution for the user makes it very cheap and really scalable, but also we can control the emitting power of the beacons and so reduce the number of them (also cost effective) as we reduce the scalability problem of multiple emitters on a very small bandwidth (it will enable a faster localization).

After some investigating by reading some scientific papers (for instance [1]), the problem will more likely to be around the sensing rather than on the emission : not a lot of microphones can listen to frequencies higher than 20kHz, and cheaper microphones often stop at 17kHz. For a comparaison, audible frequencies are between 20Hz – 20kHz but some people can listen as up as 25kHz.

Although we may need to measure the frequency response because maybe we could manage to achieve a 1kHz bandwidth by pushing the beacon signal louder. I’ve searched on some MEMS microphone datasheets for  the frequency response diagram, but none seems to display frequencies higher than 20kHz, although 19->20kHz may be usable for our needs (we need to do some tests). A 1kHz bandwidth should enable us to achieve a theorical bitrate of ~50bps using Shannon formula with a 60dB SNR (a value derivated from ST’s MP45DT02, although this one seems pretty bad, that was the only one I found with a SNR which is not A-weighted), which should be sufficient for a slow localization.

But, [1] seems to argue that mobile phones speakers may be able to generate sufficient ultrasonic sound to make sub-meter localization possible. Making the devices emitters instead of receivers is the method used by AT&T Active Bat (http://www.cl.cam.ac.uk/research/dtg/attarchive/bat/) and it could be a good alternative. In their paper, the authors are currently using the 17 to 22kHz frequency range, which is stepping a little bit in the audible spectrum.

In conclusion : we need to evaluate the different frequency response of different mobile phones, determinate in which frequencies we’ll use, in order to see whether the method of sensing at mobile-phone side could be maintained.

[1] Filonenko, V.; Cullen, C.; Carswell, J.; , “Investigating ultrasonic positioning on mobile phones,” Indoor Positioning and Indoor Navigation (IPIN), 2010 International Conference