During the last 3 days, I worked on the websockets, the music downloading and a bit on the server.
As said in my previous post, I had some problems with the websockets: sometimes the connection between the plant and the server couldn’t be established and I didn’t know why. So on Thursday morning, I carefully reviewed my code and made several tests in order to find the origin of this problem. What appeared from these tests is that when one connection was succesfully established, I had to wait for a certain amount of time in order to have another successful connection. So I made the tests by properly disconnecting the plant from the websocket instead of simply shutting it down. And this solved the problem. On the server, there is a mechanism that periodically checks if the plant is connected to the websocket and if not, close it. But if I try to reconnect the plant to the websocket before it has been closed, it creates problems and most of the time doesn’t work. It was a good news to finally solve this problem.
On Friday, I worked a little on the server, to improve it’s appearance and make it send messages to the plant. But the tests I did to check if my modifications worked well didn’t cover all the cases and Tanguy had to make some corrections after having run other tests.
After that, I decided to work again on the music download. But this time, I tried to first download it in Flash and when the flash is full, read the music from the flash. I used Arnaud’s code (that he had himself taken from someone else) to write in flash and implemented this idea. And it worked well: on Friday evening, I was able to download 900ko of music and to play it. After that, I tried to use the flash as a circular buffer in order to be able to play longer music. And on Saturday afternoon, I managed to do it. There is still one (small) problems: the sound sometimes ‘freezes’ for some milliseconds so I will try to find the origin of this problem if I have time.
Before we receive the PCB (which should happen tomorrow afternoon), I will integrate the wifi code in the dev branch, the one that holds the code that will be flashed on the PCB. And as soon as we receive our PCB we will test them and finalize our project: add the whole logic (properly react when the user ask for turning on a led for instance).
The beginning of this week has been quite productive. Indeed, I managed to have different tasks reaching a functional state.
First, I worked on the server. I added several pages to provide an interface that allows the user to trigger all the actions that the plant can perform. I also improved the login/logout functionality. For the front-end, after some discussion with my classmates, I chose to use Bootstrap. It allowed me to have an acceptable-looking website in a short time.
On Tuesday, we received the new FDC evaluation board (for the capacitive sensor). This time, I connected the pins very carefully, entirely reviewed my code and tested it. After some minor correction, I was able to configure the sensor and to read data by polling it. After that, because polling is a bad thing and that in that case, it can be avoided, I implemented the part of the driver that waits for an interrupt (that signals that a new set of data is available). With this, the driver seems operational to me. Next step will be to connect it with the detection algorithm I wrote a few months ago.
Finally, I also worked on the Wi-Fi module. I greatly improved the websocket part of the driver and after the corrections made by Tanguy on the server side of the websockets, it worked very well. Like with the capacitive sensor, it was done by polling the module. And in this case also it was possible to avoid it by using an interrupt. So I tried to implement this part of the driver. I had some problems to find a pin on which the pulldown resistor worked (because on the 2 first I tried, there was a pull-up somewhere on the board…). And I had also some (stupid) mistakes on my code that make me lose some time. But it forced me to carefully review my code and thus to improve it, so it’s been useful. So now, websockets are fully operational. I also improved a lot the management of the errors and started to implement the decoding of the commands that will be received through the websocket. As one of these commands is to play a music downloaded from the server, I tried to re-use the code I’d already written for this. And the last error re-appeared unfortunately: the Wi-Fi module takes too much time to answer to my requests. And I don’t have any idea of possible origins of this problem. I will try tomorrow with another Wi-Fi network to see if it changes something, but I don’t hope too much.
The end of the week will be dedicated to solving the problem with the music and to connect the detection algorithm with the capacitive sensor driver: assembling several parts together in fact. It’s the beginning of the integration phase and this is a good news.
During the last three days, I focused on the server. Indeed, I want it to be ready (or at least functional) before we receive our PCB (which should occur next week). I found it not so easy as I didn’t know anything about web development. But it’s interesting.
For user authentication, we use the user’s Google account. To manage the OAuth2 process, I chose to use the Velruse library. But I had a problem: when the user was already logged to a Google account, the account selection page didn’t appear thus not allowing him/her to choose another account… Samuel found that it was possible to add a parameter in the request to force this display of this page, but the Velruse library doesn’t provide the opportunity to add extra parameters. So, again with Samuel’s help, I patched the library in order to add this ‘prompt’ parameter. As this patch hasn’t been applied to the whole library and doesn’t provide a way to change the value of this parameter, I didn’t release it publicly. Maybe I will do it later.
I also spent some time to create the pages that allow the user to create an account and to manage his/her flower.
The appearance of the website is horrible, I know, so it will probably be necessary to improve it at least a little, to make it more appealing.
During the next few days, I will work on connecting the WiFi module with the websockets in order to have a bi-directional communication between the flower and the server. And I will continue to work on the server to improve it and to add new functionalities.
Happy New Year!
I hope you spent good times for Christmas and the New Year’s celebrations!
During holidays, I carried on the application. Indeed,
- I put the Bluetooth connection in a service to do that in background.
- I modified the Bluetooth Server to be able to simulate a remote device.
- I handle it to start and stop the service’s thread.
- like the server’s API has been modified, I had to changed how I made requests.
- I made tests to find bug and correct them.
- I changed a little bit the UI.
- I reviewed and cleaned the code.
Now, about the application, I will work with Alexandre to make tests and modified some parameters which will be modified with the new encrypt program. Indeed, encrypted messages will not have the same length. And to split the key and the message, I need to know their length.
Benoît for BreathUp
During the second part of this week, I continued my work with websockets. First, I made them be able to communicate in both ways (in the beginning, I was only sending data, now I can also receive some simultaneously). Then, I tried again to run gunicorn on Heroku. The problem was that gunicorn needs to the port to use in a config file but Heroku allows to use only one specific port that changes every time to deploy. I found a trick that consists in modifying the config file with sed during the deployment. So I was able to deploy our server on Heroku, but we found later that Heroku closes websockets if you don’t use them during 2 minutes, so as we don’t want to ping the server continuously, we will have to deploy our server elsewhere.
Then, I tried to have multiple websockets concurrently. I wanted each socket to have a different URL, the difference being a number at the end of it. With pyramid, it is easy to dispatch URLs with such variables in it and to use the values of these variables for normal views. But aiopyramid that I use to handle websockets like views doesn’t give access to this information. After a long search, I found the problem to be solvable only by adding a line in the code of the library. Later, I also handled the closing of websockets : it crashed on every disconnection, so when we wanted to reopen a websocket at the same URL, it didn’t work. Now, the websockets are properly closed upon disconnection and can be reopened later without any problem.
Next week, I will continue working on the server.
During the beginning of this week, I worked on the server. We want to be able to forward some commands (like setting a Led’s color) to the plant. We could have the plant polling, but it seems better to use a websocket. I managed to have a websocket working with the websockets library for python, but it didn’t work on Heroku. Indeed, I had one thread running the server and another one opening and running the websocket on another port, but Heroku doesn’t let you open ports.
So I searched a solution to this problem and decided to use gunicorn. It allows handling request asynchronously and it runs a websocket without opening a new port when I test it locally. But unfortunately I didn’t manage to test it on Heroku as I can’t find how to deploy a server with pyramid and gunicorn there.
During the rest of this week, I will continue working on the server.
Since last Sunday, I worked on the PCB and on the server. First, on the PCB, I did a complete check of both the schematic and the routing, to find the remaining errors. Alexis also did it on his side, and we both found errors. After having corrected them, I gave my approval to send the PCB to the factory, and so did Tanguy. We should have received them in 8 working days. But as we don’t really need them during Christmas holidays, a longer delay is possible (which is good as it reduces the cost). So our PCBs have still not been sent. But we received 2 evaluation boards for the Wi-Fi module. Thanks to them, we’ll have the opportunity to develop the driver to control the Wi-Fi connection with ChibiOS. Alexis will also solder an evaluation board for the Bluetooth module, so that we can also work on it during the holidays.
After having finished my work on the PCB, I spent several hours on the server. I faced many problems with
git subrepo, and thanks to Perceval help, I manage to get rid of it. So now, it’s much easier to work with Heroku. I’ve added authentication to our server and I tried to add links between the tables in our database. But I faced several problems, some due to the facts that I didn’t wrote all the code (Tanguy is also working on the server) and that I assumed things that happened to be different (for instance, we use a Postgre database on Heroku but a SQLite one for Continuous Integration because Tanguy has the impression that it’s not possible to install a Postgre database in the test environment).
As working on almost the same thing at the same time seems to be a rather bad solution, I will probably start to work on the Wi-Fi module and let Tanguy continue his work on the server. So by the end of the week, I will learn how ZentriOS works and start to write the driver that will control the Wi-Fi module.
I didn’t do a lot of things on the project since last Wednesday.
Nonetheless, I managed to make the sotware audio codec work!!! Now, I’m able to stream whatever mp3 file on the serial link, decode it on the micro-processor, send it to the codec and hear it in my earphones. I don’t really know what were the errors in my previous code, but now it works and it’s the most important. We have our first completely finished driver, and that’s a good thing as it doesn’t remain a lot of time before the final deadline.
I also had a look to the server. I managed to deploy the database on heroku (a PostgreSQL database was required), but it has a very strange behaviour: if you read it several times, you don’t always have the same result, and the result also depends on the device you use to access the website. So it’ll be necessary to have a look to it. I also read some documentation about Pyramid to really know how it works (I hate doing something without knowing what I do). Finally, I spent some time looking for a way to add security to our server (authentication and authorization). I found one library that allow the users to log in with their Google account, but it doesn’t work with Python 3.x (only Python 2.x). So I’ll continue to look for a better solution.
And our PCB will be sent to the factory tomorrow evening. We must receive them 6 working days later, which is a good news as it will allow us to work on it during the Christmas holidays.
During next week, I will focus my efforts on the server, in order to have it finished when we receive the PCB.
As said in my last post, I finished the routing of the PCB. Alexis made some reviews, and I did the corresponding corrections. It should be ok now, and the PCB will be sent to the printing factory soon.
I also (it occupied most of my time on Monday and Tuesday in fact) worked on the audio codec (again). You’re possibly fed up with it, and I understand you. I’d never thought it would be so difficult to make it work. But everyday, I go one step further, so I can’t abandon it (and I really want it to work, so I won’t abandon). Now, I’m able to play any music stored in flash when it’s presented as a single block to the software codec. And I can play one music (the one I’ve used for my tests until now) even when it’s read by chunks (and on Tuesday morning, I could send it through the serial link and play it, but on Tuesday afternoon, it didn’t work anymore, don’t know why). But it fails when I try with other musics… I will continue to work on it.
Finally, today, I deployed our (basic) server on Heroku. I spent some times trying to understand how to use
git subrepo. And it’s still not completely clear in my minds, even if it seems to work. The database doesn’t work for now on heroku (due to an error caused by waitress), a deeper analysis will be required to fix this problem. And I also wrote a template html file to display the content of the database (the leds values).
During the end of the week, I’ll try to work on the audio codec to solve the last problems. And I will work on the server also (to make pauses between 2 work sessions on the audio). If both work well on Sunday evening, I’ll be very happy.
As said in my last post, we worked on the PCB, on the sound and on the server. But contrary to what was said, it’s Tanguy who worked on the server and I worked on the PCB. On Friday, I made some minor changes on the placement and then I asked for a review from Alexis. He helped us to improve the placement and validated most of the choices we’ve made. After having applied his suggestions, I started to route the PCB. It should have taken no more than 30 minutes but I spent more than 3 hours and it’s still not finished… I hope that it doesn’t hide a bigger problem in our PCB.
I also talked with Samuel about my problems with the sound and he suggested me to use mailboxes. On Friday evening, I implemented this idea and now I have another problem: I can hear the music but also many other things… I have to investigate to see where this problem comes from (it’s related with what I send to the codec I think, so I’ll have a look on this aspect first).
For next week, I will continue to work on the audio and on the PCB. And maybe on the server also. My objective is that the PCB is ready before Wednesday and that the sound and the server work fine before the end of the week.