Categories

[AmpeROSE] ChibiOS – Sd card

Hello,

In the last two weeks, I was working on the backend, specifically on Sd card. I’m developing code and testing it on the same board that we used during TP: Olimex STM32-E407” before migrating to our real board.
I started by setting up ChibiOS and made some very simple tests (blinking the user-led, etc) to ensure the good control of the board. Then, started the real development.

ChibiOS.17.6.3 integrates fatfs (developed by Chan), an API that offers a very good level of abstraction when communicating with Sd card. It offers a lot of functions, for example f_mount, f_open, f_close, f_write, f_read etc, and it is actually well documented. Before using the Sd card, it must be formatted to fatfs (fat32 is the default one).

Besides the code of Sd card (API Calls, debug info,..), I developed six different tests, each one had a very specific objective. Let’s dive into them…

Test1: Unit test

It does the following:
– create a thread that will wait for the SD card ready state
– the thread will do the following:
* open file: test1.txt in mode rwa(read, write, append)
* write a str defined (5 times)
* write a single character (5 times)
* read a string
* write ROSE using write_sd function (buff)
* read ROSE using read_sd function (buff)
* if an error is detected during write, the thread immediately terminates (ex of errors: disk full, file full, …)
* close the file

It is a very simple unit test, that gave me the control over the Sd card.

Test2: Sd card delay characterizer – write_sd

It consists of a producer/consumer model. A timer (producer) generates data (an msg of 32 bits) and posts it in the consumer (a thread) mailbox (queue, buffer) which fetches each time there is a new message in its mailbox and write it to the Sd card using buffer mode (f_write). In the end of this test, min, max and the average of writing to the Sd card are shown.

Test3: Sd card delay characterizer – write_str_sd

Same as test1, but the goal is to characterize the delay of writing a string.

 

So far, writing each msg alone took a lot of time. Of course, it is not the optimal way to do that but I wanted to see the results to get an idea of the performance then try to optimize. Understand how the Sd card operates in write mode is essential to try to optimize this delay.

In a nutshell, writing to Sd card is read-modify-write of the smallest unit which is the sector (512 bytes most commonly used value). So when initiating a write operation, the Sd card fetches a sector into RAM, modify it with respect to the data that should be written then flush it back to the Sd card (when calling f_flush or when closing the file). So one way to improve the performance is to initiate a write operation with respect to a buffer that has the size of a sector, which brings us to Test4.

Test4: Sd card delay characterizer – write_sd bulk mode

Same as test2 and 3, but instead of writing each message alone, the messages are buffered internally and a write operation is initiated each time 512 bytes were buffered. The improvement was pretty good (the results will be shown at the end of the post).

Test5: Simulation1

AmpeROSE operates in two modes: streaming mode and the standalone mode where the Sd card is actually used to store data. The producer’s frequency is set to 100us and the simulation is run during 10s (default). This test is important because it gives us an idea of the ~minimum mailbox size in order to process all msg and discard nothing. (producer is much faster than the consumer => data might be discarded if there is no place to put it). The result was pretty good, the consumer was able to handle all of the messages without any loss.

Test6: Simulation2

In reality, the max sampling frequency is 100K (10us). This test implements the producer/consumer model with respect to this value using a general purpose timer.

You may be wondering the use of a GPT instead of a virtual timer which is easier and faster in implementation. The is due to the VT limitations. Internally, the VT uses the systick to generate the interruption, the freq of the systick is set to 10K which gives an interruption each 100us. Even when specifying a period of 10us of a VT, internally it is rounded up to the nearest value which is in this case 100us. A possible way to use a VT, in this case, would be to tune the systick variable up, but it is discouraged to do that.

Therefore a GPT is indispensable to implement the real 10us. The result is not good, ~50% of data were lost even by allocating the maximum available size on the Olimex board. In the outputs, you will notice how the percentage of loss decreases when the size of the buffer increases.

And before showing you the results of the different tests, I also added a configuration.h file which gives us the ability to choose between 3 modes (DEBUG, TEST and SHELL modes).
These 3 modes make the development process much easier and by the way, I use SEGGER Real-time coz it is much easier to use by comparing to the shell.

However, in production mode, debugging and testing are not important, in fact, their use will impact the memory space and the overall performance of the system. Therefore, this configuration file permits to disable them in a very practical way.

Tests results

ChibiOS/RTOS initialization

SD connected!
Mounting FS…
FS mounted!
******** Beginning of Tests ********
******** Test1 (unit tests) ********

–> Test1.a: write a string and a character to SD card
File opened
Str successfully written
Chr successfully written
Str successfully written
Chr successfully written
Str successfully written
Chr successfully written
Str successfully written
Chr successfully written
Str successfully written
Chr successfully written
File closed

–> Test1.b: read a string from SD card
File opened
Reading a str from SD card…
Str successfully read
Read the following str: ROSE 2018: AmpeROSE

Reading a str from SD card…
Str successfully read
Read the following str: WROSE 2018: AmpeROSE

Reading a str from SD card…
Str successfully read
Read the following str: WROSE 2018: AmpeROSE

Reading a str from SD card…
Str successfully read
Read the following str: WROSE 2018: AmpeROSE

Reading a str from SD card…
Str successfully read
Read the following str: WROSE 2018: AmpeROSE

File closed

–> Test1.c: write a bunch of bytes to SD card
File opened
Written with success
File closed

–> Test1.d: read a bunch of bytes from SD card
File opened
Read with success
Buff[i (1->4) ]: R O S E
File closed

******* Test2 (Word 32bit) ********

File opened
Please wait…

Statistics about this test
* Buffer size = 3000 (96Kb)
* Generated samples = 2048
* Processed samples = 2048
* Discarded samples = 0
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 2048
==> Min delay (us) = 3000
==> Max delay (us) = 46000
==> Average delay (us) = 4683
==> Total delay (us) = 9591300

File closed

******** Test3 (String) ********

File opened
Please wait…

Statistics about this test
* Processed messages = 2048
* Discarded messages = 0
* About write operation to SD card:
–> Number of writeSD operation = 2048
==> Min delay (us) = 3100
==> Max delay (us) = 54000
==> Average delay (us) = 4619
==> Total delay (us) = 9461300

File closed

******** Test4 (Block 512B) ********

File opened
Please wait…

Statistics about this test
* Buffer size = 3000 (96Kb)
* Generated samples = 2048
* Processed samples = 2048
* Discarded samples = 0
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 16
==> Min delay (us) = 3000
==> Max delay (us) = 21000
==> Average delay (us) = 6175
==> Total delay (us) = 98800

File closed

******** Test5 (Simulation1 – Word) ********

File opened
Please wait…
Time is up, stopping simulation…
File closed

Statistics about this simulation (0min) (3s)
*Total time of execution (0min) (50s)
* Buffer size = 10000 (320Kb)
* Generated samples (by ADC) = 15000
* Processed samples = 10649
* Discarded samples = 4351
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 10649
==> Min delay (us) = 3000
==> Max delay (us) = 94000
==> Average delay (us) = 4619
==> Total delay (us) = 49194100

******** Test6 (Simulation2 – Bulk) ********

File opened
Please wait…
File closed

Statistics about this simulation (0min) (1s)
* Buffer size = 2048 (65Kb)
* Generated samples (by ADC) = 100000
* Processed samples (written) = 27904
* Discarded samples = 72096
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 218
==> Min delay (us) = 3000
==> Max delay (us) = 37000
==> Average delay (us) = 4872
==> Total delay (us) = 1062100

No more tests to run

******** End of Tests ********

ChibiOS/RTOS initialization

SD connected!
Mounting FS…
FS mounted!
******** Beginning of Tests ********
******** Test6 (Simulation2 – Bulk) ********

File opened
Please wait…
File closed

Statistics about this simulation (0min) (1s)
* Buffer size = 4096 (131Kb)
* Generated samples (by ADC) = 100000
* Processed samples (written) = 30208
* Discarded samples = 69792
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 236
==> Min delay (us) = 3000
==> Max delay (us) = 33000
==> Average delay (us) = 4726
==> Total delay (us) = 1115500

No more tests to run

******** End of Tests ********

ChibiOS/RTOS initialization

SD connected!
Mounting FS…
FS mounted!
******** Beginning of Tests ********
******** Test6 (Simulation2 – Bulk) ********

File opened
Please wait…
File closed

Statistics about this simulation (0min) (1s)
* Buffer size = 10000 (320Kb)
* Generated samples (by ADC) = 100000
* Processed samples (written) = 35984
* Discarded samples = 64016
* Remaining samples (in buffer) = 0
* About write operation to SD card:
–> Number of writeSD operation = 282
==> Min delay (us) = 2000
==> Max delay (us) = 39000
==> Average delay (us) = 4779
==> Total delay (us) = 1347700

No more tests to run

******** End of Tests ********

That’s all.

Bilal ADDAM

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>