Last Wednesday, we finished discussing about architecture. We made some major changes.
First, we discovered that we couldn’t use the TLC59582. We made some assumptions about the way ES-PWM works that aren’t specified in the Datasheet. We thought we could control the PWM width over one segment using the 8 MSB, and we could send a VSYNC signal in the middle of the PWM period. This would have resulted in an 8bit PWM, which is ideal. Without these assumptions, we only have a 12bit PWM which period is too long compared to the display time of a single point.
After checking a lot of LED drivers and PWM generators, I couldn’t find anything suitable except for the TLC5957. Most drivers are either using 12-bits PWM, or using an 8-bits PWM but with really insufficient data rates. The TLC5957 has the advantage of being configurable from 9 to 14 bits (resulting in small enough PWM period when used with 9 or 10 bits), and with a really high data rate.
We also had to think about how to configure the Wifi. After thinking about a bunch of scenarii, we went with the simplest solution since we are running late. We will use a button on the mobile part used to turn the Wifi module into AP mode. It will also start a web server that can be used to configure the Wifi AP to connect to.
Finally, and again for sake of simplicity, we decided that the fixed and mobile parts will not communicate since it’s not strictly necessary. The fixed part will use a switch to start and stop the motor, and a photosensor to get feedback on the motor’s speed. Since the motor is controlled with 5V logics, and we do not longer need Wifi or Bluetooth, we will use an Arduino Micro instead of the ESP32. It can be powered using 12V, provides a regulated 5V/1A pin, and PWM capable 5V GPIOs.
I’m beginning to work on the schematics. For know, I designed one LED panel including its driver and column multiplexing.
We are taking the same mechanical parts as last year’s project. Now we are trying to get how much currennt the motor pulls in order to get a proper estimation of the consumption.
I ran into a problem while testing the motor though. I don’t know how to use the motor controller (which is a Ezrun max 10 SCT). So I went looking for the documentation and found the very extensive 🙂 doc which just tells us to connect a receiver and everything should work fine. On our side we are trying to drive it with an arduino or something alike.
So far I know that the controller is driven by a 5V PWM, but I haven’t been able to make it work. So I would love to know how last year’s group made it work.
For the past week, we have been discussing a lot the components we will have. We mainly focused on the power supply, our photosensor(s), our Wi-Fi module(s) and our LED driver.
For more information about our hardware decisions, you may look at Ambroise, Guillaume and Baptiste’s posts.
In my last post, I showed you how I created a vtkUnstructuredGrid that fits our display system. The past week, I managed to extract geometrical data and color from the meshes in Blender and I tried a lot of different ways to fit the mesh geometrical data into my grid representing our cylinder.
The voxelizing algorithm takes in input a list of colored meshes and outputs an image that is the slices of the scene containing the meshes.
As we have a 40×30 LED panel and 256 steps per rotation, I decided to output one frame as a 1200×256 image, where every row is a different slice of our scene and the 1200 pixels in each row represent the RGB components of the LEDs, going from left to right, top to bottom. This image is currently saved as .bmp and .raw (which is basically BMP without the header) files.
I created multiple scenes to test out our algorithm. Below, you can see the result of voxelizing different kind of meshes. The first picture will be the model of the mesh I tried to voxelize. The second one will be the output of the algorithm, which is the 1200×256 reprensenting the LED configurations at each angle. The third one is how the LED configurations should appear to our eyes with our real system (it is a simulator, written by Guillaume using Processing — it works pretty well!)
As we can see, our voxelizing algorithm is not perfect but we can easily recognize the original meshes. Let’s see the pros and cons of the current algorithm.
A cylinder of diameter 4:3 and a height of 1 is perfectly mapped to our system.
We can easily recognize shapes and texts although it does not feel perfectly aligned.
The raw output is well-fit for our system : reading sequentially the output file gives the different slices of a frame, in the correct order.
The algorithm can basically voxelize any colored mesh scene.
Blender has an animation framework and it should not be too much work to make a video with our algorithm : by simply voxelizing every frame and displaying them one after the other, we can get a 3D video.
The algorithm is slow for now (1 second for a simple cube, about 1 minute for a scene with 500 polygons) but it is written in Python. It is not really optimized for now as I did this for prototyping but it can be rewritten in C and we can have Blender call the C program instead of having it to execute the whole algorithm in Python. On top of that, it can be rather easily multi-threaded. Indeed, the treatment of each slice is independent from the others.
The complexity of the algorithm is linear with the number of polygons in the scene, making the voxelization of complex scenes in real time quite complicated.
Straight lines aliasing can be seen when the faces are displayed on a big radius. But this is rather a resolution problem than an algorithm problem and there’s not too much we can do about it. If we have a big quad, it is best to show it in the middle of the scene and with a small scale as this is where the LED density is the biggest and there is not much aliasing.
There is some noise in our sphere voxelization – but perhaps it is due to the fact that the sphere modeled in blender is an UV-sphere with not that many faces.
File size and compression
Ultimately, the frames or the video will be either put on the flash of our system or sent/streamed through the Wi-Fi. In such cases, file size is important. Here our models are pretty simple, but having a lot of unlit LEDs makes the output image of the algorithm a sparse image. When you have sparse data, it is easy to compress.
Below, you can see how much our out images are compressible. I used gzip -c6 to compress the raw files and see how much they could be compressed. Here are the results :
As we can see, the compressed file is about 1% of the raw file. On top of that, gzip is quite easy to use (there are portable libs online of about 200 lines of code and this definitely runs on a Cortex-A9) – fast (and the speed/compression ratio can be set) – and efficient.
I do not expect to see a 100:1 compression ratio on every scene I could voxelize but it is rather comforting to see that we can ultimately use compression in our system if needed.
If you have any idea on how we could improve our results or any feedback to give, please comment.
For the rotation part, we were thinking of an IR LED on the fixed part, and an IR receiver on the mobile part. It turns out there are transmissive photosensors which are exactly what we need: an emitter and a receiver, isolated from outside world except for a small hole. The photosensor will detect when something passes through that hole. We needed the component to be easily placeable on the PCB, and with the right orientation. It turns out it was not that easy since most photosensors are either SMD or oriented in the wrong way. We managed to find a sensor with the 3 possible orientations, fixed by screws, and connected with wires: Omron EE-SX3164-P2.
We think we are going to use multiple “spikes” on the fixed part. This way, we will have a finer granularity to synchronize LED driving.
We also needed to choose components for the multiplexing. Multiplexing each column will be done using a PMOS since our LEDs are high-side driven. Moreover, LEDs need to be powered with ~5V (due to the voltage taken by internal LED driver circuitry). Providing that we use 3.3V logic, we need a MOSFET driver to generate a signal capable of toggling a PMOS for which Vds = 5V.
I used the TI design note to see what components they were using. It turns out both there PMOS and driver where adapted to our situation (3.3V logic, 5V for LEDs, 5A maximum sink current, and quick commutations). This is why we are going to use the ISL55110 driver and the SI2333CDS (simple) or SI4953ADY (double) mosfet.
The LED drivers will be directly connected to the FPGA which will be in charge of the multiplexing logic.
I’ve also been thinking about Wifi. We want to process data in the Cyclone V HPS. It has the advantage of being fast and directly connected to the FPGA and shared RAM (which we will be using for buffering).
Since we need to support quite high data-rates, there are only two HPS interfaces we could use: SDIO and USB. Since SDIO is already connected to eMMC on the SoM, only USB is left.
I’ve been looking at linux friendly 802.11ac module since it can handle greater throughputs. There are 3 drivers available.
Broadcomm’s. I couldn’t find a place to easily buy their ICs so it seems it’s reserved for business buyers.
Intel’s. All of there cards communicate through PCIe but we don’t have a PCIe interface on the SoM. There are variants of the SoM with PCIe but still we want to avoid extra complexity.
Realtek’s. Feedback is that the driver is buggy and unstable. Again, unnecessary complexity.
For now, we think it’s better to stick to 802.11n and reduce the resolution or depth of our panel if needed. We’ve been suggested to use Acmesystem’s WIFI-2. It is a Wifi module based on a Realtek IC. It is compatible with USB IF so it can work out-of-the-box using default linux drivers.
We have all our “logical” components. We can then finish by choosing power components, now that we are sure of our needs.
While Paul worked on producing images that are going to be displayed on our system, I used Processing to create a simulation of the rotating LED pannel. Processing is a Java based language made for easy visualisations. My simulation is able to load an image and then works simillarly to the LED pannel. It displays a slice, rotate a farction of an angle and displays the next one. Even if it is only showing a single image for now, it should be straightforward to change it to show a video.
As we are using last year base, I dismanteled their project to find the inner working of their project. We knew that the power was transmited the revolution axis, but we were not sure how the connexion was made.
It turns out that a wire is connected to the top of the case of the motor, so to go through the axis the current needs to travel first through a ballbearing. We then learned that it did not make a perfect connection so the voltage would be unstasble. To counteract this problem we will use capacitors and also raise the voltage going to the rotating part in order to lower the current.
I looked for the power required by the System on Module (SoM) we are going to use (MCV-6DB). We are using the LVCMOS 3.3V standard, so every FPGA bank will be supplied 3.3V. According to the Wikipedia page of another SoM manufacturer, the maximal power consumption is 8.5W for a very similar FPGA (5CSXFC6) to the one we will be using (5CSEBA6). With a supply of 3,3V, we then have 2.58A. This result doesn’t include the power consumption of the peripherals included in the SoM. With the Power Calculator provided by Micron, and using the DDR3 datasheet (K4B4G1646D), I find that a single module of 4Gb will not exceed 300mW. So 600mW for the two of them. The eMMC (MTFC4GLDDQ-4M IT) has a typical current consumption of 70mA when active, according to Micron documents (link).
Compared to our last posts, we have updated our architecture. We realized that the only SDIO capable bank off the FPGA was used internally on the SoM to connect the HPS to the eMMC. This means that we can’t use SDIO to communicate with an SD card at high speeds. Fortunately, we won’t need high speeds from the SD card since we will be able to take our time at the start of the system in order to transfer the data to the much faster eMMC or DDR. So we will connect the SD card to the HPS through SPI.
We wanted to supply the mobile part with 5V which could then be directly used by some of the components (LED and synchronization mechanism for example), and converted to 3.3V for the rest of them (SoM, LED driver, …). However, it appears that the voltage will be subject to instabilities because of the way the power is transmitted through the rotating part. We have to account for that by using large capacitors as well as a higher supply voltage: at least 12V. So we will have to convert 12V to 5V and 3.3V. We are not sure if the conversions 12V->5V and 12V->3.3V are better than 12V->5V->3.3V. In the latter case we would use the component we originally considered : PTH05060W from TI (under review by Alexis).
Over this week-end, we finally agreed upon how we were going to deal with our big data throughput.
At first we wanted to take a MCU, but we noticed that if we use Wi-Fi and stream the data to our system, we needed to :
Have a reliable Wi-Fi module that works properly in an environement where other 2.4Ghz frequencies are normally used. For instance, People usually carry their phones with Bluetooth a Wi-Fi connection active and this should not break our system.
Be able to buffer the data before displaying as the Wi-Fi latency may vary. Without an external memory, an MCU cannot absorb much more than a few milliseconds of jitter.
Thus, we decided to take a System-On-Module instead. Ambroise talks about it in this post.
With now an FPGA, a dual-core ARM processor with FPU, embedded linux and 1GB of RAM, our system will be rather powerful. Thus, we are considering adding some drawing primitives directly on our system and not rely entirely on the computer to stream all the raw data.
Last time, I explained how I tried to make Blender work with VTK. I managed to voxelize a mesh and view every single voxel in the FIJI Visualization software. Unfortunately, the voxelization is done with a regular cubical grid, which is not what we want.
I succeeded into making a grid that represents our system : a cylinder which represents our 40×30 screen rotating on its Z-axis. On the cylindrical grid below, every white cell is meant to represent a LED on one of the 128 steps.
Now, I am trying to fill this grid with our mesh color data. Then, we simply need to extract each slice of the cylinder to know our LED configurations on each step.
Following my previous post regarding the choice of an FPGA, we found out that Cyclone V models only come in BGA format, which is very impractical for us to solder on our PCB. I focused my research on System on Modules (SoM), which have the advantage of providing us with an easier pin configuration to solder as well as an already built system around the FPGA.
In order to ensure that the variations in latency over WiFi (up to several dozens of ms according to our measurements) will not compromise the display of the frames, we have to consider including more memory to our system. With a 24-bit color depth and a 30Hz refresh rate, we would need more than the 4,460kb embedded memory on the 5CEBA5 if we want to account for a 50ms spike in latency. Given that our final presentation will most likely be done in a WiFi-saturated environment, we have to plan for more memory.
The Aries MCV series include a Cyclone V SE with a cortex A9 and 1GB of DDR3. We would use the CPU with Linux running on it, which would allow us to use a WiFi over SDIO module. The 1GB of DDR3 will giveus more than enough buffering capacity. It would be connected to our PCB by two qsh-090-01-f-d-a connectors positioned under the SoM.
Among the Aries products, I think the MCV-6DB would be the best for us because it keeps the same FPGA as the one I listed in my previous post.
As we wrap up our components list, I decided to look into the kind of data that we will display on our system.
Our system will probably display either :
A mesh that can be made in a 3D modeling software such as Blender
Medical or scientifical 3D data, such as a CT-scan or a geophysical map.
VTK is an open-source software system for 3D computer graphics, image processing and visualization. VTK can manage voxels pretty well while Blender only has a very low support for volumetric data. VTK is developped in C/C++ but has wrappers for Python, which means we can use it in Blender.
Someone has already made a VTKBlender module to make blender work with VTK, so I decided to use it. It is available on GitHub.
Even with VTKBlender, it was quite a pain to make Blender 2.79, Python 3.7 and VTK 8.11 work.
On this paper, I found the source code of an old Blender plugin that worked with Python 2, VTK 5 and Blender 2.49. Unfortunately, quite some code is not compatible with current versions so I am upgrading the source code.
During this week we have mostly focused on determining the architecture of our system. I have tried to look at the different power comsumption, and the solution to supply energy to our system.
Using the same structure as last year’s project, wehave decided not to change the flow of energy. No power supply will be carried on the rotating part. We will use the axis to carry the current. From the bottom part, we will input power, voltage to be determined. The top will be grounded, as will the whole structure.
There is a lot of energy to be supplied to the system. First of all the 1200 LED, then there are the controllers, the fpga, esp32, and many more power sucking devices to be determined. But for now all that we seem to bring on the rotative pannel seem to work on low voltage, 5V or lower. The problem resides in all the current our component will drain. For now I’m thinking about hacking a computer alimentation bloc. Some give out 500W (which might be overkill) in three different volatges, 12V, 5V and 3.3V. 3.3V might not be of use but 12V can be use to power the motor underneath the pannel.
There is still much researched to be done, we will focus on that this week, having a set list of components so that our project can really take of.