Categories

[SpiROSE] Last few days, close to the end, but we have finished the FPGA’s modules !

Hey, last week  I was quite pessimistic because of the time we had left for the testing, verifications and calibration of our hardware. It seems that this is almost fixed ! We are a little more than two days before the end and the three cards have been done — almost at 100%. There is still an unknown issue with one of the voltage regulator, which allegedly made it burn, but it might run well tomorrow.

We managed to implement the whole new FPGA architecture that Adrien described in today’s post and add a bunch of new SystemC tests. We made a global pinout assignment with quartus TCL definition file for the EP3C40 FPGA and a lot of different testbench for both EP3C40 and DE1_SoC targets so as to exhaustively test every part of the project on the hardware. Tomorrow morning I will finish one of these part which will test if our solutions for synchronizing the RGB input stream with the display will work correctly. The goal is to make a full RGB cylinder in 3D as input for the RGB logic, simulate the rotation and extract only one slice position. If the panel was to change color, we would know it doesn’t work.

Finally, it will probably be a happy ending, or else we can still show that we know how to blink LED.

[SpiROSE] Last updates on the hardware, happy new year !

It has been time since the last update, I hope you had a good time with the New Year and holidays. Happy new year to everyone and I hope the best for every other ROSE project.

Since the last time I posted, we managed to finish the most important subjects in our work:

  • The last components of our projects have been chosen.
  • The hardware part is (at last) completely done.

MOSFET Driver

First, about the components, we had to choose a mosfet driver so as to have good switching timing. Indeed, we want to overdrive the LEDs so they are visible even if we don’t light them long. So we have to be very precise on the

However, the LEDs are connected to the +5V by their common anode, and to the mass through the LED driver. As we don’t have much place, we have to put the mosfet to the bottom of the LED panel, so it has to connect the LEDs to the +5V. Thus, we have no choice but using P-channel high side mosfet. Difficulties stemmed from this choice.

Actually, P-mosfet are bigger than N-mosfet, and as described by Vincent last time it has been difficult to put them on the PCB. But finding a P-channel high side mosfet driver is even worse. We finally end up with the LTC1693-5 from Linear Technology. It can drive a single channel but we can safely put eight MOSFET on each channel and still meet the timing requirement. It eases the wiring as we light eight columns at a time during a multiplexing step.

Basically, it was the only one MOSFET driver we found within our criteria, so there was no room for choice.

Card-to-card connectors

Then we had to find connectors between our cards. Our first thoughts went to single row 28-pin connectors with a spacing of 1,27mm between pins, with male ones being on the rotative base and female ones on the LEDs panel. However it failed to be routable because it put the flat angle brackets 0.85cm too far from the center, making the LEDs panel bigger than 20cm, too big to go to a pick&place machine. We hereby swapped them with dual-row 1,27mm 28-pin connectors (so 14-pin long) after computing that dual-row 2mm 28-pin connectors weren’t enough and failing to find smaller connectors.

What next ?

Next we will have to reorganize our PSSC for the software, as we noticed before the holidays that most of the time they were either obsolete or overlapped: one critera would ask the work for two and both would be validated at the same time, so there was no use of having them on their previous state. However most of the new plan has been done and accepted by the teacher just before the holidays.

We also have to finish SystemC tests for SPI and framebuffer, implement end-to-end SystemC tests (from SBC output to FPGA output).

We will probably not have the PCB before the end of the project but at least we will be able to show it for the final presentation, so software has to be tested enough so that we can focus on the hardware problems as soon as we receive them.

[SpiROSE] Testing systemverilog with SystemC, final round

Among the tasks I’ve been busy with these weeks, there is the making of integration tests for the FPGA’s SystemVerilog code.

SystemC is a very good asset to put in a CI environment. Basically, its advantages are:

  • It is quite light and free.
  • You only need tools to compile C++ code in your CI, instead of having modelsim or other full simulator.
  • You can customize the output to your needs, although formatting with SystemC with only 80 characters a line is quite awful.
  • You can use other library more instinctively.
  • You can even get VCD files back from the CI.
  • The compilation files are almost usual makefiles and testbenchs are C++ (sc_)main.

But eventually, we want to use this tool against the SystemVerilog code, so we need a bridge between them.

The situation

To put it clear, I will describe the situation for the driver_controller module. This module is the one generating control and data signal to the drivers. Amid these signals there are:

  • The GCLK and SCLK signals, both are clocks.
  • The LAT signal, which has strong timing requirements. It has different possible duration at the high state, each of these associated with a different command in the driver.
  • The SIN signal, which is the input of the driver itself

Before using the driver, it has to be configured and its the role of the driver_controller to do it too.

The inputs of the driver controller are first two clocks: clk_hse running at 66MHz and clk_lse running at 33MHz. This last one is generated thanks to the clk_hse clock in the clk_lse module, that we will need to include in the SystemC testbench too.

Then we have a framebuffer_sync signal, which should be generated each time we start a new frame. Finally there is a framebuffer_data signal of size 30bits, giving the SIN for each driver.

The tools

To achieve this, we will need to translate our SystemVerilog code into SystemC modules. It is time for the Verilator tool to enter the scene.

This tool was first included as a linter into our project, but revealed to be more useful than first thought. It is capable to translate SystemVerilog code into either special C++ to use with verilator, or SystemC modules which are a bit slower than the former but far easier to use.

Our little clock module becomes the following beautiful SC_MODULE:

SC_MODULE(Vclock_lse) {
    public:

    // PORTS
    // The application code writes and reads these signals to
    // propagate new values into/out from the Verilated model.
    sc_in<bool> clk_hse;
    sc_in<bool> nrst;
    sc_out<bool> clk_lse;

    // LOCAL SIGNALS
    // Internals; generally not touched by application code

    // LOCAL VARIABLES
    // Internals; generally not touched by application code
    VL_SIG8(__Vcellinp__clock_lse__nrst,0,0);
    VL_SIG8(__Vcellinp__clock_lse__clk_hse,0,0);
    VL_SIG8(__Vcellout__clock_lse__clk_lse,0,0);
    VL_SIG8(__VinpClk__TOP____Vcellinp__clock_lse__nrst,0,0);
    VL_SIG8(__Vclklast__TOP____Vcellinp__clock_lse__clk_hse,0,0);
    VL_SIG8(__Vclklast__TOP____VinpClk__TOP____Vcellinp__clock_lse__nrst,0,0);
    VL_SIG8(__Vchglast__TOP____Vcellinp__clock_lse__nrst,0,0);
    VL_SIG(__Vm_traceActivity,31,0);

    // INTERNAL VARIABLES
    // Internals; generally not touched by application code
    Vclock_lse__Syms* __VlSymsp; // Symbol table

    // ...
};

 

Verilator will generate its files in an obj_dir/ directory, which will be in the sim/ directory.

It will generate Vmodule_name.{h,cpp} file, containing the module itself and glue code to go with verilator library, Vmodule_name__Syms.{h,cpp} files making it available in the verilator library and some Vmodule_name__Trace.{h,cpp} files.

For the simulation, you only need to link against Vmodule_name.o and Vmodule_name__Syms.o. You can add the Vmodule_name__Trace.o file if you want to get the trace written into a VCD file, but you’ll need to have a deeper look into Verilated, the Verilator library, and especially the VerilatedVCD object to make it work.

The methodology

 

The first idea with testing module in SystemC is to put all the testing code in the sc_main function, where you are able to control the progress of the simulation :

sc_main(int argc, char**argv) {
    sc_time T(33, SC_NS);
    sc_clock clk(T);
    sc_signal<bool> nrst(“nrst”);
    // create the IN/OUT signal for the DUT
    Vmodule_name dut(“module_name”);
    dut.clk(clk);
    dut.nrst(nrst);
    // bind the IN/OUT signal of the DUT
    while(sc_time_stamp() < sc_time(10, SC_MS)) {
        // advance the simulation of T
        sc_start(T);
        // do your tests
    }
}

 

However it looks like software testing and it is usually a bad idea because you won’t be able to describe every interaction at the module scale you want. In our context within the driver_controller module, this drawback is visible and won’t allow us to do correct testing.

 

Instead, we write a Monitor SystemC module, which will interact with the DUT and run the tests as SystemC threads.

 
SC_MODULE(Monitor) {
    SC_CTOR(Monitor) {
        SC_THREAD(run_test_1);
        SC_THREAD(run_test_2);
    }

    void run_test_1() {}
    void run_test_2() {}

    sc_in<bool> clk;
    sc_in<bool> nrst;
    // create sc_out for output signal to the DUT
    // create sc_in for input signal from the DUT
};

I will explain later how we benefit from this in the different tests.

The compilation

 

Now we have to compile the SystemC code into an executable simulation. We will want to recompile SystemVerilog into C++ each time there is a change and write makefile as small as possible for each testbench.

 

In the FPGA/ directory, we currently have the following structure:

  • src/ : containing the SystemVerilog code.
    • systemc/ : containing some SystemC modules, including the driver model we developed.
  • tb_src/ :
    • systemc/ : containing the SystemVerilog testbench we developed in SystemC.
  • sim/ : containing what’s needed to generate the simulation.
    • Makefile: will be the entrypoint to launch tests
    • base_testbench.mk: base makefile from which will inherit the others
    • module_name.mk: makefile for the module_name testbench

 

In Makefile, we will have a variable listing the different module we want to test. It will serve as generator to create FORCE-like tasks to generate and launch the simulation.

 

What we want for the module_name.mk is the following:

 
MODULE := driver_controller
DEPS := clock_lse

ROOT = tb_$(MODULE)
OBJS += $(ROOT)/main.o $(ROOT)/monitor.o driver.o driver_cmd.o

-include base_testbench.mk

$(MODULE).simu: $(OBJS)
    $(LINK.o) $(OBJS) $(LOADLIBES) $(LIBS) $(TARGET_ARCH) -o $@

I’ve currently put the target to generate the simulation in this makefile but it could have been in the base_testbench.mk too. I just feel that it has more sense in the module_name.mk file.

 

MODULE is the device under test and DEPS is a list of SystemVerilog modules needed to build the testbench.

 

Then, the base_testbench will first define variables describing the environment:

export SYSTEMC_INCLUDE ?= /usr/include/
export SYSTEMC_LIBDIR ?= /usr/lib/

SV_MODULE_PATH = ../src

VERILATOR = verilator
VERILATOR_ROOT ?= /usr/share/verilator/include/
VERILATOR_FLAGS = --sc --trace
VERILATOR_BASE = verilated.o verilated_vcd_c.o verilated_vcd_sc.o

Then it will create objects list and dependency list:

 
VERILATOR_OBJS = \
    obj_dir/V$(MODULE).o obj_dir/V$(MODULE)__Syms.o \
    $(patsubst %,obj_dir/V%.o,$(DEPS)) $(patsubst %,obj_dir/V%__Syms.o,$(DEPS))
OBJS += $(VERILATOR_BASE) $(VERILATOR_OBJS)
DEPSFILES = $(subst .o,.d,$(OBJS))

all: $(MODULE).simu

Then it defines the compilation options:

LDFLAGS = -L$(SYSTEMC_LIBDIR)
LINK.o = g++ $(LDFLAGS) $(TARGET_ARCH)
LIBS = -lsystemc
VPATH = ../src/ ../src/systemc ../tb_src/ ../tb_src/systemc ./obj_dir/ $(VERILATOR_ROOT) ../lib
CPPFLAGS = -I../src/systemc/ \
    -I./obj_dir/ \
    -I../lib/ \
    -I../tb_src/systemc/ \
    -I$(VERILATOR_ROOT) \
    -I$(SYSTEMC_INCLUDE)
CXXFLAGS = -DSC_INCLUDE_DYNAMIC_PROCESSES -g

And finally defines the compilation target to rebuild Verilator files or handle dependencies:

obj_dir/V%.cpp obj_dir/V%__Syms.cpp obj_dir/V%.h: %.sv
    $(VERILATOR) $(VERILATOR_FLAGS) -y $(SV_MODULE_PATH) $<

%.d: %.cpp
    $(CXX) $(CPPFLAGS) $(CXXFLAGS) $(TARGET_ARCH) -MM -MP $^ -MF $@

clean:
    rm -rf $(DEPSFILES) $(OBJS) $(MODULE).simu

-include $(DEPSFILES)

Issues

 

What has been done currently doesn’t fulfill our requirements yet. It seems that even if dependency files are correctly generated and included, the simulation won’t recompile if header files are modified. As we are prioritizing the development of the hardware at the moment, we’re not trying to debug it more. But this issue should be solved by the end of the year to make the development of the last testbench more enjoyable.

 

Except this task, I’ve been working on the improvement of the tests made by Adrien on the renderer, mainly by gathering code into sh functions so that tests and especially our use of Xvfb are more robust. I’ve also been working on the routing of the LED panel with Adrien, trying a slightly different version than Vincent’s one, but it seems that his one will be better. Finally I’ve written some tests for the column_mux module, a mux choosing the active columns on the LCD screen, but it doesn’t pass them.

 

I will continue to implement some tests as soon as the hardware is finished or if I get a window of time free.

[SpiROSE] no more EIM and we now have SystemC for testing

Hey, this week we read a lot of FPGA and SBC documentation. The goal was to be sure our project was feasible with the components we chose. We also validated many points on our design.

Choice of RGB versus EIM

It appears that we can’t use the “fast GPMC-like interface”, also known as the EIM on the board. Fortunately, as Tuetuopay explained it, the RGB is doing very well for what we ask it for.

The reason for not using the EIM are very simple : it doesn’t go on the MXM connector. Dot. So now we take a definitive stance on the choice of the communication bus from SBC to FPGA.

SystemC for testing

As written in our previous posts, we were trying to use Verilator as a compiler to SystemC. It is now an almost finished task. Each SystemVerilog file is built with a SystemC testbench and we are currently working on finishing a SystemC TLC5957 driver model and associated utils to produce test sequences and a way to test our driver and LED with C program directly, so as to validate everything. Tests will be done before the end of the week and we have a solid knowledge of the behaviour of the TLC5957.

Only one or two points are still beyond our understanding : the behaviour of XREFRESH also known as auto-refresh is part of them for instance, but we have good hopes to assert its behaviour with the previous library.

The Verilator makefile won’t produce correct dependency files too, but it will be fixed by replacing it by our own rules.

Next steps

There have still some issues with the SBC as it can’t do what we were expected at first.

Next week, we have to finish the tests quickly so as to integrate the FPGA parts into our project. After that we will fix the SBC issue by either finding a way to do what we want (openCL, CPU parts) or by replacing it by another CPU-only algorithm.

[SpiROSE] Continuous integration and the FPGA story

This week, we put a conscientious effort in making the project eventually start its journey in a more peaceful way. We were quite pressured by the choice of the FPGA and SBC as it was impeding the rest of the project and its existence. Now it has been fixed, we can really start to think about the next tasks: developing software and tests.

Even if it is still not perfect, we spent some time in configuring the continuous integration, designing the different steps for each piece of our software. As I spent most of the week doing code review (and FH work about happiness at work, which made me happy about ROSE), I highlighted some points where it failed, like some commits accepted by the tests although some files were missing, or incorrect style not failing the CI, or that the file organization in some subprojects weren’t practical for the integration of tests.

Two things are still missing. On the first hand, we have to save the artifacts so as to show demo easily, but we need more code so that it is useful and easy-to-follow manuals about how to use these artifact (flashing, running configuration, etc.) We expect to have this quickly for the LCD screen and renderer demos. On the other hand, we still don’t have very precise tests on our software, but things are to change quickly as we are writing simulation code in SystemC for the FPGA.

The idea, given by our teachers, is to use Verilator, a SystemVerilog-to-C++-library compiler, to put our FPGA controller into an environment with a simulated SBC input and simulated drivers so as to produce an output image. We will be able to cross-check the protocol used by the drivers in poker mode and check timing constraints. The SBC will generate sample image in the future and we will be able to check that it is correctly rendered by the FPGA.

To use Verilator, we have to:

  • compile each .sv file into a library (or only source/header files)
  • create a top file in SystemC which integrates each piece that we connect to the FPGA
  • monitor the input and output of the FPGA through SystemC stub modules

Eventually, we have to find a way to do the same on the real FPGA so as to have host integration tests.

We received the LED and the driver on Friday, so we didn’t try another experiment of the POV effect, but it will come quickly this week.

[SpiROSE] POV test on LED, need more experiments

On Wednesday, we tried to apply the experiment protocol about testing the LED’s POV effect with a rotating plane. We used a LED matrix taped on a cardboard that we fixed on an axis. The assembly was not perfect but might have validate the protocol, or at least the fact that we could use a camera to have better ideas on the POV effect. Actually, the second point was a success, but we couldn’t validate anything more.

Here is a picture of the assembly:

And here a picture of the result we got (5” exposition, 50mm focale, 3200iso, F/2.5 aperture):

There are two points to observe. On the first hand, the LED matrix is not centered on the axis, and has ~2cm of eccentricity. On the second hand, there are no shades on the middle of the result, so we couldn’t even test with diffusers. In fact, the fact that we have some eccentricity moved the problem to the edge of the visible arc.

However, we didn’t have any problem to see the matrix from a little less than 90° (perpendicular to the emission direction of the LED) so working on this experiment might be more complex than expected. We will try another experiment using the LED we will use for the project and flattening the panel.

This week, we will start the schematics and finalize to define what kind of data we send to the SBC, as it has become difficult to find SBC with the requirements for everything we excepted.

[SpiROSE] More insight into the project and shading issues

It wasn’t an easy week. On Wednesday we had to vindicate the role and utility of our subject. As we moved from a blade-based POV to a rotating-panel-based POV with quite a lot of eccentricity, it was difficult to explain that we switched from a promised seemingly “no-holes” display to a imperfect display with a middle black shade visible from wherever you’re looking to it, as explained by Didjcodt. So we had to find solutions which could reduce this flaw.

This shade appears because LEDs have a emission cone of 120°, so you can’t see them when the panel is almost parallel (depending of the LEDs emission cone and intensity) to your vision’s direction. Beside, this effect is amplified if you increase the eccentricity of your panel, especially in the part behind the rotation axis, because it creates a blind spot with voxels you will never be able to see. This last effect can be reduced with the thickness of the panel, but is inherent to the fact that we have matter in the display and can’t be solved completely.

However, we think the first one can be mitigated by using diffuser. The main question here is how to use them on the amazing thousands of LEDs we have without removing the ability to work on the PCB and fix damaged LEDs or costing too much time or money.

We also plan to use two PCBs for each side of the pane as it will allow as great x2 increase in resolution and maybe (as a bonus) enhance the feeling of having a good-performance screen. But it is a lot more expensive to craft a 200*200 PCB than many smaller PCB — and less secure for us if we are to make mistake and burn some of them. So we are brainstorming on how to craft, assemble and drive them column per column.

With Adrien, we also worked on a little script giving details about the specifications of the project.

Next week, we will work on switching on our first LEDs, working on the LCD screen, starting the PCBs’ scheme, ending the mechanical planification and I will try to make and test an experimental protocol to check if the idea of diffuser on the LED is viable so as to have strong cogent argument on the validity of this project.

Data path in SpiROSE

During this week, we worked on having a better understanding of the project and tried to define a clear data path between the computer and the rotating LEDs. Actually, we had to make a list of PSSCs for yesterday’s presentation and while we were finding new criteria, we realized more and more the amount of work needed to reach the end of the project. Therefore this step was a pretty urgent matter.

At first, we wanted to bind the SpiROSE to a computer through a usual Gbps RJ-45 link, and send the frames to the blades through 5GHz Wi-Fi, because it was the easiest way to scale the display to what we expected. However, we didn’t managed to find cheap and easy ways to do 5GHz Wi-Fi. Besides, we were not really sure of whether interferences effects could happen and it didn’t solve the power issue too.

So after looking for a wireless solution to send data from the computer to the display, we finally decided to find a SBC because of the 5GHz Wi-Fi capability we usually find on them.

The idea yesterday afternoon was to send compressed frames from the computer to SpiROSE in 5GHz 867Mbps Wi-Fi, then decode them on the SBC and send them to a FPGA which will do the transform and filtering computation for each LEDs.

After the presentation, we kept on looking for solutions so as to do all of these computations on the SBC directly, but we couldn’t solve the issue of transmitting the data to the blades from the SBC. We will find solutions by deleting some available voxels close to the center and make the resolution uniform across the display. However, it will be more difficult to do circular pattern painting. Maybe we can manage to do both ? At the moment we are trying to find a way to connect the blades to the SBC.

This week, I will work on having better specifications of the project so that any question like how many LEDs on a blade or what is the speed of the engine is sorted out before the middle of the next week. It will also help to have hindsight of what we choose and later will highlight what changed if we need to cut our expectation for the project.

We will also try to validate another mechanical model, try to negotiate more space or to match our expectation about how the blades are rotating by doing some great jokes with the mechanician. But even if I’m not bad at pun, I’m scared by how we will manage to fix the blades to the drum.