Video processing on STM32

Hello everyone,

On the HeRos project we are at a crossroads: either we use a card especially to stream video directly to the phone, without doing any video processing ourselves, or we use a camera on a STM32 and then  identify the contours of nearby objects (Robots, obstacles,etc) and then send this information back to the phone (without needing the bandwidth required by video streaming).

So we have a question for you: does anyone know of libraries made for video processing on an STM32 (pattern recognition, etc…)?

Until next time!

8 comments to Video processing on STM32

  • Loki

    An alternative solution could be to put a smartphone on the module, plugged in usb with the board and sending directly the stream via wifi. You can use cheap second hand smartphones for that, and even do some processing on the video to make the robots autonomous.

    Keep looking though for an embedded solution on an stm32.

  • Phh

    I agree with Loki. You can find new smartphones for 50€ if you need something you can buy in many samples, or even less second-hand.

    For processing on STM32, I think that you should be able to do color detection, and contour detections by yourself, and I don’t think their is much processing power left after that. pattern recognition doesn’t sound really easy…

  • Felix

    I don’t see the point to put a smartphone? But in a similar idea, if the robots are big enough and have enough battery you could use a SOM based on a Cortex Asomething, that will ease your life (lot of CPU power, generally Wifi+BT included, Linux/Android OS etc.).

    It’s very weird to have chosen your processor before to have specified your project.

  • Florian

    ARM recently announced Cortex M7 MCUs targeting DSP applications, this may be enough for your purposes if you manage to get your hands on it, apparently ST are currently distributing samples of their STM32F756NG.

    As Felix said you probably need a precise description of your image processing pipeline before you start looking for the right architecture.

    The first thing to look at would be image size and color depth as this puts some important constraints on memory size / bandwidth, then the algorithms …

  • Alexis Polti

    @loki @phh : no smartphone on the robot, it’s an Embedded System course 🙂
    BTW : a Rasperry would have done the job too.

  • You should have a look at the Open MV project, it’s open source:
    http://github.com/iabdalkader/openmv
    http://hackaday.io/project/1313-openmv

    A prototype is on tindie, you can try it:
    https://www.tindie.com/search/?q=openmv

    But he just succeeded his kickstarter and I backed it so let me know if you want me to ask him anything 😉
    https://www.kickstarter.com/projects/botthoughts/openmv-cam-embedded-machine-vision

    PS: Video treatment sounds like a medicinal treatment, maybe it’s more appropriate to talk about video processing ;p

  • BigFatFlo

    Thanks you all for your advice, we really appreciate it!
    About doing video processing, it is in fact doable with an STM32, as we can see here:
    https://www.youtube.com/watch?v=AwHzUILiGmA

    @Drix: it would be very cool to use openmv, unfortunately I think we won’t be allowed to! We had found another module that is similar: the CmuCAM, it does color recognition and tracking, etc.
    http://www.lextronic.fr/P1890-module-de-reconnaissance-video-cmucam4.html
    But If we do video processing, we’ll have to do it ourselves, not with a module made specifically to do it! Basically we would have to recreate a simplified version of openmv or CmuCAM…

  • Loki

    Not related but have you heard of http://www.reachrobotics.com/ ?