Advice on a microcontroller for a camera application - Page 1
Forum section: electronics / general

author: Boscoe
author_rank: Frequent Contributor
timestamp: July 10, 2022, 09:27:56 pm
content: Hi all,I wonder if anyone could give me some advice on which microcontroller family to use. I have a 25MP 12bit per pixel RAW image sensor outputting its data over MIPI CSI2 at an overall rate of 43MP/s or 86MB/s.I seem to be in a middle ground, here. I don't want to use Linux as it's very complicated for custom HW setups and is too slow at booting for my application. I want to use FreeRTOS or a bare metal application however this is not trivial on the more power A Cortex processors.So I ask here to see if there's an obscure Chinese or other microcontroller that could fit my needs.I would like to have:- MIPI CSI2- MIPI DSI or an RGB888 interface- Support for DDR2 or faster- Compatible with gcc or similar comprehensive toolchain- Ethernet supportNow the NXP RT crossover processors seem to be a good fit however the RAW12 CSI2 interface is broken and so it only support colour based inputs.Many thanks for any pointers!

author: fchk
author_rank: Frequent Contributor
timestamp: July 12, 2022, 06:28:06 am
content: You could use NVidia Jetson Modules. They have plenty CSI and DSI ports, Ethernet, a GPU for image processing, debayering hardware etc etc. You get CPU modules where NVidia has done the difficult things for you - using the modules is not that difficult.Ok, it's Linux, but it's powerfull. And instead of a full boot maybe you can get away with hibernation etc.fchk

author: rooppoorali
author_rank: Regular Contributor
timestamp: July 18, 2022, 01:14:20 pm
content: How about Raspberry Pi Pico?

author: rstofer
author_rank: Super Contributor
timestamp: July 18, 2022, 02:32:54 pm
content: Quote from: fchk on July 12, 2022, 06:28:06 amThis is a really good idea! I have a couple of Jetson Nanos and I'm using one with the Jetbot and a camera for learning routes (once I take a lot of photos for the model). NVIDIA has an ACTIVE forum for Nano related stuff. They are prolific in the AI department.I use the Waveshare incantation of the Jetbot:https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetbot-ai-robot-kit/I bought an HP laptop with an NVIDIA RTX 3070 graphics chip specifically to get the 5,888 CUDA cores. That's a LOT of parallelism!

author: peter-h
author_rank: Super Contributor
timestamp: July 19, 2022, 04:43:30 pm
content: That MIPI bitrate is huge. You will need something with custom hardware, like the CPUs which have built-in ETH controllers which do all the dirty work and have a dedicated DMA shoving the data into buffers from where you can extract it at your leisure.Also, unless you structure your code in a very specialised way, your processing will be very slow. Even a 168MHz ARM32 will be slower than you think unless you really optimise it. As an example I am currently getting 140kbytes/sec ETH transmit speed on mine, LWIP, FreeRTOS, FatFS reading of > 1MB/sec. The bottleneck is probably LWIP but it shows how much care you will need.Cameras all use specialised processors.

author: nctnico
author_rank: Super Contributor
timestamp: July 19, 2022, 07:36:15 pm
content: Quote from: rstofer on July 18, 2022, 02:32:54 pmHmmm, no. NVidia's documentation and support are not even close at the level that NXP or TI are. If you run into problems with NVidia modules, you could end up on your own (even with a decent sales volume) and very little is documented. My suggestion would be to go for an iMX6, 7 or 8 series chips from NXP. There is support for CSI and video in u-boot for the iMX series so that could be something to try if you don't need a full blown OS.