Interfacing HD cameras and MCUs/FPGAs - Page 1
Forum section: electronics / general

author: Psi
author_rank: Super Contributor
timestamp: June 17, 2013, 07:54:08 am
content: I'm looking at doing a project that will require connecting a HD camera (raw 720P 25fps) to a mcu/fpga.There is very little processing that needs to be done on the data stream so i don't need a lot of CPU power.All i need is to buffer one or two HD frames in ram and then generate some other slower I/O signals based on the data.No complex compression/decompression, complex math, rotation, or DSP is needed.I've never used a FPGA before so a MCU solution would be better if it can work.What i'm not sure about is- What sort of HD camera interface works best for a MCU? HDMI input seems overly complex but would allow for different cameras to be used.- Do i need to be looking a FPGA for working with HD pixel data?- Any recommendations on dev boards for this sort of project- Any recommendations on HD cameras that are suitable for mcu/fpga communications

author: Psi
author_rank: Super Contributor
timestamp: June 17, 2013, 08:35:54 am
content: Actually, i probably only need to buffer 3 of the HD lines (1280 pixels). Shouldn't need to buffer the entire frame.I will need a raw camera interface though, something that gives me raw pixel data and isn't compressed.I'm thinking maybe the raspberry pi with its HD camera module might work for development.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: June 17, 2013, 09:14:12 am
content: Spartan 6 can receive HDMI, and there is some example code out there. Take a look at theDigilent Atlys boardwhich has multiple HDMI ports and 128M of DDR2 RAM.They also do acamera add-on boardAnother possibility is Bunnie'sNeTV, which combines a Spartan6 and a Marvell processor running Linux.You can also get industrial cameras with ethernet, which may be suitable for interfacing with RasPi etc.A camera with a parallel pixel inerface, or LVDS interface should be fairly straightforward to interface to an FPGA.If time is more important than cost, there are some industrial cameras available with built-in FPGA to do your own applications on.Something to consider is that with any low-level camera module you will need to deal with stuff like Bayer de-interleaving and the other magic that improves image appearance yourself, whereas with a DVI/HDMI type interface this will already have been done for you and you can just focus on processing the pixels..

author: Psi
author_rank: Super Contributor
timestamp: June 17, 2013, 10:15:35 am
content: Thanks,I'm in two minds about it.Part of me wants to design the thing from scratch in which case id probably learn FPGA programming, make a PCB and have a nice neat small product.However i also like the easy of just selling a small extension board for the raspberry pi. (Let people buy the pi themselves)Since the pi already has a HD 30fps camera module available and the open driver sourcecode it makes it a tempting platform.It does limit people to only 1 camera option though which isn't as good as HDMI

author: Psi
author_rank: Super Contributor
timestamp: June 19, 2013, 04:40:49 am
content: The idea for the project is to take a HD video feed (either hdmi camera or a raw CMOS sensor) and multiplex the pixels and lines to produce 2 or 3 SD composite video outputs. eg, HD frame line 1,3,5.. goto composite1 and 2,4,6.. goto composite2. The same thing with the pixels, alternating between each composite video image.Then these 3 composite video feeds can be transmitted using cheap 5.8 video transmitters many km at very low cost using 3 different frequencies.On the ground the 3 comp video signals need to be digitized and the HD image recombined.The concept is pretty simple and i cant see any reason why it wouldnt work.Obviously the recombined image isn't going to be as good as the camera source image but it should be miles better than a regular SD image.Any ideas? comments?There are many IC's for decoding a composite video image into a digital data stream but ive not found many chips that go the other way.Is generating a comp video signal normally just done raw/in-software using a stock DAC?

author: Rasz
author_rank: Super Contributor
timestamp: June 19, 2013, 05:13:25 am
content: I like your idea of splitting HD image by lines. If you rotate by 90' you might get away with only 2 transmitters for 720p25 at the cost of one frame latency.this might be helpfulhttp://chipsight.com/chipsight-slides

author: lgbeno
author_rank: Frequent Contributor
timestamp: June 19, 2013, 05:25:01 am
content: I think that u would lean toward an FPGA to do this. This will make a much more deterministic system in the end.FPGA has no issue receiving hdmi signals and streaming out some SD based VGA/Composite signals. There's a lot of existing IP for this a well.http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=65#Category68Unfortunately the dev boards are not cheap, maybe your better off spinning your own :/

author: Psi
author_rank: Super Contributor
timestamp: June 19, 2013, 06:31:29 am
content: Yeah, i'm also leaning towards using a FPGA.I've never used one before though. So it would be quite a learning experience moving from MCU programming to FPGA programming.Quote from: Rasz on June 19, 2013, 05:13:25 amYeah, its a good idea.It would require buffering the entire frame before any composite outputs could be generated though.It would be nice if the HD lines could be sent to each composite output in realtime as the come in which would limit the system to line by line.

author: marshallh
author_rank: Unknown
timestamp: June 19, 2013, 06:38:22 am
content: If you want to do HD FPV there is a better way, but it's not been done en masse by the chinese yet.Probalby would work better to split the feed vertically into 3 stripes, and stretch them to fit your 4:3 wireless transmitters.This is the kind of thing FPGAs are perfect for.However what you are looking at is not a noob-level project, and is going to be a decent hurdle for an fpga newcomer.To generate the composite video output you could use a ADV7391 or similar.Of course it would better to transmit the info digitally in packet form, but thats anotehr can of worms...

author: Psi
author_rank: Super Contributor
timestamp: June 19, 2013, 07:21:06 am
content: Yeah, i agree, its not going to be an easy project for a FPGA noob.I'm thinking it would be better to use a ADV7611 HDMI reciever for the camera input rather than trying to decode HDMI inside the FPGA.An input pixel stream from a HDMI receiver and an output stream to a composite video encoders would probably simply the FPGA code complexityI'm also still wondering if a raspberry pi with its camera module would have the cpu power to feed 2 composite video encoder ICs.It would make things easier if i could do it without a FPGA.The 2 encoders would be fed in parallel with their clocks linked etc. So really, it's like it's feeding 1 video encoder with 2 lots of parallel data

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: June 19, 2013, 07:45:15 am
content: ..of course the optimal way to do this would be to compress the video to reduce bandwidth, and add some error correction. Obviously this is rather more complex.With a multichannel link, an error on any channel will cause glitches in the output - one way to reduce this could be to split into line-interleaved streams with some simple error detection/correction like hamming code, and if an error is detected on one stream at the receiver, copy the adjacent line to disguise the error.

author: Rasz
author_rank: Super Contributor
timestamp: June 19, 2013, 07:57:52 am
content: PI will have high latency+ Im not sure if they even let you see uncompressed VIDEO buffer, Its all closed source garbage APIs with no access to real hardware.Not to mention output options - there is plenty of processing power, but no way of outputting your data without even more latency (only USB).Beatlebone would be better in this regard, but would need camera research and blablabla, in the end more work than fpga.If you are fine with some latency get PI + wifi dongle + Panel antenna and just stream mpeg4

author: Psi
author_rank: Super Contributor
timestamp: June 19, 2013, 10:52:27 am
content: Any more than say 50ms of latency would make it useless unfortunately.One thing i'm worried about, as mentioned above, is that china hasn't started producing HD FPV options yet.When they do its going to make this project redundant.It might be better to just wait, there's a massive number of people just itching to buy HD FPV systems and use them with the consumer oculus rift.I'm sure china is working on it.

author: Rasz
author_rank: Super Contributor
timestamp: June 19, 2013, 11:45:33 am
content: Im sure they dont, they are waiting for someone else to research it so they can copy whole designIf you lookat my post history you will find another topic about same thing, I found some Chinese dude working for CCTV solutions company playing with 1 chip DVB-T encoders - they are used in HD CCTV circuits, instead of sending one low res composite signal per cable they pump many HD channels into one cable.You can buy dev board from them if you really want to, AFAIR it was $100 for USB dongle, you pump HD video thru USB and get DVB-T on antenna output.