Creating a video encoding/streaming device - Page 1
Forum section: electronics / projects

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 12:25:02 pm
content: I'm playing around with the idea of starting a project to create a device to basically act as a hardware based remote desktop. It would basically take a HDMI input from the computer, compress it and then stream. I was hoping someone here would have some recommendations on what kind of parts could be used to put something like this together, e.g. a particular SoC or such. Some of the requirements as I see them:HDMI input capable of supporting 1920x1200@60Hz 24-bitMulti-core with enough processing power to handle A/V compression/decompression10/100 ethernet controllerProgrammable USB 2.0 peripheral controllerGood assortment of I/O i.e. I2C, SPI and GPIOLow latency!I'm not sure if there are many options for SoCs with built in HDMI receiver. What kind of options would there be for interfacing an external HDMI receiver with 24-bit interface to a SoC? The HDMI signal will be 60Hz but I only plan on encoding a 30Hz video signal. It definitely will require a decent amount of processing power as I want to do all the codecs in software so that I can implement them myself. It will be encoding video and audio that is going out and decoding audio that is coming in. In addition to the codecs it will also need to be able to handle a web server as this is going to be web based.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 12:53:11 pm
content: Quote from: blueskull on February 22, 2015, 12:42:57 pmThat defeats the purpose. GbE has enough bandwidth to stream 1920x1200@60Hz uncompressed...I want to compress the videoheavilyso that I can stream it over a broadband internet connection of ~5mbit/s. As for USB, that it only for transmitting audio bidirectionally as well as keyboard/mouse.Quote from: blueskull on February 22, 2015, 12:42:57 pmI didn't mention H.264 but in any case this is for my own personal use, I am not trying to manufacture a product. If the MPEG-LA thinks I'm using their IP they can just sit on it.

author: mariush
author_rank: Super Contributor
timestamp: February 22, 2015, 01:10:02 pm
content: HDMI is kind of a bad choice because of HDCP, the encryption which is enabled when people watch movies for example. You're not guaranteed to have unencrypted output from the PC with digital outputs. Same for DVI and Displayport, these also have optional encryption. There's a chance you'd get encrypted output and it would be hard to decrypt it without paying a lot of money to get a keyYou could use a FPGA to grab video frames from VGA output and send them to a hardware encoder chip and then mux the video frames and audio to a stream and send it. If you want to avoid licensing, I guess you could do motion jpeg, theora etc but you'll complicate your life.There are such devices already if you're not aware, they're called KVM over IP .. they're expensive though.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 01:20:27 pm
content: Quote from: blueskull on February 22, 2015, 01:04:06 pmI imagine that most of these capture cards will have poor latency and may not offer raw capture.Quote from: blueskull on February 22, 2015, 01:05:46 pmThis is a remote desktop application, not video streaming...Quote from: mariush on February 22, 2015, 01:10:02 pmI don't care about HDCP. I am not watching Blurays on my computer remotely, there is not encrypted content that I need to be worrying about...No device currently exists that does exactly what i want.Quote from: blueskull on February 22, 2015, 01:12:13 pmAgain, I could not care less about HDCP...

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 01:23:53 pm
content: Also, for what it's worth, I'm not looking for pristine visual quality. Something equivalent to baseline H.264 (or perhaps even simpler, like MPEG-1 or MPEG-2) would probably be good enough. I have no idea how much processing power something like an multi-core ARM Cortex SoC would have. I know that on my i5 desktop I have more than enough processing power on a single core to encode baseline H.264 1920x1200 at a solid 30fps using x264.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 01:45:36 pm
content: Quote from: blueskull on February 22, 2015, 01:24:33 pmI want to be in control of the compression. I cannot guarantee that I will be able to do full H.264 decoding on the client as it will be web based (HTML5 + WebGL) and decoding the compressed video and then re-compressing it will only add more artifacts and latency.There is only about 30ms latency between locations, so unless you know of a HDMI/DVI capture card that can providerawpixel data to my application in much less than 30ms...

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 01:55:21 pm
content: Quote from: blueskull on February 22, 2015, 01:51:13 pmComputer vision?I want to remote into my computer at home using a web browser. The ping between my home computer and the place I would be doing the remoting is 30-40ms and I don't want to be adding 1000ms on top of that.Maybe what I should be building is a PCIe based HDMI capture card that does nothing but fill a frame buffer with the raw pixel data that can then be read out.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 02:05:15 pm
content: Quote from: blueskull on February 22, 2015, 02:01:15 pmI don't care if if the total latency is 40ms or 250ms, as long as it is reasonable, and some consumer capture card that is going to provide you with a video stream that is delayed by 5 seconds on top of everything else is simply not going to cut it.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 02:13:46 pm
content: I already know that the network performance is adequate as I am already using an HTML5 based VNC and there is no noticeable delay. The problem is that this VNC like most uses a basic codec designed to handle typical desktop content at a moderate frame rate, and I want something that can do full video at 30fps.

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 02:17:29 pm
content: Quote from: blueskull on February 22, 2015, 02:15:23 pmHow does VLC help me? I need to stream this using websockets, along with other information like cursor position, cursor image, status information, keyboard events, etc...

author: Chris_F
author_rank: Contributor
timestamp: February 22, 2015, 02:28:35 pm
content: You've been a great help.

author: nctnico
author_rank: Super Contributor
timestamp: February 22, 2015, 06:20:39 pm
content: Quote from: blueskull on February 22, 2015, 12:42:57 pmNo. A decent SoC developed for multimedia will have a hardware compressor capable of doing this. These have become pretty jellybean parts. Maybe even the 'new' Raspberry Pi can meet this requirement.