The bowels of HDMI. - Page 1
Forum section: electronics / general

author: hamster_nz
author_rank: Super Contributor
timestamp: August 05, 2015, 02:47:07 am
content: If anybody is interested in using an FPGA to explore deep in the bowels of HDMI you might be interested in my new GitHub Repo.https://github.com/hamsternz/Artix-7-HDMI-processingIt is HDMI 1080p RX and decode, audio extraction and then sending the video back out as DVI-D, but with 8 channel audio level meters that are blended into the top left corner of the video - the simplest project I could think of to show end-to-end functionality.It is a bit rough at the moment, but should get better as I fix up a few known issues (f.e. the meters double in height for 1080i inputs).Features--------Supports HDMI input formats:-720p@50- 720p@60,- 1080i (with a bug in the meters)- 1080p@50- 1080p@60and others....Colourspaces / formats supported:- RGB 444- YCbCr 444- YCbCr 422Supported Boards- Digilent Nexys Video (Artix 7 200T)Sources tested with:- Western Digital HD Live- HP Laptop- CubieboardSinks tested with:- Viewsonic Monitor - HDMI input- AOC Monitor DVI-D input- Vivo TV - HDMI input

author: diyaudio
author_rank: Frequent Contributor
timestamp: August 05, 2015, 11:41:11 am
content: Quote from: hamster_nz on August 05, 2015, 02:47:07 amNice, don't have a FPGA background, this project is a good reference if anyone want to play with something advance in the FPGA audio/video hdmi space.do you have a YouTube video perhaps ?

author: hamster_nz
author_rank: Super Contributor
timestamp: August 05, 2015, 11:55:08 am
content: Hi DiyAudio, here is a short, supper jerky, short cellphone video....

author: diyaudio
author_rank: Frequent Contributor
timestamp: August 05, 2015, 12:07:49 pm
content: Quote from: hamster_nz on August 05, 2015, 11:55:08 amThanks this looks great man, you should make a video series explaining this project, their seems to be is a serious lack of "good quality" FPGA educational videos, only fragmented knowledge.

author: marshallh
author_rank: Unknown
timestamp: August 05, 2015, 07:26:43 pm
content: Cool, that is a lot of work. You have an hdmi analyzer?

author: hamster_nz
author_rank: Super Contributor
timestamp: August 06, 2015, 02:04:04 am
content: Quote from: diyaudio on August 05, 2015, 12:07:49 pmI would be quite keen on doing a video, but I hate the sound of my own voice. Do I really sound like that? Sheesh!The other problem is that the code or design isn't of itself that interesting - for example:QuoteThe interesting bit is mapping the HDMI standard to the flow of data through the FPGA design, and in the little tidbits like the Guard Band symbols and TERC4 symbols are actually valid TMDS pixel data symbols, so you can actually move the decoding to an entirely different place than you would first have assumed... and none of that stuff actually shows up in the code.But you wouldn't find that stuff all that interesting unless you have had some prior knowledge of the protocol.

author: hamster_nz
author_rank: Super Contributor
timestamp: August 06, 2015, 02:13:52 am
content: Quote from: marshallh on August 05, 2015, 07:26:43 pmNo, I've got just the HDMI protocol spec (thanks Google!), an FPGA board, a few different HDMI sources/sinks and a lot of time in the simulator.I've been leapfrogging through projects to get more and more adept- Analog VideoGenerating basic analogue VGA signals- TMDS / DVI-D outputConverting those signals to DVI-D- TMDS / DVI-D inputReceiving and decoding high speed data.- HDMI outputBuilding valid HDMI streams that actually work- HDMI inputReceiving valid HDMI streams and decoding them.Strangely enough, the most painful bit so far was EDID/DCC - which is required to get the HDMI source to send HDMI data. I've wasted so much time over something that should be so simple, due to various timing issues, checksums, lack of clear documentation and my own stupidity - I can shift bits at 1.5Gb/s, but can't get a flipping 400kHz I2C device to play ball. Grrrrr.Oh how I now love that Saleae Logic Analyser I borrowed...

author: Stebanoid
author_rank: Contributor
timestamp: August 06, 2015, 09:31:09 am
content: Well done!Quote from: hamster_nz on August 06, 2015, 02:04:04 amI thought why I don't like my voice recorded:Usually, you hear your own voice mostly through your bones which have another Frequency response, so when you hear your real voice it differs and it makes you angry. It's not a problem for other people - your voice is most likely OK for them because they don't have any preprogrammed anticipations about it.

author: hamster_nz
author_rank: Super Contributor
timestamp: August 06, 2015, 11:19:35 am
content: Oh, I've now added 'rule of thirds' guidelines too...

author: diyaudio
author_rank: Frequent Contributor
timestamp: August 06, 2015, 01:10:32 pm
content: Quote from: hamster_nz on August 06, 2015, 02:04:04 amSame here, wait maybe Dave can give some help in this department.Maybe talking in the mirror to warm up?, I have a pretty good mic (audio technica at2020) on the desk and I feel very annoyed listing to my own voice as wellWhat HDMI specification doc did you use ?

author: krivx
author_rank: Frequent Contributor
timestamp: August 06, 2015, 01:46:22 pm
content: Quote from: marshallh on August 05, 2015, 07:26:43 pmHave you used one? I have seen that kevtris has one and used it to validate his HDMI Nes project. It would be very nice to see an open source HDMI-compliant example, it feels like analyzers are so niche and $$$ that it hampers a lot of fun potential projects.

author: Bassman59
author_rank: Super Contributor
timestamp: August 06, 2015, 05:24:43 pm
content: Quote from: hamster_nz on August 06, 2015, 11:19:35 amShaun the Sheep! He even mucks about with those who cannot bleat!(the movie finally opens here in the colonies this week!)

author: hamster_nz
author_rank: Super Contributor
timestamp: August 06, 2015, 06:04:22 pm
content: Quote from: krivx on August 06, 2015, 01:46:22 pmAlthough could arrange access to one, I'm deliberately avoiding it. To extract video data (and to know where to replace it) all you need to know is how DVI-D works (seehttp://www.cs.unc.edu/~stc/FAQs/Video/dvi_spec-V1_0.pdf), and that video data is announced in advance by a block of eight CTL codes, followed with two guard band data words (that can be ignored as they don't show).If you get a trace of the symbols the HDMI format it is pretty much self-evident - you see standard CTL (non-video) symbols, then CTL codes on Ch1 & Ch2 change for 8 cycles (Ch0 is used to transfer HSYNC and VSYNC) , followed with 1082 data symbols. If you decode the TMDS for them you get the raw pixels, with two constant values at the start of the raster.That ignores the whole HDCP issue - however some cheap HDMI video splitters can take care of that for you - one of the channels comes out clean.

author: marshallh
author_rank: Unknown
timestamp: August 06, 2015, 06:50:58 pm
content: Quote from: krivx on August 06, 2015, 01:46:22 pmActually I will be using his this weekendI convinced him to do a teardown a while back, there's a video if you haven't seen it.

author: krivx
author_rank: Frequent Contributor
timestamp: August 06, 2015, 08:32:43 pm
content: Quote from: marshallh on August 06, 2015, 06:50:58 pmHa, I have seen it. I really like his videos, it's nice to see someone cover this stuff with such expertise

author: hamster_nz
author_rank: Super Contributor
timestamp: August 11, 2015, 05:07:24 am
content: Last night I extended the project to do Sobel-like edge detection:The threshold is a little bit high at the moment so I'll fix that and get rid of the artefacts at the edges of the screen...

author: davorin
author_rank: Unknown
timestamp: April 21, 2018, 03:15:01 pm
content: Hello MikeI know..this topic is rather old...but I just happen to experiment with your HDMI processing design on my Arty 7 board (o;Just a quick question....should hdmi tx generate a signal when there is no input at the rx side?Or di I need to connect SDA/SCL on the tx side as well?Lastly I used your Artix 7 HDMI output example and I could run it with just connecting the TMDS pairs and nothing else...

author: BrianHG
author_rank: Super Contributor
timestamp: April 21, 2018, 10:07:53 pm
content: Quote from: hamster_nz on August 11, 2015, 05:07:24 amThe true method to get rid of edge screen line artifacts on enhancement and sharpening/blur filters. Pay attention since altering the enhancement level isn't a good fix if you need high levels throughout...Simplified illustration using 1 line of a 3x3 convolution matrix. You will need to expand this on the X and Y axis as needed. Include the Z axis if you are doing Z time domain filters like motion blur...x1y1|x2y1|x3y1-------------------x1y2|x2y2|x3y2-------------------x1y3|x2y3|x3y3As the video is coming in, once active video flag is enabled, for the first pixel, x1 and x2 and x3 should all be made equal to pixel 1. For the next pixel, x1 = the next pixel as x2 takes the previous x1 and x3= the previous x2. When beginning a new scan line, all the pixels in the filter coefficients become equal meaning 0 change meaning 0 artifact. After the end of the active video on the line, x1 should hold the previous pixel not changing. The shift roll effect here will have the same consequence of the edge pixels off the side of the image being equal to the pixels just before erasing that edge detect artifact.On the Y lines of video, follow the same procedure. At the First new video line coming in, feeding y1, use a that y1 as you y2 and y3 as well. On the next line as your y2 buffer has been now filled, use the new y1, use the buffered y2, for y3, use the existing y2. On the next line, the new coming in y1 with the buffered y2 and the buffered y3 are all valid. At the bottom of the video, for the line where an empty y1 is coming in since there is no more picture, use the buffered y2 data as your y1 data. The buffered y2 will also be use as y2, y3 is still used and still gets it's data from y2... Ect... you should have the idea.You are essentially doing a padding of picture data where there is not picture data. The padding data turns out to be the last pixel value data at the edge of the image. You just need to take care in handling Y axis as the padding data is held in a moving line buffer.BTW, great job at decoding & encoding the HDMI bitstream.

author: hamster_nz
author_rank: Super Contributor
timestamp: April 22, 2018, 12:50:31 am
content: Quote from: davorin on April 21, 2018, 03:15:01 pmThe SDA/SCL are only needed on the source (input to the FPGA) side, so it can 'see' a valid HDMI sink.On the output side it is DVI-D, and uses the pixel clock from the source (so no output is generated if the source is not active). It ignores the EDID info of the final device.

author: davorin
author_rank: Unknown
timestamp: April 22, 2018, 07:28:10 am
content: Ah okay....seems there is another problem then with the Arty 7 board....the clocking region:Code:[Place 30-149] Unroutable Placement! A MMCM / (BUFIO/BUFR) component pair is not placed in a routable site pair. The MMCM component can use the dedicated path between the MMCM and the (BUFIO/BUFR) if both are placed in the same clock region or if they are placed in horizontally adjacent clock regions. If this sub optimal condition is acceptable for this design, you may use the CLOCK_DEDICATED_ROUTE constraint in the .xdc file to demote this message to a WARNING. However, the use of this override is highly discouraged. These examples can be used directly in the .xdc file to override this clock rule.< set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets i_hdmi_io/i_hdmi_input/clk_pixel_x5_raw] >And changing the TMDS input clock to 720p (13.47nsec period in constraint file and hdmi_input.vhd) gives me:Code:[DRC PDRC-34] MMCM_adv_ClkFrequency_div_no_dclk: The computed value 371.195 MHz (CLKIN1_PERIOD, net tmds_in_clk) for the VCO operating frequency of the MMCME2_ADV site MMCME2_ADV_X0Y1 (cell i_hdmi_io/i_hdmi_input/hdmi_MMCME2_BASE_inst) falls outside the operating range of the MMCM VCO frequency for this device (600.000 - 1200.000 MHz). The computed value is (CLKFBOUT_MULT_F * 1000 / (CLKINx_PERIOD * DIVCLK_DIVIDE)). Please run update_timing to update the MMCM settings. If that does not work, adjust either the input period CLKINx_PERIOD (13.470000), multiplication factor CLKFBOUT_MULT_F (5.000000) or the division factor DIVCLK_DIVIDE (1), in order to achieve a VCO frequency within the rated operating range for this device.Using "BUFIO_x5_inst : BUFG" instead of "BUFIO_x5_inst : BUFIO" the design runs through and I can program it...but the timing requirements are not set with BUFG. Remember a post on the Digilent forum that you had to use BUFIO for faster clocking.Guess I am out of luck with my Arty 7 board (o;

author: davorin
author_rank: Unknown
timestamp: April 22, 2018, 11:48:51 am
content: Interesting...with the Zybo Z7 board the hdmi processing design works with feeding 1080p@60 from a Raspberry Pi...had to use BUFG as well for the x5 clock....and worst negative slack is reported with -3.874nsec.First thought it might to do with HDMI TX/RX being in a different IO bank as only on the Nexys video board both TX/RX are on the same bank.Maybe the Arty 7 RX side doesn't see any input signal...

author: james_s
author_rank: Super Contributor
timestamp: April 22, 2018, 04:54:32 pm
content: Quote from: Stebanoid on August 06, 2015, 09:31:09 amFunny, I'm the same way, hearing my own voice doesn't make me angry, it just sounds weird and I don't like it.