Need to build On-screen display for VGA - Page 1
Forum section: electronics / projects

author: isso
author_rank: Contributor
timestamp: February 05, 2019, 04:13:21 pm
content: I need to build an On-screen display module that will sit in between of a graphics adapter and monitor with VGA input and will modify the RGB video stream by inserting simple graphics or characters. Target resolutions are up to 1920x1080. Are there any specialized ICs for this purpose? I found STV9937 which seems to fit the job, however it's discontinued. Or any other ideas how to implement this using general purpose FPGA or MCU? Thanks

author: Berni
author_rank: Super Contributor
timestamp: February 05, 2019, 04:31:54 pm
content: For 1080p it will be indeed difficult to find purpose built chips for the job, but it is a pretty easy task for a FPGA.The FPGA just has to look at the Hsync and Vsync signals to determine what location on the screen is being drawn at. When it wants to draw something at that location you can have a digital output pin flip a fast analog MUX chip to disconnect the original video stream and instead connect the output to the FPGA pins where you generate the color you want to show. You can get 8 available colors in the OSD text by just using 3 digital pins each driving its own color, or if you add a simple 3 bit R2R DAC cirucit to the pins you can push that to 512 colors.Pretty ideal task for a FPGA really.

author: isso
author_rank: Contributor
timestamp: February 05, 2019, 04:56:35 pm
content: Thanks for fast response Berni,I have no experience with FPGA, so maybe a dumb question, but would the solution that you described require very fast RAM where the OSD data would need to be stored?

author: Berni
author_rank: Super Contributor
timestamp: February 05, 2019, 05:14:12 pm
content: Well expect a significant learning curve in programing FPGAs then, but it is a useful skill to have in your toolbox, so if you are not on a tight deadline its worth giving it a shot.The fastest RAM is the one inside the FPGA itself and its way way faster than needed for 1080p video. As for if you need external RAM is mostly a question of how much RAM you need. Reasonably priced FPGAs tend to have about 10 to 300 KB of built in RAM. So if you want to overlay a picture into the video that is 1920x1080 in 24bit color you will need external RAM to hold it as that's about 6MB. But if you quadruple each pixel(use 1/4 the resolution of 1080p for the OSD) that brings it down to 389KB, if you settle for 256 colors that brings it down to 128KB.But FPGAs also have a significant amount of processing power available so you can actually generate the OSD image on the fly as needed. This means that the FPGA can hold a array of ASCII text and process it into a OSD image as its being drawn from a font array. This brings the memory requirement to 20KB for a 100x200 field of text and about 32KB to hold a reasonably high resolution bitmap font. This might make it fit into a even smaller and cheaper FPGA and if the text is being updated by an outside MCU it saves it a lot of work because it can just spit a string at it rather than drawing text by flipping individual pixels in the FPGAs memory. Pretty much all computers from the 80s used this trick because it was easy to do and it saved them a lot of RAM and CPU power to get things on a screen.

author: isso
author_rank: Contributor
timestamp: February 05, 2019, 05:42:36 pm
content: Thanks a lot, that's very useful information.One more question: do you think any MCU could handle the same task? I realize it would require a very fast MCU, however maybe with some tricks (like those you described) and additional ICs the requirement can be lowered?

author: Berni
author_rank: Super Contributor
timestamp: February 05, 2019, 06:41:09 pm
content: Yes it can, but might require some clever coding to meet the timing requirements at 1080p.Hardware timers can be used to trigger from Hsync and Vsync signals to provide an accurate interrupt call at the time the correct location is being drawn to the screen, this might then trigger a DMA transfer that shovels a large RAM array into a GPIO port where you connect your MUX signal and OSD color signals. Perhaps using 8bits on the GPIO port to have 1bit MUX signal, 2bit red, 3bit green, 2bit blue. The speed of the DMA could be tuned to make 1 transfer for every 4 pixels drawn to make fat long pixels and the DMA could be triggered to pump out the same array 4 times in a row to make 4x4 sized OSD pixels on the screen.While the DMA is pumping out pixels this gives the CPU 4 video lines worth of time to fill up another array in RAM with the data for the next set of lines. Once the 4 lines are done the DMAs source address is then switched to the new array location to pump out the new pixels while the CPU starts preparing the next one, this repeats until the whole video frame is over and once vertical blanking starts the CPU now has a few microseconds of time to do anything else it needs to do, like perhaps user interface stuff, once a new video frame starts being drawn it all repeats again.How much resolution and color depth you can afford depends on how well the code is optimized and how fast the CPU is. Maybe you need huge 16x16 sized pixels to have enough CPU time to do it. Or maybe you optimize the hell out of it and use the SPI bus running at 130MHz to get high speed output and get 1x1 sized pixels but only in a single color. Depends on what you optimize for.Another trick that might be possible is to get a very powerful MCU that is capable of outputting 1080p video on a 24bit RGB bus. This bus is designed to connect a LCD to it, but with some hardware interrupt and register twiddling trickery you can perhaps synchronize the start of the frame on this video output bus with the start of frame on VGA. In that case you can overlay a high resolution 24bit color 1920x1080 picture over the original video by simply using MUXing between the video coming from the MCU and from VGA by using one of the color bits as the "transparency flag" to switch the mux at the right moment. Tho chips that can output 1080p video tend to also run Linux

author: isso
author_rank: Contributor
timestamp: February 05, 2019, 06:54:30 pm
content: Wow, that was some awesome ideas! Thanks a lot!

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 05, 2019, 07:01:18 pm
content: An FPGA would probably be the best fit here.But since you are dealing with VGA, which is purely analog, I guess you don't really have to neither decode the full video input nor generate a video output at up to full hd video clock.You may be able to use even a modest MCU, use the HSYNC and VSYNC signals to synchronize your picture generator (which can run at a much lower clock rate than the original video), and mix the input and output video (R, G, B signals) on a purely analog level. Wouldn't that work?Edit: for instance, with this kind of IC:https://www.analog.com/en/products/lt1675.htmlthis should be relatively straightforward if you don't need transparency, but opaque OSD is alright? It's a fast MUX. You could switch between your own RGB signals and the source signals, and it has integrated output amplifiers. Sounds doable with a reasonable MCU.

author: isso
author_rank: Contributor
timestamp: February 06, 2019, 01:50:55 pm
content: Hi SiliconWizard,Thanks for your suggestion, and LT1675 looks like a very useful component. I was thinking along the same lines re MCU, however I agree than FPGA is a better choice, partly because I need high quality OSD picture which is hard to get with MCU.

author: NiHaoMike
author_rank: Super Contributor
timestamp: February 06, 2019, 02:39:26 pm
content: What's your budget? You can go as low end as the ICE40 series of FPGAs or as high as the Artix 7 series, the latter of which can do 1080p HDMI. There are also HDMI ASICs with overlay support but they tend to not be well documented to the general public. (Is there a specific reason to use VGA? It's been obsolete for years and some of the latest GPUs don't even have analog outputs anymore.)Quote from: Berni on February 05, 2019, 06:41:09 pmHas anyone managed to genlock the parallel RGB output of a Raspberry Pi to an external source?

author: Scrts
author_rank: Frequent Contributor
timestamp: February 06, 2019, 03:04:08 pm
content: Look into TechWell (now Renesas?) chips. They have video processors that do video overlay - either graphics or text. I had TW8836 evaluation kit and it worked well. See here:https://www.renesas.com/us/en/www/doc/datasheet/tw8836.pdfThe TW8844 is widely used in the vehicles for overlay generation of text and graphics:

author: Yansi
author_rank: Super Contributor
timestamp: February 06, 2019, 04:04:33 pm
content: Quote from: Berni on February 05, 2019, 06:41:09 pmForget any SW solution at 1080p pixel clock speeds.You need a dedicated HW for that and at 1080p data bandwidths, no SW hack will produce any good results.

author: Berni
author_rank: Super Contributor
timestamp: February 06, 2019, 04:21:42 pm
content: Well with a software MCU solution obviously the OSD would be of lower resolution than the 1080p video its overlaying on. But it is perfectly possible with some skillful C programming and a good bit of time studying the MCUs documentation to abuse the peripherals in the right way People are doing similar stuff on a 8bit MCU while also generating sound trough PWM at the same time(Tho in those cases its all hand optimized assembler with instruction cycle counting)But yeah if you want to overlay a 1920x1080 resolution full color OSD image onto 1080p 60Hz video it will be the easiest to just bite the bullet and use a FPGA to do it properly.As for genlocking the video output of a Raspberry Pi, i think it would be very difficult to do because video output comes from the internal GPU that is running closed source firmware and drivers. There are some work in progress open source drivers to run the GPU, but they are quite complex stuff and im not sure if they go low level enough to allow you to trick the GPU to restart a frame on command. Also getting the sort of timing accuracy under the Linux OS can be rather tricky and timings in general are not very deterministic on these big boy ARM chips. There are lots of buses, memory controllers, caches, pipelining etc. that cause the execution speed and interrupt latency to jitter around. Smaller ARM chips execute directly from internal flash/ram and don't pipeline and as such can provide much more tightly controlled execution timing (in a lot of cases cycle accurate). To pull off this sort of trickery you likely require cycle accurate execution timing or your image would shake all over the place.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: February 06, 2019, 04:23:46 pm
content: 1080p @60Hz HSYNC frequency is ~ 1080*60= 65 kHz, pixel clock ~ 1080*60*1920= 125 MHz, 1080p @30Hz half that: 32.5 kHz and 62.5 MHz.All you need is one parallel in serial out shift registerthan can run at those pixel clock frequencies(*). Or two...Any µC can count HSYNCs @ 65 kHz, and load the shift register(s) between lines.(*) Edit: not really, the OSD can run at its own pixel rate.

author: Yansi
author_rank: Super Contributor
timestamp: February 06, 2019, 05:39:59 pm
content: Shift registers = dedicated hardware.I stay behind what I have stated. Forget SW solution. Not gonna happen at these speeds, this ain't no PAL. Even the slightest timing jitter will become clearly visible, rendering the result likely not very appealing.Of course one can argue with "clever programing", clever this or that. But dedicated HW will be probably much less complex as a solution and will give you much more flexibility.Even though I am not any kind of FPGA expert and I tend to stick to MCUs as much as possible, in this case I clearly recommend using a cheap FPGA or even larger CPLD to help with the job.There are tons of cheap FPGAs available that would likely fit the job with less hassle and bucks spent, than a MCU with "just the right clever this or that peripheral abused" for the job.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: February 06, 2019, 07:56:52 pm
content: To make an OSD he's going to need a µC anyway.

author: Yansi
author_rank: Super Contributor
timestamp: February 06, 2019, 08:14:43 pm
content: I do not think that making an OSD is a requirement for uCText and graphics rendering can pretty sure be done even in the FPGA. Depends on the application of course.Just wanted to make a point, that a a $5 FPGA would make for a pretty OSD overlay generator. $5 MCU would be likely struggling to do the job properly, with you losing precious time on creating hacks over hacks to make that work.

author: Scrts
author_rank: Frequent Contributor
timestamp: February 06, 2019, 09:20:46 pm
content: Quote from: GeorgeOfTheJungle on February 06, 2019, 04:23:46 pm1080p pixel clock is 148.5MHz usually. I haven't seen a microcontroller that can handle such resolutions. Not even Spansion Traveo series.

author: Yansi
author_rank: Super Contributor
timestamp: February 06, 2019, 10:36:37 pm
content: Quote from: Scrts on February 06, 2019, 09:20:46 pmThat is for a rather unusual 85Hz refresh rate. But anyway, the pixelclock is above 100MHz for sure, in any Full HD mode.You will get in trouble trying to hack any MCU for making OSD, at such frequencies. Even the quite high performance MCUs (400MHz Cortex M7, with dedicated video hardware) are way slow for any image processing at such resolutions and pixel clock speeds.You could barely find a peripheral, that could be hacked to accept a 100MHz+ external clocking.Slapping together tons of glue logic (74 series and such) around the MCU will become a major pain at 100+ MHz anyway.Not to begin with the fact that you would need a device to do the TMDS data deserialization and serialization. There aren't many devices on the market to do this. Many of them also closed under NDA.The FPGA is the obvious solution here. Comparatively even cheap and quite simple to use.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: February 06, 2019, 10:47:07 pm
content: The shift register is the only thing that runs at the pixel clock frequency. And you can choose the pixel clock frequency, it doesn't have to be the same as that of the underlying image, can be whatever you want, a lower or higher pixel rate. The µC's job is to load the pixels into the shift register before the scan line begins, at HSYNC frequencies, it's easy-peasy: no assembly hacks needed whatsoever.By the way, the OP wantsRGB VGA, no LVDS, no TMDS, no things of that kind.

author: Psi
author_rank: Super Contributor
timestamp: February 06, 2019, 10:49:46 pm
content: DMA of SPI bus running as fast as it can is the way to go if you want to make a MCU solution.If you have 3 SPI buses you can even do RGB and get some colours.This is the method used by ATMega's and STM32's to do an OSD over the top of composite video cameras for drones/RC planes.VGA is similar except It's faster, the V/H sync is already broken out and colour is separated instead of a single B&W signal with a colour burst.The trick is to code it in the right way so you have zero timing jitter.If you use an interrupt it you will get jitter because instructions have different length. So whatever instruction is currently executing when the interrupt occurs has to finish and this timing changes depending on what the instruction was.

author: Yansi
author_rank: Super Contributor
timestamp: February 06, 2019, 10:52:50 pm
content: I am yet to see a comparatively cheap MCU, that can run SPI clocks at 100MHz, with the serdes circuitry around.1080p TMDS is not analog PAL. Forget the software hackery bullshit.

author: Psi
author_rank: Super Contributor
timestamp: February 06, 2019, 10:54:58 pm
content: In almost all cases you dont need to run the SPI at pixel clock frequency.Most of the time you just want to draw some large lines/text and so your needed pixel size is much bigger than the native pixels size.(I shouldn't really call them pixels. 1920x1080 is lines of 1920 divisions or 1920TVL)Obviously you still need to fire the DMA for every line though. But that's triggered from the Sync.As long as the MCU can keep up with the SYNC you could draw pixels on the display in 10x10 grid with quite a slow spi clock.Its all about timing from the start of SYNC to the SPI output dataIf your MCU has enough ram you can have all the SPI data in ram for the entire frame, that way you're not rushing to move anything around between DMA ops.Then you can update pixels in ram as slow as needed for the MCU to keep up. Obviously you'd want to double buffer it.

author: Yansi
author_rank: Super Contributor
timestamp: February 07, 2019, 06:14:26 am
content: Fair point! However how will you keep the MCU clock (the SPI clock) in sync with the HDMI video stream?You need to cross between two clock domains. If you think your both clocks (MCU and HDMI) are perfect, then no, they aren't and we are back to what I have said: Jitter. You will get garbage data sampled at the imprecisely timed video data transitions. Any vertical edges in the OSD will become jittery.Keep in mind HDMI is a synchronous clock driven protocol, unlike PAL, where there is no requirement to the pixel clock be in sync with anything. The timing just needs to be without jitter - analog video has no pixels to begin with.

author: Psi
author_rank: Super Contributor
timestamp: February 07, 2019, 06:20:40 am
content: When doing composite video it doesn't seem to matter to keep STM32 CPU clock and video pclock in sync.I guess the jitter is too small to see.But yes, i think at 1920x1080 you might run into at least some jitter issues due to the clocks drifting in and out of sync.Maybe a 260mhz STM32F4-7 would be fast enough to be too small to see.If you really wanted to you could PLL to the sync pulses and generate a fast clock to run the MCU from that.

author: Berni
author_rank: Super Contributor
timestamp: February 07, 2019, 06:32:35 am
content: If you have the HDMI clock available then you simply run that into the MCU and PLL it up inside to the correct speed to run your peripherals. If not then you can still use a external phase detector to phase lock yourself to the Hsync line on VGA. The higher resolution just means you need more careful timing and you can definitely get cycle accurate timing and 100MHz SPI ports fed by DMA on todays ARM Cortex chips.Good hardware aware programmers can squeeze amazing impossible seaming functionality from hardware. In the same way you can bitbang USB or Ethernet out of a IO pin. Just because most programmers live in a abstracted high level object oriented world doesn't mean all do, some still are in touch with the hardware.But yeah if a full resolution detailed overlay is needed on 1080p the FPGA is the proper way to do it.

author: Yansi
author_rank: Super Contributor
timestamp: February 07, 2019, 06:38:20 am
content: An then solve the issue of the PLL running with no HDMI video input etc, etc. Hack over hack, complication over complication.Why not just doing it right from the beginning? There really are available FPGAs for as low as $5. A high performance ARM MCU like you suggested will cost multiple times this figure.Also, do you have any specific tip on ICs to ser/des the TMDS video data? (Just asking). Dedicated RGB (DPI) to TMDS and back chips are quite rare, often expensive and under NDA (due to the HDMI money suckers).

author: Berni
author_rank: Super Contributor
timestamp: February 07, 2019, 06:52:16 am
content: On a lot of modern MCUs you can run a collection of peripherals from a separate clock than the CPU and the rest and even offers automatic switchover if the clock fails. Some sort of MCU is likely needed in the design anyway and if a fairly crude OSD is needed it could do it.It all depends on the scenario and requirements. For this case the OP has stated high resolution high quality OSD so the FPGA is the more sensible solution anyway.Lattice Semiconductor makes a lot of video conversion chips along with there well known cheap FPGAs. They make HDMI chips capable of 4K with some signal processing or scaling and some can even work with encrypted HDCP streams. People like Texas or Analog also make HDMI converter chips.

author: Yansi
author_rank: Super Contributor
timestamp: February 07, 2019, 07:09:03 am
content: QuoteSo you do not have any specific tips? I can also name all major vendors making chips.I just want a specific example of a chip (or two), that could be used here to de-serialize and serialize back the TMDS video data. Plus what the cost of such chip is.

author: Berni
author_rank: Super Contributor
timestamp: February 07, 2019, 08:00:42 am
content: Its the job of the engineer designing the product to choose the chip:https://www.latticesemi.com/Solutions/Solutions/SolutionsDetails01/HDMIInterfaceBridginghttps://www.analog.com/en/products/adv7611.htmlhttp://www.ti.com/product/TFP401A... etcIf you do need my engineering services i do sometimes do it freelance for a fee.

author: Yansi
author_rank: Super Contributor
timestamp: February 07, 2019, 01:17:56 pm
content: Keep your cheeky comments for yourself. Nobody asked for them and nobody forces you to help on this forum for free, if you think you have better jobs to do.I have just asked for specific part numbers to helpissowith his project. Do you see anything wrong with that?//EDIT: ADV7611 is an example of a "closed under NDA and paywall" type of part.

author: isso
author_rank: Contributor
timestamp: February 07, 2019, 01:31:20 pm
content: Wow, this thread's been quite busyThanks for all the useful info, I'm doing more research based on your input now.After I'm done with OSD for VGA, my next task will be OSD for HDMI, so all of the related information that's been posted here is appreciated.@NiHaoMike - you mentioned there were HDMI ASICs with overlay support, I'd really appreciate some part numbers if you have any. Thanks!

author: NiHaoMike
author_rank: Super Contributor
timestamp: February 07, 2019, 02:09:56 pm
content: Quote from: isso on February 07, 2019, 01:31:20 pmsil9687 is commonly found in PIP splitters and is one I really would like to find the register map for in order to be able to hack that sort of splitter for PC control using an Arduino.

author: Berni
author_rank: Super Contributor
timestamp: February 07, 2019, 03:52:18 pm
content: Sorry didn't mean to sound rude. Just that when you have the manufacturer you can quite quickly find a list of chips for the job on there website and choose what suits the job the best.We used one of the HDMI receiver chips from TI and its not really that much of a hassle. Datasheets are available, chips are easy to buy from the big component vendors. But yeah the chips are not the cheapest out there and they have a rather high power consumption. For some reason i noticed the high power consumption being a common trend with HDMI chips, there is not sll that much processing going on in there do i don't see why that is the case.In the case of confidential documentation its often not that hard to get it if you just ask them for it. As long as you look like you are a company its fine.Sometimes getting the documentation just involves digging the internet really deep. There have been cases where i pieced together a LCD controller datasheet from datasheets of multiple other similar displays, or found a linux driver for a undocumented controller chip and reverse engineered the info out of the linux kernel driver source code for it. Also for example to get MIPI video output working on a Raspberri Pi i reverse engineered the display detection by connecting a real display and probing what its doing.So lack of easily available documentation is not always a showstopper, but it does sometimes mean a good deal of extra work.

author: Yansi
author_rank: Super Contributor
timestamp: February 07, 2019, 11:50:40 pm
content: Hence why I have asked for an IC recommendation, so one could avoid such digging, as probably some of us may have experience with some ICs, that have good stock (and documentation) availability.

author: Dicko101
author_rank: Newbie
timestamp: May 14, 2019, 02:34:18 am
content: I need a similar solution although this all seems out of my depth now. I want to show Credits remaining and a timer over an arcade game in my store over VGA for PC or some other way for Playstation, I thought it would be simple but apparently not.

author: Dicko101
author_rank: Newbie
timestamp: May 14, 2019, 03:12:01 am
content: Would it be possible to manipulate just the red channel of a VGA source to do very basic red text?

author: Berni
author_rank: Super Contributor
timestamp: May 14, 2019, 05:12:39 am
content: Yep you can inject only into the red channel, the other colors will simply 'ghost' trough it, so if you do it on a cyan background the text ends up white instead.But injecting the signal in is usually the easy part, the harder part is extracting the timing information from the existing video signal and syncing your overlay signal to it.Tho on a PC you can probably do what you want all in software.