Graphics stream processor for TFTs. - Page 1
Forum section: electronics / general

author: paulca
author_rank: Super Contributor
timestamp: June 11, 2025, 11:42:59 am
content: I wrote a dashboard style app with a 3" TFT screen on STM32.It was great, got data via ESP32 with it's vanilla factory firmware as "comms module", how quaint, the ESP32 in it's natural habitat for once. A command processor was added to give it a command line for debugging.An idea began to form.So I added a command - any command that starts with "*" is considered a "GFX Message Stream".It is passed to the GraphicsProcessor, which converts terse, condensed, ascii structs into display driver commands.*c - clear screen*C - set foreground color (RGB888)*B - set background color (RGB888)*f - set found for index (int)*x - set x pos (int)and so on and so forth.*c*f2*C00FF00*BFFFFFF*x100*y100*r050,050*x102*y102*R046,046*x110*y110*tHello WorldWould clear the screen and draw a green on white rectangle and then draw a filled rectangle inside it and then write Hello in the same colour so you wouldn't see it.Surprisingly it's pretty fast. I don't think you will be using it for animations or a scope, but that was not it's intent.The intent is to not have to design basic UIs on a TFT display using C code. That gets really, really tedious, really, really fast. The lengths you need to go to to try and keep it all tidy ... sod that. Seems pointless. Seems the perfect use of an abstraction layer.Somewhere else I can generate this "byte code" and send it to the device at runtime. This means it becomes a "drop in" component and is reusable across projects. Not least that it means you don't need to write any C code at all to display stuff. Not unless you are sending the gfx stream from C code of course... and not over the Wifi like I intend to.So rather than the "first model", where I might send it:{"temp1":"10.6"} from a message bus and it would then do all the lines, rectangles, boxes to place a "widget" box on the screen with that value in C code..I can now do that in a much higher level environment like python and instead sent it the gfx primitives as a stream. eg:On topics/dashboard/gfxstream:{"draw":"*c*f2*C00FF00*BFFFFFF*x100*y100*r050,050*x102*y102*R046,046*x110*y110*t10.2C"}and it does the needful. The python or whatever I choose upstream can generate those primitives and track dynamic layouts, "hierarchical update/redraw" mechanics so it only redraws or clears what it needs to. "Write last text in background, write new text in foreground" optimisations etc.... all do-able in C code, but far easier .. not in the C code.Adding bitmap loading is also fairly easy ... assuming you upload and index them firstBase64 inline bmps... I mean, sure, why not. It's not like you have a massive amount of pixel real estate and ... if you get clever there is more display ram than the view portal of the screen.At the moment it only works from it's debug command line and "Streams" are only 1 command long, so it IS quite slow.I just thought I would ask for feedback and thoughts before I consider advancing it further.I know someone is going to say:* You can use STM32's FusionGfx or whatever... yes I know. I still have those nightmares.* Someone else did this - insert git repo - pointless as STM32 code is about as portable as Stone Henge is.* Someone else did this, it works, has a GUI, the MCU side is light weight and widely supported.... this I might pay attention toEDIT: Bonus. I already have a PCB platform build for it.

author: peter-h
author_rank: Super Contributor
timestamp: June 12, 2025, 11:00:50 am
content: You will have to re-code it for every different Chinese LCD module. You've probably seen thishttps://www.eevblog.com/forum/microcontrollers/anyone-here-using-the-ramtex-graphics-library/

author: paulca
author_rank: Super Contributor
timestamp: June 12, 2025, 11:33:54 am
content: Quote from: peter-h on June 12, 2025, 11:00:50 amInteresting point. Although again that is an abstraction, though not a very good one. It uses a display driver off github which only work for "close family" of screens in question. I just "route" and "sequence" function calls on it.I'll check the link. A more robust abstraction library which supports different screens, might be useful later. "might". It would depend on the target audience/market. If it's just for me, I don't see it being a problem "re gluing" the Gfx command processor to a different lower level gfx library.

author: paulca
author_rank: Super Contributor
timestamp: June 12, 2025, 11:40:49 am
content: Anyway, I hope to get a proving test this weekend.1. Process the full (max 2048) 'command' as a stream of unions, not a single.2. Write a python generator to create a "data tile" display in gfx primitives. Make 8 or 10 of them.3. Generate individual updates for individual "tiles". Time it by a scope and GPIO edge.4. Load test it with a continuous stream of "Invalidate all and redraw" sequences to draw all tiles as quickly as it will let me.If it looks like a kids BASIC spectrum graphics tutorial and takes longer than 1 second to full refresh the screen it's a failure without optimisation. I don't mind a noticable "blink" even on a single tile update, but I don't want to watch it draw each sub component individually at human time scales. For a full refresh I don't mind if it goes "blink, blink, blink" as long as the screen if all done in a second.Tests will tell.

author: ajb
author_rank: Super Contributor
timestamp: June 12, 2025, 06:31:31 pm
content: There's nothing intrinsically wrong with the idea, and if offloading the graphics to an embedded device via streamed commands works for your project, then go for it!But building and using -- including porting to different platforms or displays -- a graphics system in C shouldn't be THAT hard or cumbersome. It's all about breaking things down into the right layers and keeping things loosely coupled. You build the low level SPI or whatever drivers, then some basic graphic commands to put pixels on the display on top of those, then more complex commands or widgets on top of those, etc. Once you have the lower level stuff in place, the level where you assemble widgets into a screen doesn't need to look that different whether it's done in C or in Python. Of course you also have to choose the level of implementation complexity that makes the most sense for the intended use-case. Of course when writing general purpose libraries it can be tempting to include features that add more complexity to the implementation than their real-world usefulness justifies. Doing that in Python might not be a big deal, where in C it might blow a project up into an unmaintainable mess -- but keeping an eye out for that is just good design practice in any language.

author: langwadt
author_rank: Super Contributor
timestamp: June 12, 2025, 07:21:40 pm
content: controllers like for example RA8875 can do similar operations (and much more) and can be jumpered for SPI/I2C/8-16bit Parallel interface

author: peter-h
author_rank: Super Contributor
timestamp: June 13, 2025, 10:15:48 pm
content: I've spent last few weeks doing some of this (see the other thread I linked) and it all takes a lot more than you thinkEspecially when you realise that even a 320x480 display, fed optimally with DMA (yes; more work), takes about 300ms to paint with one colour. That's via SPI, 10MHz. Via parallel, much faster for sure, but then you lose most of your CPU's GPIO, and I suspect that is a killer for a lot of projects. So you end up doing a lot of optimisation to make the thing usable.QuoteIf you can't program in C, that's trueI would suggest that 10 years from now Python will be as "popular" as Ruby on Rails was 10-15 years agoIt is bloody slow for sure. Also a big code and RAM overhead; I had someone look at implementing it on my board (which is a C user programmable box).

author: thm_w
author_rank: Super Contributor
timestamp: June 13, 2025, 10:21:56 pm
content: What stm32 and what LCD? Normally I would expect 30-60MHz SPI to be feasible.edit: stm32f417 and ILI9488 is covered in the other thread.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 14, 2025, 03:13:02 am
content: I play with 240×320 (targeting ~ 6 Mpixels/second) and 320×480 (targeting ~ 12 Mpixels/second) panels, typically using 16-bit color (RGB565), which would require 96 MHz and 192 MHz SPI clock rates. It would be much nicer to use (DMA to) an 8-bit parallel bus; this is natively supported by ILI9341, ILI9488, ST7789 variants, and most others; not all display panels do expose all the mode select pins, though. (BuyDisplay2.4" 240×320,2.8" 240×320,3.2" 240×320,3.5" 320×480, for example do.)You'll want your DMA to be slaved to the /WR strobe, so that on falling edge the DMA sets up the 8 data pins, and on a rising edge the display controller will latch the data. All this is relatively straightforward thus far. The two annoying things are A) generating exactly the correct number of /WR strobes (falling and rising transitions), and B) the D/C or data/command strobe. For a command byte, D/C should follow /WR strobe. For parameter and data bytes, it should stay high.Thus far, the best method I can think of is to actually output the command byte when setting up the next DMA transfer, manipulating the D/C as a GPIO pin. This is suboptimal, because each command must be a separate DMA transfer. For example, to update a rectangularH×Wregion, you need three commands: aColumn Address Setcommand with two parameter words (starting and ending columns), aPartial Areacommand with two parameter words (starting and ending lines), and aWritecommand withH×Wpixels of data (two bytes per pixel for RGB565). You then need to generate the correct number of /WR strobes. Any excess, and you'll overwrite the beginning of the region. Using SPI CLK or MOSI line, padding the pixel data with 8 extra pixels (copied from the beginning of the region), rounding the length up to a multiple of 8, and DMA'ing 0x55 = 01010101₂ to SPI is the "best" way I've found.A small 8-bit parallel to 8/16-bit parallel expander with a pulse counter would be very useful. The MCU supplies a suitable clock, and an open collector control output that is pulled high by the expander. Whenever the MCU pulls it low, the expander connects the clock to the MCU DMA trigger line (so this isslaveDMA, triggered by GPIO pin), and starts by reading four bytes. This specifies the 24-bit length (in bytes or byte pairs) of the data to follow, and the mode (read from or write to the external display controller). If the length is zero, the expander disconnects the clock to the MCU, and stops, pulling the control high. Otherwise, it passes through that many byte pairs from/to the display controller, then reads the next four bytes. D/C is low for the first word only. At minimum, it would have 11 data lines to the MCU (clock out, clock in, control, D0..D7), and 12 or 20 data lines to the display controller (/CS, /WR, /RD, D/C, D0..D7 or D0..D15). I'd also connect TE pin from display controller to MCU, for detecting vertical retrace. 12 pins for a display module would not be too much.If implemented in a small CPLD or FPGA, it probably would have enough resources left to implement a write-through lookup. The idea is to add two more modes: one reads 256 pixels into cache (palette), and the other reads 8-bit indexes from the microcontroller, and does an 8-bit lookup for each pixel.Unfortunately, all this added complexity is just not worth the component and development cost for a hobbyist like me. You still need a microcontroller with a minimum of 320×480×16bit = 307,200 bytes of RAM for the display alone.Now, I can buy a fullMilk-V Duofor about 10€, or aSipeed LicheeRV Nano-B(basic, without Ethernet or WiFi) for about 12€, shipped including taxes, from AliExpress. These have a dual-core 64-bit RISC-V processor with 64M/256M of RAM, and a high-speed (480 Mbit/s) USB 2.0. They have just enough I/O pins for a parallel 8-bit 8080-style bus, so that the simpler core can be a glorified display controller, combining the output graphics from multiple planes with transparency and blending, and the bigger core can do all the rendering. Lots of memory for pixmap caches, too. And a high-speed microSD card socket for the firmware and reading graphics resources from. LicheeRV Nano-B istiny, but still has a 31-pin MIPI DSI (2-lane) connector for displays, a backlight LED driver, and a 6-pin I²C connector for capacitive touch controllers, plus the four-lane CSI connector has 10 consecutive GPIO pins at 1.8V logic. Which is not a problem; a SN74LVC8T245 and similar translators are cheap. The documentation on these is not complete, but the schematics show exact connections, and Sipeed and MilkV try to be friendly to open source, so I'm basically happy enough. (It turns out that even though LicheeRV Nano-B is smaller, it maybe much better suited for my needs, if I map the I/O pins differently. I already do have some Milk-V Duo 64M and 256M boards to play with.)Is this utter overkill for a glorified HMI or programmable display?YES, definitely. But I don't care. I haven't gotten it to the demo stage yet, though; I'm not in any hurry.I know from doing the software-display-controller/renderer on Teensy 4.0 (Cortex-M7 running at 600 MHz) that this approach works well for me. I can cache 4-bit (opacity) anti-aliased pixmap fonts, do paletted "sprites" and blitting, and so on. Frame-sync'd animation is no problem. I can even implement ananosvgrenderer, and have some/all graphics assets in SVG, rather than pixmaps. My host is always a Linux machine, so USB is perfect, but otherwise using one of the UARTs at a suitably high baudrate would work too.

author: peter-h
author_rank: Super Contributor
timestamp: June 14, 2025, 07:20:30 am
content: QuoteSurprisingly, if you go through the Buydisplay site etc, you don't find many controllers that go that fast. There are plenty of 240x240 displays that do (60MHz) but paradoxically the bigger ones are slow and some are ~5MHz. That isn't too bad if the controller has a block fill capability but most don't.Also a 168MHz 32F4 can do 42MHz on SPI1 (which you lose if you also want 4 serial ports) and 21MHz on SPI2 and SPI3. Probably the 500MHz chips can do more. I actually wish I had kept SPI1 somehow because 42MHz is nice for fast reading of my Adesto SPI FLASH which holds a FAT2 filesystem.QuoteYeah, but then that is another big project, dev tools which change frequently, lots of learning curves, and more money... I did a lot of FPGA work years ago; it is fun but a total waste in most cases. But if you enjoy doing things the hard way, why not?I've just spent many days working around the D/C stuff. Some controllers support 9 bit SPI which then avoids waggling D/C and then you could use DMA for the whole lot. I am implementing just two controllers, and they are different enough. But one tends to find that the way graphics objects are generated, you tend to get a "set address" command (4 bytes) and then multiple "load data" commands (1 2 or 3 bytes for each pixel depending on colour depth). The latter can be done with DMA. The trick is to get the library to generate data in that way; e.g. fills within a viewport need to be clipped cleverly. The lazy coders will check each pixel against the viewport and then you get just one pixel at a time.BTW you should also have variable backlight, via a DAC-programmable current source/sink. Some displays, like the 9488 one I am using, show a bright grey screen when you do the display off command, and you want to suppress that. Block fills can also look a lot better if you blank the display.

author: paulca
author_rank: Super Contributor
timestamp: June 14, 2025, 10:32:54 am
content: So expectation seem very wide.The purpose I have is a "Dashboard". A desktop display widget.The screen is an il7 something (I don't have it open) and an F411 @100Mhz. It's lower layer is SPI w/ DMA.I want it to show data which updates at most once every few seconds. So if I fit in 10 data displays per page the update rate is unlikely to exceed once a second. It will probably be updating one tile at a time.On python, horses for courses. It's strengths in where I am leveraging it here is it's inherent string manipulation abilities and it's "RAD" rapid application development ethos.No it's not fast to run. It will probably take Python as long to build the GFX stream on an 64bit AMD core than it will take the MCU to draw it.It's fast to write though.Also, this is (should be obvious) a network endpoint device. It is part of a wider system. If you think I am going to start writing MQTT code in C... well I might for "fun", but I'm not going to do that unless I actually need the performance lets say. It would honestly take 10 times longer in C than in Python and have 10 times more bugs. I "did" have to do the payload parsing in C on the MCU end and parsing the original full JSON payloads the "big system" normally uses and ... well... the little F411 was doing it's best, but the "big iron" could easily swamp it into underruns very quickly. All the MCU was doing was iterating a pointer and calling a "token" method for each new token.On foot print. My system consists of 20 odd "services". Each is a python script running in it's own little containerised linux OS (Alpine) on a vHost. All 20 services include 3 MQTT servers and all micro-services, and the vHost running them all is hard limited to 2 cores and 1Gb of RAM.In big iron terms, C would probably able to undercut that foot print by a margin, but in grand terms it's "tiny".

author: paulca
author_rank: Super Contributor
timestamp: June 14, 2025, 11:07:24 am
content: There is a different approach. Move the graphics driver to the "big iron" at that layer instead.Instead of even having a il7xxx_tft.h driver with the likes of DrawRect() functions, I can strip it right back and just leave the DMA transfer functions. Then I have the "big iron" do the full screen render, calculate the diff and send memory clip areas as raw bytes to the MCU buffer and "latch" it to the display.That puts the pain on the middleware and unfortunately the link between ESP32 Wifi buffers and the STM32 is UART. So we aint going to be wining any refresh rate awards this way either.

author: ajb
author_rank: Super Contributor
timestamp: June 14, 2025, 06:23:22 pm
content: Quote from: peter-h on June 13, 2025, 10:15:48 pmWe must assume some basic competency of course-- but languages like Python do have capabilities that just make some things much easier than they would be in C.Quote from: Nominal Animal on June 14, 2025, 03:13:02 amIf you've got an MCU with an external memory interface (and are willing to dedicate enough GPIO to use it), you can allocate one of the address lines to the display D/C pin, and let the memory controller hardware handle the write strobes and timing. In non-DMA mode, you can simply write commands to an address where that line is low, and write data to an address where its high. If you select a high enough address line, you can set up a DMA transaction to sequential addresses such that the D/C pin changes after the first write and then maintains its state for the rest of the transaction. eg, if you use A16 for D/C, do a sequential DMA write starting at offset 0x00FFFF which has A16=0, then the next writes will be to 0x01000, 0x01001, 0x01002, etc, which all have A16=1. (Adjust as needed for the actual size of the command transaction of course.) None of the other address lines matter, so you just have to ensure the address line you select doesn't roll over back to 0 before you finish your largest transaction. On higher performance architectures you do have to make sure the memory range is tagged appropriately to ensure that all of the writes are issued to the memory interface and in the correct order. (Armv7m, for example, does not guarantee that writes to general memory will happen in program order, quantity, or size, so command/data bytes may get reordered without the write MPU tags -- although this may only be a problem with accesses by the core and not DMA accesses).Quote from: peter-h on June 14, 2025, 07:20:30 amApparently many of the ILI controllers seem to run quite well at much higher SPI speeds than their datasheets imply. This isn't too surprising when you see that they often specify a higher speed for the parallel interface, but I've heard of people running them well into 60MHz territory with no apparent issues. Whether you would want to rely on that for commercial production is a harder question. I don't know if the silicon has been improved since the datasheets were first drafted, they're just extremely conservative, or they assume horrible SI conditions or weak line drive in the end application orQuote from: paulca on June 14, 2025, 10:32:54 amSure, there are endless ways to optimize things depending on priorities, and optimizing for ease of development/iteration is a valid choice. If your priority is minimizing the data that needs to be handled over a low-throughput channel, then transmitting relatively high level graphics commands like you describe is probably the best strategy. I assumed you were running the display from the ESP32 -- that would seem to simplify things quite a bit, and remove the limitation imposed by the serial link between it and the F411, but presumably you have reasons for doing it that way.

author: paulca
author_rank: Super Contributor
timestamp: June 14, 2025, 10:10:01 pm
content: Quote from: ajb on June 14, 2025, 06:23:22 pmOn the board I made, the STM32 is the "MCU" the ESP32 is just treated as a comms module. The default (factory) comms module firmware even supports MQTT out of the box.I just got bored writing UIs in C.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 14, 2025, 10:48:57 pm
content: My use isn't that different from peter-h's, except that I use USB to connect the display (and perhaps some buttons, encoders) to a Linux host.Note thatSipeed LicheeRVNano-E has an 10/100 Ethernet connector, and Nano-W a dual-band WiFi 6 + BT module, and Nano-WE both. It could be used as a networked display unit with a 2-lane MIPI DSI display (31-pin connector; pinout is in the schematic; displays with OS image support use ST7701 controller), with main core running a minimal Linux userspace. The board itself is just 23mm × 36mm, so absolutely tiny.For the backlight, there are two options: PWM and DAC (current output). As noted, I didn't count the pins needed for these in my previous post. LicheeRV Nano uses an Awinic AW9962E that supports PWM at up to 100 kHz, with two resistor positions to control the current (R=0.2V/I).Of BuyDisplay IPS panels using "4-wire" SPI (select, clock, data, D/C),2.79" 142×428uses the NV3007 controller, which can go up to 100 MHz SPI write clock (10ns write cycle); read clock maxes out at 6.67 MHz (150ns read cycle), so display identification and other read operations have to be done with a low SPI clock. It samples D/C at the same time as eighth bit in each byte is transferred (so command bytes cannot be DMA'd; be careful with SPI write buffer), and supports 16-bit RGB565 and 24-bit RGB666 (two least significant bits of each byte ignored). It does not require a stable or symmetric clock, and is happy as long as each state is at least 5ns long. The TE pin (retrace) is available and triggering line should be adjustable. The display contains 60,776 pixels, thus full 16-bit color framebuffer is 121,552 bytes, 24-bit 182,328 bytes. I don't know its native refresh rate, but tear-free full-display updates are definitely possible if your SPI can do about 75 MHz (142×428×16×77 Hz < 75 Mbit/s); I think it likely at 60 MHz, but might be possible at as low as 30 MHz, depending on the refresh rate, as "35ms typical response time" does not exactly tell.Quote from: ajb on June 14, 2025, 06:23:22 pmThose lack sufficient RAM for a random-access framebuffer.For the command/data parallel bus, one suitable option is a peripheral like RP2040/RP2350 PIO (but their USB is limited to FS, 12 Mbit/s), or NXP FlexIO, which can generate the /WR strobes (one low pulse per output word) and control the D/C signal.When bit-banging this, GPIO with TOGGLE support is quite useful (and SG2002 RISC-V cores do not have these, but NXP i.MX RT1062 Cortex-M7 does). There, the color lookup and GPIO pin mapping can be combined, allowing any mapping between output color bits to pins in the same GPIO bank (or you can use more than one bank), without affecting any other GPIO pins in the same bank. The trick is that if you keep the state of the affected output pins inold, with others zero, and the desired new state of them isnew, then one simply calculatesset=old^new, copiessettoold, and writessetto the TOGGLE register.Without GPIO TOGGLE register, you need to keep a shadow copy of the GPIO bank output states, and do a couple of ANDs and an OR/XOR per bank.Many MCUs/processors do have a dedicated LCD peripheral for continuously generating the display data (9/12/15/16/18/24 RGB lines, dot clock, data enable, vsync, hsync), although the existing developer boards like Teensy 4.x (for NXP i.MX RT1062, MIMXRT1062) do not expose all required pins. BuyDisplay2.4" 240×320,2.8" 240×320,3.5" 320×480,4.3" 480×272,4.3" 800×480,5" 800×480,7" 800×480,7" 1024×600, and10.1" 1024×600IPS display panels all support this protocol. If we add a SPI interface for configuration,BuyDisplay has many more optionsin various aspect ratios, and even one2.1" 480×480 roundIPS panel with capacitive touch, which I find extremely interesting for some gadgetry.

author: DavidAlfa
author_rank: Super Contributor
timestamp: June 14, 2025, 10:53:11 pm
content: Nice, I did something similar in assembler using a PIC18F1220 to interface graphic nokia LCDs 20 years ago.But, it kinda defeats the purpose with powerful, modern MCUs.

author: paulca
author_rank: Super Contributor
timestamp: June 15, 2025, 12:33:39 pm
content: The need for "horsepower" would come from another idea I had for such a TFT.Taking the "Bouncing ball" demo into the present era, using an actual accelerometer and gyro to allow the ball to actually interact like it was stuck inside the screen when you jiggling or shake it.Then my eager FPGA learner piped up and scoffed. "One ball, pfff, why not 100 snow flakes, all independently responding with their own inertia and edge collisions."Then I wondered... if you had such a device and placed it on a speaker cone would it also draw patterns like salt would?You know why this project isn't for me? I know the very next words someone is going to say is "Particle simulation - Maths department floor 3". ... and that me out. Once it goes greek, I stop being a geek.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 15, 2025, 06:25:01 pm
content: Quote from: paulca on June 15, 2025, 12:33:39 pmIf you use the accelerometer as the source of movement: no, because the patterns come about from each particle being subjected to different forces.If you use the accelerometer as the peak, as if it measured a single point source say an inch below the device, and simulated the effects on the particles, then yes.Quote from: paulca on June 15, 2025, 12:33:39 pmNot that much math, really. It is modeling the simulation volume as the thing that moves, with particles moving with respect to the outside world, that gets most programmers; and the fact that you cannot use Tait-Bryan or Euler angles for the rotations, as this is firmly versor/unit quaternion/bivector rotation stuff.I think I could fit all the needed math on a single A4/Letter sheet (two pages) using 12pt font, laid out nicely.(The display is a window into a simulated limited-volume 3D box. It is this box that moves as the device/accelerometer/gyro moves, not the particles within the box, so you need to chain some rotations and inverse rotations to get it all right. But it is notthatmathy –– more like 3D vector stuff plus non-Euleri rotations; matrix-vector multiplication and conversion of orientation to a matrix. All fixed formulas. While it is integrating coordinates from accelerations, even that is done using either the rectangle rule (x = v×t, v = a×t) or trapezoidal rule (similar, but x and v calculated t/2 apart); former is easier, latter is slightly better. In proper simulations, much better integration methods are used, for example Gear5, which predicts jerk and snap, and integrates velocity and position.)

author: peter-h
author_rank: Super Contributor
timestamp: June 15, 2025, 06:39:48 pm
content: One could also do the D/C bit together with a parallel interface by (if using 8 bit parallel e.g.) using 16 GPIO lines (basically two CPU ports) and driving these with DMA. In the RAM buffer you will have 16 bit words, with one byte being the data and the other byte being 0x00 or 0x01, and the top 7 bits wasted.In terms, of pins, you could probably set them as inputs and get 7 x GPIO input pins.But as I said earlier, the situation is much better because in reality you use a "set address" command (4 bytes; actually 1 D/C=0 byte and 3 D/C=0 bytes) which is then usually followed by a load of "write pixel" commands which arealljust DATA i.e. D/C=1. You don't need to optimise the former. Of the latter, if say drawing a full-width line, say 480px, you set D/C=1 and squirt out 480x3 bytes with DMA. I've just done this on the Ramtex library; apparently in just two places: fill frame and fill within viewport. The fill frame is all-DMA, with set address =0 and then 320x480x3 bytes to send. In reality you can't do 320x480x3 bytes because you don't have that much RAM so I do one line (480x3 bytes) at a time and the overall time is practically the same. I don't know if there is a way to link two DMA controllers to send out 320 lines but it hardly matters.If all three bytes were the same, say 0x00 or 0xff, then you could run the DMA with a non-incrementing source address so you could do a frame fill with it... except that 32F4 DMA cannot do >64k bytesSo, yeah, three DMA blocks.If your graphics lib is doing a lot of random pixels, the above won't work and you just need a fast CPU and a fast LCD interface. But there again much depends on how the graphics is drawn.That LCD work I did surprised me when I realised the LCD controllers are mostly (all??) just crude frame buffers, with a parallel and SPI load interface one end, and a scanner for driving the LCD at the other end. They are primitive. No fonts, no rectangles, no fills, not even a frame buffer fill. So far from the chips in the 1980s like the UPD7220. I don't think they even contain a CPU.Unsurprisingly, the ILI9488 runs fine at 21MHz (15.5 is max spec) but the margin will be temperature dependent.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 16, 2025, 12:13:02 am
content: It's the cost-optimized memory interface in these: reads are about 10× slower than writes, sometimes even slower.If reads weren't so slow, you'd need minimal logic for combining (blending) writes and color index lookup (writing 1/2/4/8-bit data, expanding to full color). When you look at e.g. BuyDisplay/EastRising IPS panel prices, you realize the controllers must be very, very cheap.Adding a ridiculously powerful MCU or SoC to drive one only costs about 10€ more. These tend to have the LCD/RGB interface – RGB data, CLK, DE, HSYNC, VSYNC – for continuously refreshing even simpler panels (with no RAM at all, common in larger panels), and easily enough memory for triple buffering; often even pixel engines or blitters. They can of course still drive these parallel command/data display controllers too. Still, it is the amount of RAM and RAM bandwidth (remembering the DMA bandwidth taken by continuous scanning) that tends to limit one here. PSRAM suffices for static storage of glyphs/pixmaps/sprites/opacity maps, but for the framebuffer, you really need SRAM or fast DRAM.For ILI9341, ILI9488, ST7789VW, ST7789P3 –– so most 320×240 and 320x480 panels –– wecanuse RGB555 with high bit as the D/C, by wiring the output as RGB565 by duplicating the most significant bit of green as the new the least significant bit of green, because all but the low 8 bits are ignored for parameters when using 16-bit parallel comms. Parameter byte values just need to be encoded weirdly,pas(p& 31) | (p& 32) << 5) | ((p& 192) >> 1), the reverse of what the wiring does. The corresponding 32-bit expanded form most useful for alpha blending is│31 24│23 16│15 8│7 0│├─┼─┼─┬─┬─┬─┬─┬─┼─┬─┬─┬─┬─┬─┬─┬─┼─┬─┬─┬─┬─┬─┬─┬─┼─┬─┬─┬─┬─┬─┬─┬─┤│0 0│∘ ∘ ∘ ∘ ∘ G G G G G│∘ ∘ ∘ ∘ ∘ R R R R R│∘ ∘ ∘ ∘ ∘ B B B B B│└─┴─┼─┴─┴─┴─┴─┼─┴─┴─┴─┴─┼─┴─┴─┴─┴─┼─┴─┴─┴─┴─┼─┴─┴─┴─┴─┼─┴─┴─┴─┴─┤│29 25│24 20│19 15│14 10│9 5│4 0│unpacking is done usingu32= (u16& 0x7C1F) | ((u16& 0x03E0) << 15),packing usingu16= (u32& 0x7C1F) | ((u32>> 15) & 0x03E0),which leaves the high bit clear; and blending betweena32(p=0) andb32(p=32) is done usingu32= ( ( (32-p)*a32+p*b32+ 0x01004010 ) >> 5) & 0x01F07C1FThe most common way to expand 5-bit opacity to 0..32 is to increment it whenever the fifth bit is set, skipping the midpoint value (16), i.e.p+(p>>4)orp+(p>=16). The other options are to skip either the 0% opacity or the 100% opacity. Many developers skip the 100% opacity without realizing it, as 96.875% is visually almost identical to 100% at just 5 bits per color component.Let's say you have a 320×480 capacitive touch screen, and have found that users would prefer to have a visual indication when they're poking the display, and have the action occur only when they take off their poky bit; and let's say you discover this indicator needs say four frame animation at a 64×64 pixel size.One way to do that without modifying your 16-bit truecolor framebuffer data (which may be costly to re-generate) is to dynamically compute the update region.Furthermore, let's say that the indicator is single-color, and you only really need four opacities per pixel: 00=0%, 01=33% (11/32), 10=66% (22/32), 11=100% (32/32).If you use 64×64 = 4096 bytes for the animation, so that first frame is bits 0-1, second bits 2-3, third bits 4-5, and fourth bits 6-7, you only need four expanded pre-multiplied pre-rounded RGB555 values and the correspondingopacityvalues (pre-substracted from 32), with their lookup replacing one multiplication and one addition in the blending formula. If you consider the intersection of the old ("erased") region and the new region, both 64×64 pixels, their intersection is one of three cases: 1) single 64×64 rectangle, 2) two separate 64×64 rectangles, 3) three rectangles whose total area is at most 2×64×64 pixels. So, we know that we need 2×64×64×16-bit DMA buffer + overhead for three rectangular region sets, say about 16428 bytes, and we'll do one 64×64-pixel copy (source and target only 16-bit aligned!), and 64×64=4096 blend operations.What sounded like a lot of computation became easy-peasy.Now, since such indicators tend to be X and Y symmetric, youreallyonly need something like one 16×64 or 32×32 quadrant, drawn four times in different orientations (perhaps even adjusting their distances from the center point for added animation), and the rectangle overlap combination is a tiny bit more complicated (when joining the old four erased regions and the new four drawn regions as a DMA'ble set of buffers), but still the same amount of blend operations. If you expand the rectangles primary axis coordinates to even values, then you know it is an even number of pixels wide, all even pixels are at 32-bit aligned addresses, and your "clear" is a 32-bit aligned copy from existing framebuffer (per sub-rectangle), two pixels at a time, at the cost of few (up to 2x2×64, but about 64 on average) extra pixels transferred to the display.Same for "busy" indicators et cetera. If you use 4 bits per pixel, so two separate images or frames in the same byte region, you can use two colors (say foreground and shadow) with alpha, using indexi=fg+4×sh, wherefg=0..3 are the 0%/33%/67%/100% opacities of the foreground, andsh=0..3 are the 0%/33%/67%/100% opacities of the shadow. You still only need 16 lookup values (32-bit expanded pre-multiplied RGB55 colors and 8-bit opacities). Or you can just draw them in Gimp, save originals as 32-bit RGBA PNGs, then quantize the PNGs to indexed color with 16 entries; each entry contains one specific color at one specific opacity. I use Gimp, ImageMagick, NetPBM tools, and pngcrush for automating such quantization and formatting. This works particularly well for overlaid text with a dark boundary/outline to ensure it is visible against the background. The lowest opacity of the outline can also be used as a slight shadow, to have the text pop out a bit more. But ask a graphic artist and discuss with an UX specialist as to what works best, since my UX-fu is not up to the latest trends of what is "cool".

author: NorthGuy
author_rank: Super Contributor
timestamp: June 16, 2025, 02:23:20 am
content: Certainly, similar things have been done before. Once you add more features, you will end up re-inventing X sever:https://en.wikipedia.org/wiki/X.Org_ServerEven though it's used in Linux, the protocol does not require Linux. It might be very helpful to use Linux as a test bed.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 16, 2025, 06:51:59 am
content: As an auxiliary display to my own Linux appliances and SBCs, I actually use the display controller also as a sort of a watchdog, communicating with an userspace status daemon (simple, written in C, but may execute external scripts et cetera to gather status information every few seconds). The display acts as a separate simple UI device displaying some of that information in a non-technical-human oriented way. I do not use these for any kind of control or configuration interfaces, though; it is rather important for many of my use cases that it is strictly one-way, and nontechnical humans poking at the display won't affect the operation of the appliance.So, in my case, I am not tending towards X11 protocols at all. I'll likely have just a simple formatted text stream provided by that watchpuppy daemon. I do like the idea of high-speed USB, though, because then I could stream video to the display (using e.g.ffmpegdecoding on the host, raw uncompressed RGB565 stream over USB). 320×240@30fps is 4,608,000 bytes/second, or 36,864,000 bits/second, and no problem at all even over USB Serial on high-speed USB devices. You know, say using a dedicated USB serial endpoint, a different one to the watchpuppy daemon text stream.I might very well end up running a Linux kernel and minimal Linux userspace on the big RISC-V core on CV1008B/SG2002, and just the rasterizer/renderer stuff on the smaller RV64GC core, though. Don't know yet; technically, even Teensy 4.x have enough memory and computational power for my displays (320×240 and 480×320).As to what paulca is doing, I do suspect that they, too, will find that just providing the data to the display controller, and having the controller render it into nice graphics, may be a better approach than creating a protocol for doing graphics layout on the host and just render on the display like X11 protocol does.The interesting middle ground would be to implement an SVG renderer (see e.g.nanosvg), and have the host send the initial SVG state, then later on inserting/deleting/replacing SVG elements (nodes) based on IDs as needed. For example, a gauge needle would be just a single<path ... />element. Text can be rendered as paths or use an actual TrueType/OpenType renderer (see e.g.pango). All this is easy to mock up using a browser an JavaScript because browsers implement a full SVG renderer nowadays.I've used SVG for well over a decade now –– evenmy home pageuses only embedded SVG for the visuals –– so it definitely allows as high fidelity graphics as one might want. Use Inkscape to create graphics; its "Inkscape SVG" format contains all additional metadata Inkscape needs like grids and such, but the "Plain SVG" and "Optimized SVG" formats produce pretty well simplified SVG code you can edit, examine, and understand in any text editor.

author: peter-h
author_rank: Super Contributor
timestamp: June 16, 2025, 09:55:20 am
content: Is there any market for a box which has a 10/100 RJ45, LWIP, and an LCD? With some sort of language, which could be C, Python, etc?I know; it is called a PC

author: paulca
author_rank: Super Contributor
timestamp: June 16, 2025, 10:37:31 am
content: Quote from: peter-h on June 16, 2025, 09:55:20 amOr a phone or umpteen 1000 embedded linux systems with 800Mhz+ RISC-V processors and 8Gb of RAM for <£200.By the time you buy a few dev boards and print a few prototype PCBs, could probably have bought a mini-not-quite-embedded RISC-V platform and a secondhand 4k monitor.The only real reason MCU screen exist is because the above costs £200-300 a product pop. The MCU+TFT costs a tenth that or a 20th. I want to do it for fun though.EDIT: Key distinction in my model ... The "big iron" server side of this is a (relatively) tiny foot-print which could easily serve hundreds of screens at once, multi-threaded and barely be noticed on the metrics.QuoteInteresting concept, however, is it (still?) XML-centric? ie. Verbose self defining markup which needs to be heavily compressed to "ship"?

author: paulca
author_rank: Super Contributor
timestamp: June 16, 2025, 11:30:36 am
content: Going one step further on abstractions...UIs are a pain in the back side regardless of where and how. So ... there are tool kits everywhere. How do I leverage them?There are toolkits which generate intermediary UI definitions. There are tool kits which generate lower level code like C/C++ from said definitions. At some point/layer there might be a useful hook to generate "LCD Driver primitives" instead of QTComponents - for example.The subset I would need is limit to, pages, containers, text boxes. I don't even need a Z plane if I design the need out. "Modal" or just "Alert" popups would be handy, but not essential. I need none of the external event handlers (slots/signals/listeners) just the internal "invalidate and redraw" events that propagate to redraw only what has been changed. Without a Z plane that is rather easy.It loops back to FusionGfx really when you go that far.

author: peter-h
author_rank: Super Contributor
timestamp: June 16, 2025, 02:00:41 pm
content: Are you implementing a touch screen? It's not exactly trivial.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 16, 2025, 02:03:58 pm
content: Sipeed LicheeRV Nano-E is 23mm × 32mm dual-core RV64GC RISC-V with 256MB RAM with main core running at 1 GHz, second core running at 700 MHz, a 10/100 Ethernet port, a 31-pin two-lane MIPI DSI 0.5mm FPC for a display, 6-pin I²C FPC for capacitive touch, and an SD card (or SD NAND) for booting from. The SoC is a SophGo SG2002 SoC. It can run Debian Linux, but there are also e.g. Buildroot recipes. The toolchain is a standard GNU 64-bit RISC-V one. The second core usually runs a RTOS, and there are guides on doing that, but you can also program the entire Soc on "bare metal" as a microcontroller also. It costs about 20€ shipped from official Sipeed Aliexpress store. Their schematics are available on the net from Milk-V and Sipeed, too. There are other similar SG2002-based tiny Linux boards available at similar price.OpenWRT runs Linux on devices with just 64M of RAM, so 256M should definitely suffice for a HMI box. One might program the second core as a RTOS blitter/rasterizer/renderer in C.For about 35€ shipped, you can get aRadxa Zero 3Ewith 1G of RAM, 4-core Cortex-A55 @ 1.6 GHz, micro-HDMI supporting up to 1080p@60fps (via Mali G52 OpenGL ES acceleration for e.g. Python + Qt fullscreen use with a minimal userspace without X or Wayland), GbE RJ45, and some GPIO pins. OS still needs to be on an microSD card. This is sufficient for a highly polished HMI, where the UI is written in Python on top of Qt on raw DRM framebuffer (OpenGL accelerated Qt).For about 150€ shipped, you can get anOrange Pi 5+with 16G of RAM, 4×Cortex-A77@2.4GHz + 4×Cortex-A55@1.8GHz, Mali G610 GPU (OpenGL ES), 2×HDMI out (8K@60 decoding), ES8388 audio codec, 4-lane M.2 PCIe 3.0 2280 for storage, M.2 E-key for WiFi/BT if you want to add that, some USB ports, IR receiver, mic/headphone connector, 4-lane MIPI DSI for LCD (including capacitive touch), and some GPIO pins (exposed to Linux userspace, pin functions configurable via Device Tree). Oh, and two 2.5 GbE RJ45's, too. The RK3588 has PassMark score of 4530 compared to e.g. Intel N97 (4C/4T, 3.6 GHz) score of 5621, and similar per-core difference, so you can roughly consider RK3588 as 80% of the speed of a 12W Intel N97. You can run Debian, Arch, Ubuntu, or Android on it; OrangePi try hard to be open-source friendly.It's really the RAM and the display that cost, not the SoC/SBC needed. I only use IPS and similar panels with very wide viewing angles, but even those in 4.3" with capacitive (multi-)touch can be had for ~ 30€ (parallel RGB or MIPI DSI). BuyDisplay8" 1280×800 IPScosts around 50€ with capacitive touch. Larger HDMI-connected displays with capacitive touch are easily 300€+ here.What annoys me with Milk-V Duos and Sipeed LicheeRV Nanos is that I'd prefer to use them with ILI9341/ILI9488/ST7789 and similar small display controllers using a parallel 16-bit bus (plus /CS, /WR, /RD, D/C, and TE input), but the FPC connectors just aren't compatible. However, SG2002 also contains a Designware 8051 core (that can run at up to 300MHz), that I might use as a glorified DMA/PIO engine, controlling arbitrary GPIO pins as needed. I will need to fiddle with the DeviceTree overlays (to disable the CSI and DSI interfaces as they use a contiguous bank of pins), and somehow try to jumper those pins to my own FPC connector.

author: paulca
author_rank: Super Contributor
timestamp: June 16, 2025, 04:47:33 pm
content: Quote from: peter-h on June 16, 2025, 02:00:41 pmI know. So only for "press" to cycle pages. Nothing else

author: Doctorandus_P
author_rank: Super Contributor
timestamp: June 19, 2025, 10:29:44 am
content: Every now and then I am toying with a very similar idea, but I'm having huge problems with getting a grip on STM32 and progress is extremely slow. I bought "Blue Pills" when they were still real. I think I'm sort of stuck on the "blinking led" level for 8+ years.For the TFT. It seems logical to me to make it act like a web browser. So with the node with the TFT panel, you connect to some other node, and that other node generates a valid HTML web page. (ESP32 is often already used for this). There is no way that a full blown web browser fits into an STM32, so it would only support a very limited sub set of HTML...But HTML has become quite complicated too. For making personal notes, I have been experimenting with Asciidoc lately, and it seems to work quite nice. I also have installed a plugin in Firefox, so it can directly render Asciidoc documents.I also want to mention the Nextion displays. These have an on board STM32 and also have some sort of command interface. They are bundled with PC software to create a GUI by clicking and dragging. I think this is Windoze only and that means I'm out, but once I have some real STM32 projects going, I would have to verify that.And as mentioned before, it's a bit of a niche project. Full blown Linux PC's which have no trouble with rendering HTML start at around EUR 15 these days (The "compute modules"). And combined with a relatively big TFT, it's never going to be very power efficient. However, with an STM32, you have much direct control over power save modes and such, and you don't have to boot a whole OS. For a battery powered remote control it may still be interesting. It can probably go from "deep sleep" to "fully on with rendered TFT" in about a second.Another idea on this line I sometimes think about is to use a bog standard E-book reader as a remote control device. The "pocketbook" brand is pretty open, it does not hide they are using Linux as OS and they have a github repository with some example programs (Chess, Sudoku, a simple drawing program, some more). But it's also been years since I last checked it.Edit: I just saw the link below (Have not read it yet, no time right now). It's of a similar / related project.https://hackaday.com/2025/06/18/zpui-could-be-your-tiny-embedded-gui/