Building UIs on Embedded - Page 1
Forum section: electronics / general

author: don.key
author_rank: Newbie
timestamp: October 21, 2015, 10:23:30 am
content: Hi Folks,I am a experienced web developer who has recently moved into Embedded with the idea to make some IoT things. It was quite overwhelming, fascinating and I felt like I went 25 years back in time to when I used to hack in C and Assembly.One thing which still confuses hell out of me is user interface. I understand how to access display in it's most basic way (I once wrote Pong in Assembly), I know some very basic, low level libraries such as uGFX but those only allow very very very basic stuff, UIs made this way would either look like 80es interfaces or require enormous amount of work to make them good looking.Is there anything in low end (Cortex M level) Embedded world which is kind of like Bootstrap for web? Nice looking widgets / buttons, scrollbars etc which I can apply to the screen?How about drawing charts?I saw some products such as TouchGFX and Embedded Wizard. Are those products the only options I have if I want to make a nice GUI in resonable timeframe?Thanks!!

author: __daz__
author_rank: Newbie
timestamp: October 21, 2015, 11:26:06 am
content: You may want to give a look to emWin libraries from Segger

author: ajb
author_rank: Super Contributor
timestamp: October 21, 2015, 04:51:19 pm
content: Haven't used them myself, but 4D Systems make some LCD modules with embedded controllers that are supposed to be really easy to program. If you're doing something low volume or just need to do a quick mockup, the savings on development time may well offset the additional hardware cost versus other solutions with a less intelligent LCD.

author: matherp
author_rank: Contributor
timestamp: October 21, 2015, 05:48:53 pm
content: If you want a really easy way in to embedded GUI development, Geoff Graham is currently in the late stages of beta testing the Micromite+ (firmware available on the Backshed forum).This is a PIC32MX470 loaded with a very sophisticated Basic interpreter that also allows calling compiled C routines where ultimate speed is required. The Micromite firmware includes support for a range of TFT displays including the SSD1963 and ILI9341 as well as support for their touch controllers and SDcards. I've attached a summary of the Basic commands for GUI development below. In addition there are all the usual drawing primitives BOX, CIRCLE, etc.The combination of interpreted Basic with compiled C and extensive GUI support makes writing embedded applications very easy. Minimum hardware is the processor chip, a crystal, a resistor and a few capacitors. Once the Micromite firmware is programmed onto the MX470 then Basic programming is via a standard TTL UART or there is also on-chip support for USB.Don't be put off by Basic. This is an incredibly powerful development environment for embedded applications and it is all FREE!!!Demo of some of this onhttps://www.youtube.com/watch?v=BkJ2WbzO6dI&feature=youtu.be______________________________________________________________________________________________________The advanced graphics controls are:FrameGUI FRAME #ref, caption$, StartX, StartY, Width, Height, ColourThis will draw a frame which is a box with round corners and a caption. A frame does not respond to touch butis useful when a group of controls need to be visually brought together. It can also used to surround a group ofradio buttons and MMBasic will arrange for the radio buttons surrounded by the frame to be exclusive – that is,when one radio button is selected any other button that was selected and within the frame will be automaticallydeselected.LEDGUI LED #ref, caption$, CenterX, CenterY, Radius, ColourThis will draw an indicator light (it looks like a panel mounted LED). When its value is set to a non zeronumber it will be illuminated and when it is set to zero it will be off (a dull version of its colour attribute). Thecaption will be drawn to the right of the LED and will use the colours set by the COLOUR command. A LEDdoes not respond to touch.Check BoxGUI CHECKBOX #ref, caption$, StartX, StartY, Size, ColourThis will draw a check box which is a small box with a caption. Both the height and width are specified withthe 'Size' parameter. When touched an X will be drawn inside the box to indicate that this option has beenselected and the control's value will be set to 1. When touched a second time the check mark will be removedand the control's value will be zero. The caption will be drawn to the right of the Check Box and will use thecolours set by the COLOUR command.Push ButtonGUI BUTTON #ref, caption$, StartX, StartY, Width, Height, FColour, BColourThis will draw a momentary button which is a square switch with the caption on its face. When touched thevisual image of the button will appear to be depressed and the control's value will be 1. When the touch isremoved the value will revert to zero. Caption can be a single string with two captions separated by a |character (eg, "UP|DOWN"). When the button is up the first string will be used and when pressed the secondwill be usedSwitchGUI SWITCH #ref, caption$, StartX, StartY, Width, Height, FColour, BColourThis will draw a latching switch with the caption on its face. When touched the visual image of the button willappear to be depressed and the control's value will be 1. When touched a second time the switch will bereleased and the value will revert to zero. Caption can be a single string with two captions separated by a |character (eg, "ON|OFF"). When this is used the switch will appear to be a toggle switch with each half of thecaption used to label each half of the toggle switch.Radio ButtonGUI RADIO #ref, caption$, CenterX, CenterY, Radius, ColourThis will draw a radio button with a caption. When touched the centre of the button will be illuminated toindicate that this option has been selected and the control's value will be 1. When another radio button isselected the mark on this button will be removed and its value will be zero. Radio buttons are grouped togetherwhen surrounded by a frame and when one button in the group is selected all others in the group will bedeselected. If a frame is not used all buttons on the screen will be grouped together.The caption will be drawn to the right of the button and will use the colours set by the COLOUR command.Display BoxGUI DISPLAYBOX #ref, StartX, StartY, Width, Height, FColour, BColourThis will draw a box with rounded corners. Any text can be displayed in the box by using the CtrlVal(r) =command. This is useful for displaying text, numbers and messages. This control does not respond to touch.Text BoxGUI TEXTBOX #ref, StartX, StartY, Width, Height, FColour, BColourThis will draw a box with rounded corners. When the box is touched a QWERTY keyboard will appear on thescreen. Using this virtual keyboard any text can be entered into the box including upper/lower case letters,numbers and any other characters in the ASCII character set. The new text will replace any text previously inthe box.The value of the control can set to a string starting with two hash characters (##) and in that case the string(without the leading two hash characters) will be displayed in the box with reduced brightness. This can beused to give the user a hint as to what should be entered (called "ghost text"). Reading the value of the controldisplaying ghost text will return an empty string. When the control is used normally the ghost text will vanish.MMBasic will try to position the virtual keyboard on the screen so as to not obscure the text box that caused itto appear. A pen down interrupt will be generated when the keyboard is deployed and a key up interrupt willbe generated when the Enter key is touched and the keyboard is hidden.Number BoxGUI NUMBERBOX #ref, StartX, StartY, Width, Height, FColour, BColourThis will draw a box with rounded corners. When the box is touched a numeric keypad will appear on thescreen. Using this virtual keypad any number can be entered into the box including a floating point number inexponential format. The new number will replace the number previously in the box.Similar to the Text Box the value of the control can set to a literal string with two leading hash characters (eg,"##Hint") and in that case the string (without the leading two characters) will be displayed in the box withreduced brightness. Reading this will return zero and when the control is used normally the ghost text willvanish.MMBasic will try to position the virtual keypad on the screen so as to not obscure the number box that causedit to appear. A pen down interrupt will be generated when the keypad is deployed and a key up interrupt willbe generated when the Enter key is touched and the keypad is hidden. Also, when the Enter key is touched theentered number will be evaluated as a number and the NUMBERBOX control redrawn to display this number.Spin BoxGUI SPINBOX #ref, StartX, StartY, Width, Height, FColour, BColour, Step,Minimum, MaximumThis will draw a box with up/down icons on either end. When these icons are touched the number in the boxwill be incremented or decremented by the 'StepValue', holding down the touch will repeat the step at a fastrate. 'Minimum' and 'Maximum' set a limit on the value that can be entered. 'StepValue', 'Minimum' and'Maximum' are optional and if not specified 'StepValue' will be 1 and there will be no limit on the numberentered. A pen down interrupt will be generated every time up/down is touched or when automatic repeatoccurs.CaptionGUI CAPTION #ref, text$, StartX, StartY, Justify, FColour, BColourThis will draw a text string on the screen. 'Justify' is one or two letters where the first letter is the horizontaljustification around X and can be L, C or R for LEFT, CENTER, RIGHT and the second letter is the verticalplacement around Y and can be T, M or B for TOP, MIDDLE, BOTTOM. The default justification is left/top.This command is similar to the basic drawing command TEXT, the difference being that MMBasic willautomatically dim this control if a keyboard or number pad is displayed.If the colours are not specified this control will use the colours set by the COLOUR command.Interacting with ControlsUsing the following commands and functions the characteristics of the on screen controls can be changed andtheir value retrieved.? = CTRLVAL(#ref)This is a function that will return the current value of a control. For controls like check boxes or switches itwill be the number one (true) indicating that the control has been selected by the user or zero (false) if not.For controls that hold a number (eg, a SPINBOX) the value will be the number (normally a floating pointnumber). For controls that hold a string (eg, TEXTBOX) the value will be a string. For example:PRINT "The number in the spin box is: " CTRLVAL(#10)? CTRLVAL(#ref) =This command will set the value of a control. For off/on controls like check boxes it will override anytouch input and can be used to depress/release switches, tick/untick check boxes, etc. A value of zero is offor unchecked and non zero will turn the control on. For a LED it will cause the LED to be illuminated orturned off. It can also be used to set the initial value of spin boxes, text boxes, etc. For example:CTRLVAL(#10) = 12.4? GUI FCOLOUR colour, #ref1 [, #ref2, #ref3, etc]This will change the foreground colour of the specified controls to 'colour'. This is especially handy for aLED which can change colour.? GUI BCOLOUR colour, #ref1 [, #ref2, #ref3, etc]This will change the background colour of the specified controls to 'colour'.? = TOUCH(REF)This is a function that will return the reference number of the control currently being touched. If no controlis currently being touched it will return zero.? = TOUCH(LASTREF)This is a function that will return the reference number of the control that was last touched.? GUI DISABLE #ref1 [, #ref2, #ref3, etc]This will disable the controls in the list. Disabled controls do not respond to touch and will be displayeddimmed. The keyword ALL can be used as the argument and that will disable all controls. For example:GUI DISABLE ALL? GUI HIDE #ref1 [, #ref2, #ref3, etc]This will hide the controls in the list. Hidden controls will not respond to touch and will be replaced on thescreen with the current background colour. The keyword ALL can be used as the argument.GUI RESTORE #ref1 [, #ref2, #ref3, etc]This will undo the effects of GUI DISABLE or GUI HIDE and restore the controls in the list to fullvisibility and normal operation. The keyword ALL can be used as the argument for all controls.? GUI REDRAW #ref1 [, #ref2, #ref3, etc]Will redraw the controls on the screen. It is useful if the screen image has been corrupted. The keywordALL can be used as the argument for all controls.? GUI DELETE #ref1 [, #ref2, #ref3, etc]This will delete the controls in the list. This includes removing the image of the control from the screenusing the current background colour and freeing the memory used by the control. The keyword ALL canbe used as the argument and that will cause all controls to be deleted.? GUI DEFAULT HIDDENGUI DEFAULT SHOWThis will set the default state of newly created controls to hidden (ie, they will not be displayed on thescreen until GUI RESTORE is used). This is useful when creating controls that will be later made visibledepending on the program logic. The SHOW setting can be used to restored the original behaviour.

author: Maxlor
author_rank: Frequent Contributor
timestamp: October 21, 2015, 10:23:52 pm
content: When I was wondering about the same thing a few years ago, I eventually concluded that there is no high level GUI lib that is suitable for me. I was using an ARM926 chip and a small 160x128 LCD, but no dedicated graphics chip. The reasons I couldn't find an existing GUI lib are:- There is no standard look for embedded devices. Each has its own look and feel, which is often specified by your corporate design guidelines.- Interaction capabilities vary widely (capacitive touch screen? Maybe just a resistive touch screen where you can't swipe? Soft buttons? Physical controls?), as does screen resolution.- To get good performance, you can't abstract too much, you need to keep the hardware capabilities in mind. One simple example is the bit format for pixels in the frame buffer. It should be the same as what the display uses in the end (and there is no common standard among displays), otherwise the per-pixel format conversion is going to slow you down a lot, because often you don't have a graphics card that does it for you, you'd need to do it on the CPU.Note: if you go with a more capable platform, such as a GHz class ARM chip with embedded graphics, things are different. At that point, you're probably using an OS, 3rd party drivers, and there's much less diversity in the hardware. Look at i.MX6 based boards for example, they're all quite similar from software point of view. I assume here that we're discussing something a bit smaller, without a standard graphics controller.Given that, the attempts at writing generic embedded GUI libs I've seen haven't impressed me. They're full of #ifdefs, and the abstraction layers break frequently because the underlying hardware lacks some capability that the abstraction layer requires.Therefore, I would approach the problem like this:- pick a set of helper libraries that are hardware independent, and that you can easily port to a new platform. This would include: drawing geometric shapes into a framebuffer, and something like libpng to load images, also into the framebuffer. Usually you have to add a little adapter for your specific pixel format, but it is a small addition.- Write the framebuffer-hardware interface yourself.- If you have the memory, push as much of the UI creation out of your code and into images as you can. If you're memory limited, or that workflow doesn't work well for you, there are some compromises you can do, e.g. drawing a button based on a 9-segment image.At this point, you can draw many types of GUI and you haven't invested that much time. A few hours maybe, a bit longer if you do it for the first time. That leaves capturing and acting on user input. I haven't really found a fast way to implement that yet, because the input mechanisms are so different from project to project. It's always a bit of work. I considered writing a portable generic abstraction layer that takes hardware input and spews out something like mouse/touch/key events, but I quickly got the impression that with the hardware-specific adaption I still would have to do, such an abstraction layer wouldn't save me much time, but it would increase code complexity.

author: jnz
author_rank: Frequent Contributor
timestamp: October 21, 2015, 11:35:40 pm
content: Quote from: Maxlor on October 21, 2015, 10:23:52 pmGood post.I also like the post above about emWin. Keil has a pro-middleware with good graphics handling as well, but...Honestly... I think the days of adding another screen to things is coming to an end. A lot of new popular IoT products aren't using screens (Amazon's microphone thing, GoPros, Withings scales, fitbits and lots of other stuffs).If you're starting today and looking to definitely include a screen, I'd tell you to take a step back and see if a user's pocket screen wouldn't be a better suit. You can do a lot more with a phone/tablet connection than you can an on-board screen. Just my 2c

author: senso
author_rank: Frequent Contributor
timestamp: October 22, 2015, 10:56:02 am
content: There is also uGFX.

author: Kalvin
author_rank: Super Contributor
timestamp: October 22, 2015, 11:26:27 am
content: GUI is more than writing graphical elements. GUI needs an event queue which will hold and forward the events as the result of user actions (button presses, screen tapping etc.) and device's state changes (charger connected/disconnected, timer timeouts, alarm start, alarm stop etc. A state machine (preferably hierarchical) will handle the events. Specifically, hierarchical state machine privides a nice way to propagate the events automagically to the parent handler if the focused GUI element doesn't handle the event. For more information on the hierarchical state machines, seehttp://www.state-machine.com/resources/articles.php/Creating a GUI for an embedded device can be a breeze or a nightmare, depending on the framework you are using. If you want a good one, consider rolling your own.

author: Jeroen3
author_rank: Super Contributor
timestamp: October 22, 2015, 11:36:49 am
content: Quote from: Kalvin on October 22, 2015, 11:26:27 amYou need to be a skilled developer to efficiently program and maintain a GUI framework. It is also more expensive than buying emWin.Unless you outsource it to some Asian country.There is also the new (chinese) Nextion, which offloads a lot from the application mcu.I build a small product with an M3 and Emwin. Without external memory... which leaves you with only the basic draw functions. No fancy window managers because of the little ram and slow update of the serial lcd. On it uses little power, nut on the other side it's a hell of a job to program a black/white vector graphics parameter editor thing, from basically bare metal LCD routines.

author: don.key
author_rank: Newbie
timestamp: October 22, 2015, 01:43:28 pm
content: Wow, that's some input! Thanks everyone. I really like the Geoff Graham's UI, very clean and clear, will check, although I am a bit weary of Basic, not that I doubt it's functionality but I wish my code to be reasonably portable between MCUs. Nothing speaks against using the Basic as driver for HMI display.What I understand so far is: There are few OSS and commercial options as far as I see, but regardless which way one takes, making UIs that go beyond simple buttons, menus and input fields seems to be much, much harder on Embedded, so, if one already makes connected devices it would be best to handle complex UIs via external application. I would not go as far as eliminate the screen entirely but leave it purely for status display as well as basic setup (Network for example).One more thing I wish to leave in this thread for people totally new to embedded: flashy and complex UIs require raw MCU power and fast connection to display (lots of io pins), both are precious resources one might not have on the 'main' MCU. So often displays are paired with dedicated MCUs to handle the UI, this pairing is called HMI Modules, intelligent displays etc. Such displays have their own firmware, written by hand or created with UI design tools, those modules use serial protocols to communicate events with main MCU. So main MCU does not need to track touch events or send actual image of button to display. Sometimes, MCU in display is more powerful then actual 'main' MCU. This might sound obvious but it was not really clear to me at first.Is there a list of known libraries and solutions for UI on Embedded? Here is what I found:Basic graphicsVendor independent libraries that allow drawing of primitive shapes, text and pixels on the screen. There are many of those, some famous examples are:* uGFXhttp://www.ugfx.org/* u8glibhttps://github.com/olikraus/u8glib* NuttX GFX subsystemhttp://www.nuttx.org/doku.php?id=documentation:nxgraphics* Others such as Adafruit etc.http://hackaday.com/2015/06/01/optimizing-avr-lcd-libraries/Widgets and Window Managers* NuttX widgets and WM, based on NuttX GFX subsystem, open source and freehttp://www.nuttx.org/doku.php?id=nuttx#graphics_support* emWin, commercial, prices start at 5k EURhttps://www.segger.com/emwin.htmlfree to link against for NXP MCUshttps://www.lpcware.com/content/project/emwin-graphics-libraryUI Design toolsAllow visual design of UIs, often have embedded scripting languages to control UI behavior, compile UIs into byte code.* emWin has a designer tool, commercial, prices start at 5k EURhttps://www.segger.com/emwin.html* Embedded Wizard, commercial, costs about 7k EUR,https://en.wikipedia.org/wiki/Embedded_Wizard* Mentor Graphics Embedded UI Designer, commercial, OEM of Embedded Wizard, price unknownhttps://www.mentor.com/embedded-software/nucleus/ui/* Keil Professional UI Libraries, commercial, some say this is OEM of emWin,http://www.keil.com/rl-arm/gui.asp* TouchGFX, commercial, priced starting at 3k EURhttp://touchgfx.com/* VisualTFT, commercial, priced at 99 USD, I am not sure if it requires their HWhttp://www.mikroe.com/additional-software/HMI ModulesModules with integrated MCUs *AND* custom UI Design tools that come in a bundle* Nextion, priced between 14 and 75 USDhttps://www.indiegogo.com/projects/nextion-a-cost-effective-high-performance-tft-hmi#/* 4D Systems intelligent display modules, priced between 35 and 200 USDhttp://www.4dsystems.com.au/products

author: tggzzz
author_rank: Super Contributor
timestamp: October 22, 2015, 03:08:43 pm
content: If you are after low-end embedded, you might look for products based around FTDI's EVE chip.http://www.ftdichip.com/EVE.htm

author: Rasz
author_rank: Super Contributor
timestamp: October 22, 2015, 07:15:13 pm
content: Quote from: don.key on October 21, 2015, 10:23:30 amYou arent mistaken, because current embedded products offer performance level of ~20 year old PC.Do everyone a favor and if you really need complex interface implement remote UI over BT/wifi/ethernet.

author: autobot
author_rank: Regular Contributor
timestamp: October 22, 2015, 11:59:36 pm
content: For easy solutions that enable smartphone based UI, there's blynk.cc for internet connected devices and simblee for bluetooth controlled devices.

author: Jeroen3
author_rank: Super Contributor
timestamp: October 23, 2015, 09:18:54 am
content: You can even use the audio port of a smartphone for data if BT/WiFi/USB is too expensive or not allowed on the platform (as with Apple).http://www.nxp.com/board/OM13069.htmlI've create a simple handheld parameter editor with basic STEmwin on a stm32f103. I do not recommend this, but it works. The ui is only 4000 lines long.

author: Psi
author_rank: Super Contributor
timestamp: October 23, 2015, 09:33:41 am
content: Sparkfun also sell some LCD with build in libraries for drawing gui controls.

author: VEGETA
author_rank: Super Contributor
timestamp: May 12, 2016, 05:39:38 am
content: I want to create an application controlled by FreeRTOS that has a GUI showing video from camera and other stuff. Is there a good suitable tool for this (using STM32 MCUs)? It will be a student research.

author: rstofer
author_rank: Super Contributor
timestamp: May 12, 2016, 07:49:58 pm
content: Quote from: VEGETA on May 12, 2016, 05:39:38 amProbably not. The software for displaying streaming video may be a problem.It's probably pretty simple in Linux on a Raspberry PI since you have access to all the tools and libraries.As to GUI, first you have to be able to draw stuff, then you have to be able to receive events against them, then you need to dispatch the event to a chain of handlers. In other words, a full windowing package. I remember using the ZINC library about 25 years ago to create a very nice looking 'fill-in-the-form' application. There was a ton of code behind my simple GUI but I didn't have to write it!The thing is, the entire library started out as a) draw a dot, color the dot b) draw a line, c) make the line width adjustable, d) draw a rectangle, circle and ellipse, e) fill them, f) text them and so on up the hierarchy. First you crawl, then you walk, then you run. First you draw a single pixel dot.Today, you use Raspberry PI and one of the graphics packages such as Qt or Tkinter, maybe using Python. I was playing with this a few months ago on a PC and it seemed pretty straightforward.Microsoft's IoT stuff for Win 10 is pretty neat. They replace the OS on a Raspberry PI with their own abbreviated Win 10 and then use it as a remote device. I haven't gotten very far with this but I played with it earlier this year.https://microsoft.hackster.io/en-US?page=2&sort=trendingThe question re: where is the GUI is very interesting. There is a huge advantage to leveraging a cell phone for the GUI and leaving the details to the IoT device. Google makes the Android SDK available so creating apps isn't all that hard. Apple makes something available as well but you have to register as a developer.

author: VEGETA
author_rank: Super Contributor
timestamp: May 12, 2016, 08:07:18 pm
content: Quote from: rstofer on May 12, 2016, 07:49:58 pmSo you mean that I divide my application into 2 control environment: GUI (Pi...) and Main controller (STM32 with FreeRTOS).using freertos is essential to me, so is making a gui. the idea is about a multi-robot controller that monitors each of them (mobile robots) and has the ability to control anyone or leave it autonomous... kinda like that.

author: rstofer
author_rank: Super Contributor
timestamp: May 12, 2016, 08:58:24 pm
content: Quote from: VEGETA on May 12, 2016, 08:07:18 pmNo, you can do the GUI wherever you want, with several thousand lines of code. Draw a dot, draw a line, draw a rectangle, etc. Hopefully you can find a library. That's just drawing shapes on a screen. If you don't get too fancy, like 3D controls, the code may be a reasonable amount. Now you have to write code to detect an event and somehow dispatch it to the proper place in FreeRTOS. That wouldn't be hard if functions were waiting on events but generally, they aren't. Events are asynchronous. And your code has to determine which form got the event, unless you only have one form. You have to cause some kind of event interrupt to propagate through a bunch of non-idle functions looking for a handler. Add another couple of thousand lines of code... Forms overlaying forms? Where do you store the non-displaying part of the screen, the part that's overwritten? Drop down menus? Buttons, knobs and dials? Resizable? No GPU?I would try to do the UI on a much higher level system/board. Yes, I would use a Raspberry PI for the UI functionality and I would use it to display video - another huge undertaking to code from scratch. Then I would use a WiFi dongle to connect to the end device. That leave the STM32F doing what it can do best - toggling pins. Of course, the Raspberry PI can do the same thing but it tends to be more limited than a dedicated microcontroller particularly in terms of pins and real-time response. FreeRTOS will be much more responsive simply because there isn't as much code involved with swapping tasks.I haven't seen too many projects that use the feature but the Raspberry PI comes with a camera interface.Maybe I would take a hybrid approach and use some kind of microcontroller, maybe the STM32F to handle the mechanics of the project, whatever they may be. Switch inputs, motor outputs, whatever. There are lots of pins for talking to the outside world!Then I would stack a Raspberry PI on top to handle everything else. The PI could handle the UI, networking, camera, high level decision making and the two processors could communicate over SPI. Invent a simple language for communicating.Obviously, this isn't the cheapest possible solution. A Blackfin would provide that. But it balances the strengths of platforms and the amount of original code that needs to be written from scratch.

author: tggzzz
author_rank: Super Contributor
timestamp: May 12, 2016, 09:23:42 pm
content: Look at FTDIs EVE product.

author: nctnico
author_rank: Super Contributor
timestamp: May 12, 2016, 09:44:56 pm
content: Quote from: tggzzz on May 12, 2016, 09:23:42 pmIIRC some forum members where not very pleased with it.

author: engineer_in_shorts
author_rank: Regular Contributor
timestamp: May 12, 2016, 10:05:18 pm
content: Hi,I have built a couple of products using Segger emWin and an open source u8g. These projects can become quite complex quickly, a couple of the larger issues i had:1. Say you want to add bitmaps and so on, you need to add external flash to the micro controller and deal with executing code from external flash. Fine, but does take a little while to get the code production ready.2. Then with the larger bitmaps wanted by the marketing department you get a comment that the whole screen loads rather slowly. Even at QVGA resolution, SPI connection is not fast. So you then connect the display via parallel and want to use DMA.....3. Then you find the exactly display is not longer available (they have very short production runs in china) so for an equivalent display you need to spend time changing the driver code for the new glass.. GrrrActually now i have a complex front end to make and came to the conclusion that Win 10 IoT is a very good option for the following reasons:1. The UI is built using world class tools, Visual Studio 2015 has all of the features you could want to build a UI. Much better then any embedded IDE2. The UI is built using XAML and supports asynchronous actions and events3. Any of the code can be compiled to also be accessed with a PC app - so if you have to make a PC app with common logic you can use exactly the same code and therefore development and unit testing etc is only done once.4. The XAML controls are completely customizable. Often with GUI toolkits all the widgets have a fixed look.5. Win 10 IoT is on a couple of different boards/chips so there is some spread of sources6. Tools and GUI is free. back in the day we paid a ton of money for embedded GUI7. XAML GUI is very web orientated way of thinkingIncidently I have tried the 4D Systems displays. Although they aren't too bad the tools to build the GUI is very painful and crashy, debug cycle was slow.

author: dannyf
author_rank: Super Contributor
timestamp: May 12, 2016, 10:06:58 pm
content: Emwin is probably the best known and widely used too.I would, however, suggest that you take a different approach. The mkt is really bifurcating, with products like emwin on the low end and os-based systems plus SOC on the high end. The SOC approach has considerably lower cost, both in terms of software development and deployment, especially going into the future.So as SOC prices decline - thanks to proliferation of smart phones - the sco approach may become more and more appealing.

author: rstofer
author_rank: Super Contributor
timestamp: May 12, 2016, 10:31:07 pm
content: Quote from: dannyf on May 12, 2016, 10:06:58 pmWell, they certainly don't give that away! Not too many hobbyists are going to spend 9k Euros for a GUI library. Nevertheless, if I had a commercial project, I would consider it.QuoteThe goal is to not write code. Find something that will do the job without all the effort. At least for hobbyists... Even in the commercial end, writing code is expensive. Maintaining code is even more expensive.Win 10 IoT is a very interesting approach. I need to get back to playing with it.

author: tggzzz
author_rank: Super Contributor
timestamp: May 12, 2016, 10:37:16 pm
content: Quote from: nctnico on May 12, 2016, 09:44:56 pmBut is it of any use to the OP?

author: nctnico
author_rank: Super Contributor
timestamp: May 12, 2016, 10:49:16 pm
content: Quote from: rstofer on May 12, 2016, 10:31:07 pmRead more carefully: some microcontroller vendors already paid the license for you so it helps to look whether you can use a commercial library royalty free before looking at writing your own. AFAIK NXP offers several commercial libraries for use with their microcontrollers.

author: westfw
author_rank: Super Contributor
timestamp: May 13, 2016, 01:27:44 am
content: The embedded world is sort-of slowly adjusting to the idea that the "display" they might use for a UI (ie a smartphone or tablet) is a much more computationally able device than the "embedded" part of the system...

author: Jeroen3
author_rank: Super Contributor
timestamp: May 13, 2016, 05:28:15 am
content: That is because the hardware is not the determining price. Software is.I could take more than 100 hours to build your own solid graphical lib. Which will justify buying a module until you're looking for +10k units to sell.

author: nctnico
author_rank: Super Contributor
timestamp: May 13, 2016, 06:11:30 am
content: Quote from: westfw on May 13, 2016, 01:27:44 amThe problem with hopping onto a mobile device bandwagon is that specifications and software change about every year so your device is likely to become obsolete very quickly.

author: Rasz
author_rank: Super Contributor
timestamp: May 13, 2016, 06:16:07 am
content: Quote from: nctnico on May 13, 2016, 06:11:30 amwifi and html is not going away any time soon

author: nctnico
author_rank: Super Contributor
timestamp: May 13, 2016, 06:43:08 am
content: Quote from: Rasz on May 13, 2016, 06:16:07 amTry and browse websites with the original Netscape or even a browser which was released a couple of years ago and you'll see what I mean.

author: kripton2035
author_rank: Super Contributor
timestamp: May 13, 2016, 06:51:12 am
content: take a look at Xojo IDE :http://www.xojo.comyou can compile to a Raspberry Pi 2 or more a nice modern desktop application(and also almost any linux driven arm7 or more architecture)

author: Rasz
author_rank: Super Contributor
timestamp: May 13, 2016, 08:34:16 am
content: Quote from: nctnico on May 13, 2016, 06:43:08 amexcept we are talking reverse here, try browsing 20 year old website on modern browser - does it work?

author: Jeroen3
author_rank: Super Contributor
timestamp: May 13, 2016, 08:49:25 am
content: The HTML <5 part does.The Javascript and other active content (there is a lot) does not.

author: JoseR
author_rank: Newbie
timestamp: May 13, 2016, 12:00:13 pm
content: That's an interesting theme, I would like to know the most common ways to build UIs on the industry.The best ones I've seen where made with QT in ARM boards, but I think for beginners it would require a very hard work to achieve those results.There's a Labview version for PI/Beaglebone for non commercial use:https://www.labviewmakerhub.com/doku.php?id=get_labview

author: nctnico
author_rank: Super Contributor
timestamp: May 13, 2016, 12:09:42 pm
content: BTW I'd advice against using GUI builder software which outputs a file (or code) to position the buttons and other controls at fixed positions. The problem with such a file is that you can't scale to a different display size. Ofcourse some UI frameworks have some support for auto sizing controls but in many cases you want buttons to just move and a graph or grid to grow bigger. I design my user interfaces using MS Visio (a vector drawing program) and then use formulas to do the actual positioning and sizing based on the size of the display area.

author: dannyf
author_rank: Super Contributor
timestamp: May 13, 2016, 12:27:41 pm
content: The competition from the soc people (=phone people) migrating into the embedded space creates some interesting dynamics.The most difficult part of integrating a gui into an embedded project is the GUI, its interaction with an OS. Data acquisition and algorithm are mundane and typically not a challenge for most embedded programmers. So the gui folks in the embedded space are generally better compensated, with limited supply.for the phone people, the opposite is true: there are plenty of apps programmers who can handle the gui in their sleeps. It is the data acquisition (on the bottom layer) that is a challenge for them trying to migrate into the embedded space.So a good marriage would be a team consisting of those phone people handling the user level interactions and the embedded people handling the hardware interactions, thus cutting out those high-priced folks who handle the gui today.I think we will see more and more embedded applications running on some OS in order to minimize software / development costs and speed-up time to market.

author: VEGETA
author_rank: Super Contributor
timestamp: May 13, 2016, 09:44:16 pm
content: Quote from: dannyf on May 13, 2016, 12:27:41 pmwell, that is good info. For me, the hardship is the selection of the tools. From what I saw, Altia is the best and most attractive solution but it will cost thousands, which is just insane! other good solutions are quite the same.And my requirement seems hard to do with a typical ARM MCU as someone told me here, he said M4 can not do it, but A9 can. The whole ARM MCU world is more complicated than PIC to begin with so I have many issues regarding picking the right IDE and toolchain (Keil is not free), integrating FreeRTOS (essential to me), and doing a good GUI that offers video output on the screen itself.The easiest way as the suggestions say is to use respberry pi 3 for gui and internet connectivity while using STM32 mcu with freertos for all other tasks... do you agree?someone even suggested Win10 IoT which is very new to me! didn't know it is a valid option. By picking Pi3 as GUI cpu, I will be saved from pricey GUI tools but I will stuck again with the tool to use with it.

author: rstofer
author_rank: Super Contributor
timestamp: May 13, 2016, 11:39:17 pm
content: Before I get married into a system, I like to do little experiments. Since the GUI and graphics are going to be the most difficult, particularly the streaming video to a portion of the GUI, I would do that part first. Try a PI 3 and see how it goes. Can you connect the camera? Will OpenCV display the stream? Can you embed the output of OpenCV into your GUI.The answer to all of the above is "YES" but the devil is in the details.http://opencv.org/Start with "High-level GUI" just to see the minimal GUI with a video capability:http://docs.opencv.org/3.1.0/d7/dfc/group__highgui.html#gsc.tab=0QuoteRemember, you are counting on Linux to provide a metric dumpster load of code. Stuff you don't want to write!Once you get a rudimentary GUI/display running, it will be pretty easy to do the real GUI in something like Qt and you can write it in C, C++, JAVA or Python. There is a Qt library for each of these (although I have only used the Python variant). You will also find Python code for dealing with the hardware IO on the headers. It's in the PI distribution.Incidentally, OpenCV also runs on MacOS, iOS, Windows and Android. So, you can migrate your GUI to an Android tablet and use WiFi to connect to the STM32F. You might want to consider not coupling the systems with SPI if networked connections can be responsive enough. That assumes, of course, that you can find a TCP/IP stack for the STM32F. If not, Microchip has a stack for their PIC32 - free!

author: SL4P
author_rank: Super Contributor
timestamp: May 14, 2016, 12:11:36 am
content: Usually, if I want a UI that is easy to evolve and maintain, I run a separate processor and environment that is suited to the specific requirements - and is responsive to the user input.this saves the main process - letting it handle the realtime events without any concern for the vagaries of the user.Interprocess Comms can be tightly coupled, or a fast serial topology.Adding other UI methods and/or remote capabilities then become a lot easier.

author: Iamraj
author_rank: Newbie
timestamp: June 05, 2019, 01:28:41 pm
content: Quote from: dannyf on May 12, 2016, 10:06:58 pmHi Danny, you mentions "os-based systems plus SOC" in the above post, can you please elaborate on it. Or share some link where I can study further.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: June 05, 2019, 09:38:12 pm
content: Regarding low powered embedded bitmaps, I wrote this compressor; it's designed around the ST7735 LCD controller.https://www.seventransistorlabs.com/compr.htmlThe code to read this format only takes a few hundred bytes on AVR, and the compression rate is fair (very good for line art). For example, I used a somewhat blocky background graphic on a project; the raw bitmap is 20.48kB, the PNG is 2.13kB, the RLE is 5.56kB, and the ST7 encode is 2.56kB.PNG pretty much always beats it, especially for more complicated, high color images, of course; but a stripped-down zlib inflator takes several kB of code and probably needs input and output buffers too, so isn't economical for just a few graphics. On most ARM platforms, you might as well use PNG, or something related to it (for example, you could strip it down so that graphics are whole chunks, stored in arrays: simplifying, or obviating the need for, PNG headers and structure).Tim

author: mubes
author_rank: Regular Contributor
timestamp: June 06, 2019, 06:41:59 am
content: Not sure whyhttps://github.com/littlevgl/lvglisn't getting more love here. Frankly, it could do with more graphically impressive demos, but the capabilities are quite impressive. No connection to the project other than as a user.

author: rstofer
author_rank: Super Contributor
timestamp: June 07, 2019, 01:07:49 pm
content: This may have been buried above TL;DRhttps://www.st.com/content/st_com/en/stm32-graphic-user-interface.html

author: sairfan1
author_rank: Frequent Contributor
timestamp: June 07, 2019, 04:44:52 pm
content: Though you are asking about graphic library, but you also want a quick solution. I would recommend cheap Chinese HMI you can buy around $15 and super graphics with touch panel.

author: agehall
author_rank: Frequent Contributor
timestamp: June 10, 2019, 12:59:54 pm
content: Quote from: sairfan1 on June 07, 2019, 04:44:52 pmThis was my first thought when I saw the topic as well - I recently discovered the Nextion HMIs (and I'm sure there are others just like them) and I'm 200% sold on using those for all my embedded UIs for the foreseeable future if I need to have a UI on my projects.

author: janoc
author_rank: Super Contributor
timestamp: June 10, 2019, 01:56:50 pm
content: Quote from: sairfan1 on June 07, 2019, 04:44:52 pmGood luck streaming video to one of those, though.