User upgradable firmware - Page 1
Forum section: electronics / general

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 04, 2017, 06:20:33 pm
content: Imagine a device with a bog standard AVR microcontroller. The chip has a fairly mundane job and resides on a PCB with a JTAG port for in system programming. Often, that would be it. However, shipping devices out without the option of fixing bugs irks me. I have been pondering how to allow laymen users to upgrade their firmware in the field:- USB. Even though it seems attractive at first, I do not consider this ideal. To do it is not terribly complex, but to do it reliably and without fuss (missing drivers, for instance) is a lot harder. You will also have to create a desktop application that allows the user to upload the firmware to his device, which in turn probably means having to support several platforms. The whole thing starts ballooning out of control fairly quickly.- Programmer: another options is to provide another programming device with the main device. The programmer could be preloaded with software, it might be a thumb drive that allows files to be transferred onto it or it could have a SD Card slot. Even though it is somewhat easier to develop than the USB option, it is still a hassle for the non technical user.- SD Card: this is the best option I could come up with. The device has an SD Card slot that accepts cards with an image. The user just puts the preloaded card in the slot and boots the device, or loads the files on an SD Card himself.The last option has a few consequences. The most obvious one would be that a second microcontroller would be needed to program the first. In some projects the BOM increase is small and irrelevant, but it might be a hurdle for others. The second microcontroller would use some extra power, but appropriate power gating should limit that to almost nothing. However, a big concern would be the increased complexity. It is no use adding an extra feature if it compromises the main attraction. The operation would need to be dead reliable and easy, but proper development and care should take care of that.Am I missing significant disadvantages of the suggested approach? Are there other options that might be more suitable?

author: rstofer
author_rank: Super Contributor
timestamp: August 04, 2017, 07:05:26 pm
content: You probably want to come up with a way to handle 2 images. You would keep the one that is running until the new version is known to work whereupon it can mark the first image as redundant. Somehow...One manufacturer I know uses an AVR on a small PCB to reprogram his devices. I didn't look at the underlying communications but in many cases, it could be the processors built in ICSP or JTAG. The manufacturer mails out the programmer with a stamped self-addressed return envelope and hopes for the best.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 04, 2017, 08:09:14 pm
content: Quote from: rstofer on August 04, 2017, 07:05:26 pmTime to introduce a third microcontroller to the mix. One with the current image, one for the new one and a third to flash the secondMore seriously: changing out the processor itself might be viable too. You would need something like a DIP package and a ZIF socket, though other solutions might be possible. A recent mailbag contained some creatively repurposed DIP packages in a pocket translator device. I am not sure I trust the mechanical part of that deal enough, though, knowing that old computers tended to wiggle socketed DIP chips loose regularly.

author: dmills
author_rank: Super Contributor
timestamp: August 04, 2017, 08:28:09 pm
content: My usual approach is to set up as a USB HOST, so that the user can just stuff a USB key in the hole and hold down a button at power up, that and a butch SPI flash that the data from the key gets read into, then verified then programmed to the rest of the processor on chip flash.Maintain two images a golden one (the first one programmed) and a updated one so that in the even of something wicked happening you can roll back.The flash is also handy as a place to store fonts, graphics, web pages (Some of them need a stupid amount of javascript to do anything).regards, Dan.

author: nctnico
author_rank: Super Contributor
timestamp: August 04, 2017, 08:30:07 pm
content: Quote from: rstofer on August 04, 2017, 07:05:26 pmThat is the way I always implement this. Upload an encrypted firmware image which is checked at startup by a bootloader. If the image is newer and it checks out OK, the old firmware is replaced. It is nice if the bootloader has some capability of receiving a firmware image so a device can be unbricked if necessary.The image itself can be uploaded by whatever works. Serial port, USB (serial port emulation) or network. Windows 10 has good support for USB to serial devices (including CDC serial port) out of the box so drivers are not such an issue as they used to be.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 04, 2017, 08:57:36 pm
content: Quote from: dmills on August 04, 2017, 08:28:09 pmDo I understand correctly that you keep two images in the flash chip, one static 'golden image' that never changes and a new, recently added image that gets updated every time? How do you write that new image to the microcontroller?

author: rstofer
author_rank: Super Contributor
timestamp: August 04, 2017, 10:24:52 pm
content: First, you need to write a bootloader. It checks some external condition and decides to update the second image.In memory, the first image may stay resident forever or it may alternate with the second image. The bootloader deals with that.If the bootloader doesn't see the external condition, if there is a second image, it loads it. If not, it loads the first image. Or, it tries to remember which is the most current image. But there needs to be a way to force the alternate image should there be an 'oopsie' in the most recent image.Something like that... You can probably find a lot of schemes on the Internet.At a lower level, the internal bootloader on a chip like the LPC 2148 looks at a specific memory address and if it has the right answer, the bootloader launches the code. If the contents are not correct, the bootloader waits for ICSP programming. Or, it looks at an external pin and if the state at boot time is correct, it ignores the internal code and the contents of the secret squirrel memory address and branches straight to the ICSP programming code. This setup carries only one image and is primarily used by developers who aren't using JTAG.

author: dmills
author_rank: Super Contributor
timestamp: August 04, 2017, 10:28:04 pm
content: I have a boot program (That is not as small as I would like) that checks for a USB update, checksums the main executable region of the processor and compares the checksums to see if the processor code is valid, then either jumps to the main program, copies the appropriate flash image to the main program area of the processor , then jumps to it, or copies the file from the USB stick to the flash, as appropriate.On a PIC32 it is not entirely negligible (Mainly due to the USB stack), but it is worth the space for the ability to easily update things.You will learn much about the wonders of linker scripts if you go down this path.Regards, Dan.

author: rstofer
author_rank: Super Contributor
timestamp: August 04, 2017, 11:45:27 pm
content: Quote from: dmills on August 04, 2017, 10:28:04 pmAmen, skippy!That must be an impressive script!

author: alank2
author_rank: Super Contributor
timestamp: August 05, 2017, 12:48:16 am
content: I sell a very tiny and easy to use automatic programmer for AVR's. It does ISP and PDI. It has features that protect your firmware from being extracted from its EEPROM as well. You can make it update only if it sees the fuses you expect, only update 2 times and then no more, etc.http://www.ebay.com/itm/292196966471It was designed as a "remote firmware upgrade" product, but honestly I use it for production programming and many other tasks as well.http://home.earthlink.net/~alank2/autoprog_manual.pdfhttp://home.earthlink.net/~alank2/autoprog_pcapplication.zipIf I need to update a customer, I ship it out to them with a return envelope so they can do it and then just drop it back to me.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 05, 2017, 12:56:47 am
content: Quote from: dmills on August 04, 2017, 10:28:04 pmYou can have a microcontroller programming itself while its running? I was not aware of that. That is a pretty neat party trick. I was under the impression that you would need a second chip to program the first, especially since I have found a number of projects of that nature on the web. That seems comparatively cumbersome compared to your solution.And yes, it looks like I will have to dive into programming bootloaders and somewhat more involved stuff. That is okay, as the bottom has already fallen out of this rabbit hole. Especially since I am always eager to do things 'the right way'. To not completely drown myself in the deep end, I will stay with AVR chips for now, simply because I am most familiar with that platform and toolchain.I found useful information on this subject here:http://www.avrfreaks.net/sites/default/files/bootloader_faq.pdf

author: rstofer
author_rank: Super Contributor
timestamp: August 05, 2017, 10:43:38 pm
content: Quote from: Mr. Scram on August 05, 2017, 12:56:47 amThe process is known, in the NXP world, as In Application ProgrammingHere is one example:https://developer.mbed.org/users/okano/notebook/iap-in-application-programming-internal-flash-eras/

author: alm
author_rank: Super Contributor
timestamp: August 06, 2017, 01:13:10 pm
content: Depending on your exact requirements as far as interfaces, there are some open-source bootloaders that you may be able to use. I believe there is also a bootloader that bit-bangs USB. Not sure about SD. If you use an external programmer, then you could use something simple like RS-232.

author: rstofer
author_rank: Super Contributor
timestamp: August 06, 2017, 02:44:39 pm
content: Note that nctnico said 'encrypted image'. It is not uncommon for images to be encrypted and they will almost certainly be read protected once flashed on the uC. IP protection...I really think the easy way to do this is with SD. In fact, the entire update program can be on the SD. Plug in the device and the uC reboots to code on the the SD which will handle the complexities of update the resident image.No reason the code on the SD can't keep an installation counter on the SD itself if necessary

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: August 06, 2017, 02:52:55 pm
content: Quote from: Mr. Scram on August 05, 2017, 12:56:47 amI don't know of any vaguely recent MCU that can't do this. Typically called "self-programming"In terms of hardware and software overheads, SD card is probably a good compromise - no USB peripheral stack or hardware needed.You only need to implement a read-only filesystem which makes things a lot simpler, and with SD you can bit-bash the SPI if youdon;t have an SPI port available.If your system already includes SD ( or any other storage) functionality, a very "lightweight" way to do it is to use the existing filesystem to copy a new firmware image to a memory device that's easier to access with a very simple bootloader - this can be SPI flash, or a reserved section of main program memory, or RAM if you have enough to spare.More info here :

author: alm
author_rank: Super Contributor
timestamp: August 06, 2017, 03:05:13 pm
content: That is a cool trick. The only limitation is recovering from firmware that does not work on that particular device. For example, our new firmware does not boot on X% of our devices due to a component tolerance or substitution issue. Then there is no way to recover except I(C)SP/JTAG.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: August 06, 2017, 03:38:04 pm
content: Quote from: alm on August 06, 2017, 03:05:13 pmA fairly unikely situation,and pretty poor design if an issue like that can make it hang rather than giving an error. Up to you to manage updates to avoid this sort of thing. The main issue is managing product variants so you it can't load FW for a different product - this is easily handled by including some ID values in the firmware image.You can also take measures to recover, e.g. with external flash you can have a backup image for recovery

author: Kjelt
author_rank: Super Contributor
timestamp: August 06, 2017, 09:10:39 pm
content: Indeed every fw update should have a header with important administrative data such as company id , product family id, hw and sw version id etc. Also to be futureproof a security version id, that determines which encryption and hash algo was used, ofcourse just abstract ids can be a single byte and i recommend using tlv structure so you can expand with new parameters in the future without having backwardcompatibility issues.

author: technix
author_rank: Super Contributor
timestamp: August 06, 2017, 10:05:02 pm
content: Speaking of protection of IP, it is dangerous to reuse the keys (adding depth to the key) and can result in disasterous scenarios. You need measures to make sure keys are not reused.For me I don’t encrypt the firmware when it is installed on the device. It makes program loading easier and the use of CDN available. It is cryptographically signed using a strong hash (SHA256) and a public key certificate (ECDSA) though. The signature is verified by the bootloader upon boot.When the user user is downloading the firmware it is always the device performing a TLS 1.2 handshake with strong cipher suite and client certificate authentication to make sure the keys are not reused and no intermediatary can read the firmware image.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 06, 2017, 11:03:01 pm
content: Thanks a lot, guys. You really are not disappointing me. This thread has lots of useful information and considerations.Quote from: rstofer on August 06, 2017, 02:44:39 pmApparently, it is fairly cheap to have someone decap an MCU for you and read the data in there, no matter the fuses. Protecting your IP that way does not seem very productive. It is not a hurdle for any competitor or copy cats, but it is a hurdle for the guy wanting to tinker with his own hardware. That does not really work for me.Going from there, I am not sure what encrypting things is going to help. Your key will need to be in there somewhere to decrypt and run your code. I can see the value in signing or hash checking, however.Quote from: mikeselectricstuff on August 06, 2017, 02:52:55 pmThanks for the video - and the great channel, of course.

author: ejeffrey
author_rank: Super Contributor
timestamp: August 07, 2017, 05:34:07 am
content: Quote from: Mr. Scram on August 05, 2017, 12:56:47 amFor modern flash based microcontrollers, yes. Assuming the MCU has the needed charge pump, and can be programmed over jtag or serial rather than requiring an old school high voltage programmer, it is basically a trivial feature with a huge benefit. It is a little bit tricky, since you can't read from the flash while it is performing a write or erase -- and read includes execute. Some micros have multiple banks of flash, or have a bootloader in ROM, but usually you will have to copy a small function to RAM that starts the flash command and then waits for it to complete. You also have to disable interrupts or move the interrupt handlers to RAM.There are other reasons you might still use a second microcontroller. It is more bullet proof in terms of dealing with failed updates -- assuming the auxillary program is simple enough that it never needs to change. It can also act as an interface bridge. The arduino does this (in some revisions) -- one micro does the USB interface, and helps program the other which runs your application. This is especially handy when using a bootloader provided by the MCU vendor that may not talk to the peripherals you want to use such as a wiznet SPI ethernet interface. Or the bootloader might be limited in size -- some micros may allow you to write protect a small block of memory to avoid accidentally corrupting the bootloader. But if you want a filesystem driver in your bootloader, it may not fit in the small space.In my experience, field updating is usually addressed late in the development cycle. When evaluating micros for a project you check of all the features including having a supported bootloader. Then forget about it until you have some functioning prototypes. I could imagine that this leads to last minute "hacks" like adding an auxillary MCU when you find out that the manufacturers bootloader doesn't work quite the way you need.

author: Kjelt
author_rank: Super Contributor
timestamp: August 07, 2017, 08:36:55 am
content: Quote from: Mr. Scram on August 06, 2017, 11:03:01 pmI find that naive. Fairly cheap is an understatement I think you have to pay at least $10000 to do this kind of job depending on the microcontroller.You will not deter goverments or big companies or even medium companies, but those are probably not your real problem.The real problem these days are the very small one or two person chinese companies that buy a western device and copy it over the weekend to bring out exact clones the next month on the market.Not read out protecting your firmware or equally distributing non encrypted firmware to customers can be ok for non commercial or open source hw or hobby projects but if you want to make any money at all in this world this is the first step you at least take to make it more difficult. Hell even the chinese copycats amongst themselves do it, go figure.

author: Kjelt
author_rank: Super Contributor
timestamp: August 07, 2017, 08:44:27 am
content: Quote from: technix on August 06, 2017, 10:05:02 pmNIST has a paper about secure keyderivation, a single key can go a long way

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: August 07, 2017, 09:04:31 am
content: Quote from: Kjelt on August 07, 2017, 08:36:55 amIf it's a 2-person Chinese uoufit it will not cost $10K for a weekend. I've heard figures of a few hundred for most MCUs.http://www.break-ic.com/index.asp

author: technix
author_rank: Super Contributor
timestamp: August 07, 2017, 10:45:18 am
content: Quote from: Kjelt on August 07, 2017, 08:44:27 amIf you do this you would not be able to use existing CDN infrastructure. Those services in their cheapest form serves static files with good TLS. Client certificate authentication is useable in CDN with PKI. If your server have metered traffic or less than ideal connection you will want to use CDN to ease the burden on your server.

author: MT
author_rank: Super Contributor
timestamp: August 07, 2017, 12:21:20 pm
content: Quote from: mikeselectricstuff on August 07, 2017, 09:04:31 amWell, so now there is no point in using flash security hardware? So will encryption do? The encrypted firmware still needs to be decrypted to be able to run on the MCU i presume?The chinese hackers complains aboutdirty companies!Quote

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 07, 2017, 12:32:47 pm
content: Quote from: mikeselectricstuff on August 07, 2017, 09:04:31 amI have heard about similar numbers. If all it takes is what an average consumer easily has on his bank account and a few weeks patience, I can hardly imagine the trouble of development and added complexity is worth the hassle. Security theatre to fool yourself does not seem to be a productive route.DRM that hampers paying customers, while not being a problem for organised opposition is not what I call a success. It makes sense to do it right or to not do it at all. Also, if FTDIgate teaches us anything, going after customers, even if they turn out to use fakes, is going to hurt you regardless.Granted, there are a few different sides to this story and all have merit. I know of a few people who successfully deal with the problem by simply out-competing copycats by providing a better quality product and service and staying ahead of the curve by constant development. Admittedly, that is much easier to do with some products than others. The product being a critical application or in a niche helps.Quote from: MT on August 07, 2017, 12:21:20 pmAs far as I know, running code that is encrypted is not possible. You will either need to have the code stored unencrypted, or unencrypt it at runtime with an unencrypted key. Both the code or the key can be reverse engineered by the aforementioned method.However, I stand to be corrected.

author: MT
author_rank: Super Contributor
timestamp: August 07, 2017, 12:41:46 pm
content: Quote from: Mr. Scram on August 07, 2017, 12:32:47 pmCan ion beam and scanned electron microscopy technology penetrate a metal layer whos placed above flash/ram area?(yes i know an extra metal layer is expensive)

author: alm
author_rank: Super Contributor
timestamp: August 07, 2017, 12:46:42 pm
content: I would expect MCUs with encrypted flash (e.g. LPC3154) to protect the memory where the keys are stored. Yet break-ic lists the LPC3154 among the micros they can read. Maybe the caveat that it only works if the keys are not programmed? Or maybe they charge more for them? It would be pretty terrible design by NXP if reading the keys is just as easy as reading the flash. Might as well use your own decrypting bootloader.

author: Kjelt
author_rank: Super Contributor
timestamp: August 07, 2017, 01:48:17 pm
content: Quote from: technix on August 07, 2017, 10:45:18 amI don't see your point, the firmware file is selfprotected (encrypted and hashed) so you can put it on a public forum if you like no-one is gonna be able to do something with it, no need for tls or other infrastructure, the microcontroller with its program is the only one able to do something with the file.

author: Kjelt
author_rank: Super Contributor
timestamp: August 07, 2017, 01:51:33 pm
content: Quote from: mikeselectricstuff on August 07, 2017, 09:04:31 amThat's scary and what scares me the most is that this site is still up and running on the open net and no-one takes it down, ridiculous.If they can really do this for < $1000 than a lot of western companies are in deep trouble.We have to wait for the next gen of microcontrollers that have onboard vaults that have the same kind of protection as tpm's, although I don't know silicon manufacturers that are building them which is even more scary.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 07, 2017, 02:23:13 pm
content: Quote from: Kjelt on August 07, 2017, 01:51:33 pmReverse engineering is not illegal. Doing a clean room design is not uncommon and has even been historically significant, as Phoenix sold their reverse engineered BIOS to several other companies, but without it containing any original IP. That way they were successful, where others were sued into submission.As often is the case, security through obscurity can provide an extra hurdle, but hardly adds any actual security.https://en.wikipedia.org/wiki/Clean_room_design

author: Kjelt
author_rank: Super Contributor
timestamp: August 07, 2017, 02:37:07 pm
content: Quote from: Mr. Scram on August 07, 2017, 02:23:13 pmThat is different in that case that you should look at the system as a black box, what it does and then create your own black box with your own code that acts the same way.In the case of the BIOS it was only allowed in court because the programmer that wrote the new BIOS did NOT have any knowledge of the source code of the IBM Bios, there was purposely built in knowledge gap between the engineers that looked at the bios assembly code and the programming team for the new Bios, still it was on the edge of legality IMO.But what my biggest problem is here is not that they reverse engineer and improve the code but just copy it and replace the company's name with their own name.I have proof from my own company that this happens, we found products that were 1:1 copies of our product and we knew because one of the programmers left an easter egg in the code and that was still present in the copyStill we could not make a courtcase out of it since it was a Chinese firm and they have different laws, we could however seize their products at the border of the EU (import restriction).I have no problem with engineers that improve and create their own, but I have a big problem with copycats because they ruin the company of the original R&D engineers, because invention and creation costs a lot of money.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 07, 2017, 03:07:42 pm
content: Quote from: Kjelt on August 07, 2017, 02:37:07 pmNote the link I posted. It describes a process in which a product is reverse engineered by looking at a competing product, creating a set of requirements based upon that work and then having a second team create your actual product. This method is purposefully designed to wash off any IP infringements and comply with the law. The process is not on the edge of legality, as it has been purposefully been designed to produce results that are in the clear.QuoteVery true. Using other people's work in products without permission is and should be illegal, while reverse engineering in itself is not. Whatever the case, copycats are a reality and seem to always have been.

author: Mr. Scram
author_rank: Super Contributor
timestamp: August 15, 2017, 10:29:37 pm
content: When I wrote the opening post of this thread, I had never seen a device of which the firmware could be upgraded using an SD Card. A lot of devices use similar set-ups, like using a USB drive, but never an actual card. After starting the thread, I came across a clip where Dave upgrades the firmware on his upcoming GW121 multimeter... using an SD Card. How cool is that? Great minds think alike, or something of that natureObviously, I wasn't the first to come up with the method. It's fun to see it applied regardless, and to see confirmed that it is a viable and fairly streamlined method.

author: alank2
author_rank: Super Contributor
timestamp: August 15, 2017, 11:49:27 pm
content: My device mentioned in post #9 has up to 512K of EEPROM on board used for firmware. It also allows you to program a configuration that can evaluate what it is connected to and make a decision as to which firmware to flash with, etc.