Custom STM32 bootloader guidance - Page 1
Forum section: electronics / general

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 06, 2023, 08:37:32 pm
content: I want a way to upgrade a custome pcb board for a product in the the field in case there is a bug of some sort. My board uses the STM32F030K6T6 ic. I haven't done any work with bootloaders so I started by reading through the available documentation ST provides (DataSheet, AN3155 and AN2606 ). From this, I gathered that I can access boot mode from the boot0 pin but can only flash an image via USART (pins pA10 and pA9 or pA14 and pA15). Problem with this is that the pcb has already been designed to use pA10 and pA9 for I2C communication with modules, pA14 is for SWD clk for flashing and pA15 is for a random LED. A full redesign isn't really possible as the product is set to launch in the next few months. My initial thought was to finish testing and functionality and work on upgradable capabilities later but it looks like bootloaders are a lot more complicated than I thought. On some of our older products, we use I2C to update our firmware but those products use Microchip ICs (PICs). With this issue in hand and a miniscule amount of experience, I came up with 2 possible solutions:1. Make a custom bootloader for the stm32f030k6t6 that will allow for me to update via I2C2. switch pA9 and pA10 to USART peripherals, switch into boot mode via code (the boot0 pin was not considered when the circuit was made) and update firmware via USART.Both these options seem way out of my scope of understanding, but I am willing to put in the work. What advice can you give me in terms of tackling this? Is there one option that is more practical than then the next given my experience level?

author: Siwastaja
author_rank: Super Contributor
timestamp: June 08, 2023, 11:59:32 am
content: There is absolutely nothing complicated in bootloaders, just some work and learning curve as always with anything.Most robust and probably also easiest way to deal is dividing the flash into three parts: bootloader, application A and application B. Bootloader only verifies the validity of application A & B (against a checksum) and boots either one; a few dozen lines of code. Then application itself is responsible of receiving the firmware update data, erasing theotherapp and writing there. This way, power loss or other issues leave the working application intact, and the bootloader will just boot the old firmware again in case the new one fails at check.This way, bootloader itself does not need any flash writing or I2C receiving functionality, and therefore, can be simple enough that bootloader itself does not need an update mechanism, either. I don't frankly understand why people prefer complicated bootloaders over doing most of the work in the application itself.You have to:* Learn how to use linker scripts to describe the memory addresses where application code is placed;* Learn how to erase and program the flash memory - this is all explained in STM32 reference manual and simply involves some register writes and polling for status flags* Learn the bootloader magic itself, which is just a few lines of code of inline assembly copy-pasted from the internetz.

author: Psi
author_rank: Super Contributor
timestamp: June 08, 2023, 01:07:28 pm
content: The bootloader isnt very complicated, it's just like any other appThe most annoying bit is usually figuring out the correct settings for the linker script.You usually also need to setup some variables no-init and have them at a fixed memory address. (so the bootloader and app have some shared space to exchange flags bits and stuff)

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 08, 2023, 01:33:07 pm
content: So basically application A is my default ap that will act as a default (when I don't have an ap B or there is an error with app B's checksum) and application B would be the updated version of the application?

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 08, 2023, 01:35:38 pm
content: Quote from: Psi on June 08, 2023, 01:07:28 pmHmmm linker scripts seem to be a big part of the process. I guess I will have to dive into what it takes to make one. I assume it is different than the bootloader code or application code?

author: Siwastaja
author_rank: Super Contributor
timestamp: June 08, 2023, 01:51:45 pm
content: Quote from: Tripplecheckuser on June 08, 2023, 01:33:07 pmNo, the "default" app would alternate between A and B, because A must not write to A and B must not write to B in order to remain bootable if power is lost during update.I deal with this writing an increasing counter value such that if app_a is now running with counter value 5, it writes app_b with counter value 6, and bootloader boots the app with larger counter value (if it passes the CRC check, that is).

author: Siwastaja
author_rank: Super Contributor
timestamp: June 08, 2023, 01:54:05 pm
content: Quote from: Tripplecheckuser on June 08, 2023, 01:35:38 pmYeah, maintain the bootloader and the application as two completely separate projects, with their own linker scripts, makefiles etc.In my double buffered A-B scheme, I use two linker scripts for the app, producing two differently linked but otherwise identical application binaries.

author: Psi
author_rank: Super Contributor
timestamp: June 08, 2023, 10:29:33 pm
content: Yeah, they are two separate projects and each produces it's own intel hex file as output. The intel hex file format includes the flash location to write the program. So the boot loader hex has different address range since its going to a different location in the MCU flash block.So you erase the chip, program one hex file and then program the 2nd hex file (or you can merge the two hex files together if you want to, since they are for different flash address ranges. just need a tool to merge intel hex files, there are many)That is one of the things the linker script does, it sets up the locations in flash where each program will be and their sizes.So you can have two projects and they cant step on each others toes. If your bootloader gets too large your bootloader project will error out when you try to compile it saying it does not fit any more even when the MCU has plenty more free space. Because you set the linker script to give it some space and are now trying to use more than that. etc..It's more the 'syntax' of the linker script which can be annoying. but there are plenty of examples around the place.Just need to read them and understand what they are doing so you can modify your one.If you are doing the 'intermediate bootloader' thing where you have a block for the main app (APP) and a block of empty space (APP2) to store a new image to be updated to by bootloader (copy APP2 to APP). Then you need 3 blocks.A small block for the bootloader, and two large blocks of equal size for APP and APP2.The downside of a intermediate bootloader is that you waste bit under 1/2 the space of your MCUs flash memory for the APP2 block. But if your project needs a block of flash for storing your own data/logs you can sometimes combine that block with APP2. It just means your data/logs has to be erased to perform a FW update.

author: nctnico
author_rank: Super Contributor
timestamp: June 08, 2023, 10:37:09 pm
content: Quote from: Siwastaja on June 08, 2023, 01:51:45 pmAnother option is to check the validity of B and overwrite A if B has a different (*) firmware version than A. That way you don't need to remap any of the memory to make a firmware image work from different base addresses (or have different firmware images to work from different addresses). Retry if the checksum for A is invalid after programming. Unless the flash is broken, this doesn't leave you with a bricked system.I second the suggestion to do uploads through the application. This allows to update the update mechanism itself and makes the bootloader smaller. A downside is that you'll need to include a working firmware image into the bootloader image so the microcontroller can run the application straight away after production programming. Including a firmware binary is possible with an assembler file which includes a firmware binary which is then placed into the flash area where the application needs to sit (ld script again).* checking for a different firmware version also allows to downgrade firmware by simply uploading an older version into the device.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 09, 2023, 07:37:21 am
content: Quote from: nctnico on June 08, 2023, 10:37:09 pmI don't understand what you mean by "including" a firmware imageintothe bootloader. Quite obviously, when you do production programming, you can just program the whole thing with combination of bootloader + app.

author: redkitedesign
author_rank: Regular Contributor
timestamp: June 09, 2023, 07:55:54 am
content: Quote from: nctnico on June 08, 2023, 10:37:09 pmIt also depends on how much flash you need for the actual product functionality.If that is much larger that the update functionality, having two fullblown app's effectively halves your flash.Thirdly, a bootloader with app-install/update functionality should not brick itself, and in extension the update mechanism when installing a new app, making this less risky.In the A/B app scheme, it is possible to install an update that seems to work but for example has a bug preventing writes to the upper half of the flash. Thus the update installs properly, passes CRC testing and runs fine.But now any further updates are blocked, as this version cannot overwrite the old version! Prior testing can reduce this risk, but not totally eliminate it, especially if your business has grown and you now have multiple hardware versions with multiple customers with multiple firmware requirements.(there are further mitigations possible, like starting the old version from the new to test if there is a way back, allowing an second update for the new app. But if you think it through properly, it is an unsolvable problem, you can only reduce the risk)

author: Psi
author_rank: Super Contributor
timestamp: June 09, 2023, 08:35:07 am
content: Yep, if you have the space, just have two app areas, one where your app is and one as sort of a 'temp storage area' where you app itself can assemble a new fw image based on data your APP receives, (uart/spi/lan/ble etc). When it's all there and any checksums confirmed good you set some flag variables and reboot into bootloader. The bootloader starts and sees from those flags that there's an image in the temp area that is ready for copying and it copies all of the temp storage area over the top of the main APP area, then reboots (app area now contains new fw).Downside is if you intentionally flash an image that is bad you might brick the thing, since you can't load a new image into the temp space if the main app isn't running properly.Upside is your bootloader is ultra simple, it doesn't need to know about anything, no comms, no reading packet of FW data or anything. All it does is start up and either copy a block of flash to another block of flash or not.The other way of doing a bootloader is you duplicate all the code to talk to the uart/SPI, decode packets, understand protocols etc.. all in the bootloader itself.So to do an update you have to first start the bootloader and then you talk to the bootloader to send blocks of FW for writing to flash.It makes your bootloader more complicated and bigger but you can never brick the device because the bootloader is always available and always runs first so you can always connect to it and upload a new fw image.

author: nctnico
author_rank: Super Contributor
timestamp: June 09, 2023, 09:02:07 am
content: Quote from: Siwastaja on June 09, 2023, 07:37:21 amYes, and including a working firmware image into the bootloader image makes that easier. Just one file to program so no chance for mistakes.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 09, 2023, 09:36:38 am
content: I usesrec_catto combine multiple intel hex files into one file which is easier to program on any device without having to think how one would use (possibly manufacturer-specific) programming tools to program multiple files. Highly useful small piece of software.

author: nctnico
author_rank: Super Contributor
timestamp: June 09, 2023, 09:58:55 am
content: Quote from: redkitedesign on June 09, 2023, 07:55:54 amThere is a simple mechanism to prevent this from happening and that is by only allowing firmware that is known to be compatible with the hardware. But in the end part of testing firmware updates on every hardware version needs to be part of the firmware release test.

author: nctnico
author_rank: Super Contributor
timestamp: June 09, 2023, 10:42:33 am
content: Quote from: Siwastaja on June 09, 2023, 09:36:38 amIncluding the binary image during compilation does the same but produces an ELF file (given the OP uses ST's CubeIDE and/or GCC). From there you can create any output format. It might not be possible with other tools. In the end it depends on your preference & situation.

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 14, 2023, 01:02:02 pm
content: Quote from: Psi on June 08, 2023, 10:29:33 pmFunny you mention that. I was going through a simple bootloader tutorial by ST () and I think I encountered this issue of the bootloader suddenly being too large(specifically the heap). The issue with this is I don't know enough to modify the bootloader allotted memory properly so it takes the required space. In tutorial 1-4 (its a 6 part series) I had no problems (I'm using a different Nucleo board STM32F030R8) but as soon as I add the contents of tutorial 5, my debug session terminates immediately and erro is set to ENOMEM(in the sysmem source file) confirming I am running out of space. I tried doubling the length of memory for my bootloader (32->64kb) to no avail.I tried checking the ST community but it looks like they are doing some website maintenance/revamp of some sort (just my luck). So I am kind of lost as to where to go next.

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 14, 2023, 02:10:12 pm
content: Quote from: nctnico on June 08, 2023, 10:37:09 pmSo for a use case example:I have aLED BLINKING CODE(A) and anNEW LED BLINKING CODE(B).I flash mybootloader+Ain my controller.Mybootloaderchecks to see if there is any code inBand if the fw version is different. If not,bootloaderrunsA.I want to update my controller withNEW LED BLINKING CODE.I getNEW LED BLINKING CODEthroughA(using some i2c or uart communicationGETNEWFW();) and store it inBthen restart controller.Bootloaderthen checks ifBis a different fw version thanA(and it is now) so it overwritesAwithB(NEW LED BLINKING CODE) and jumps toA(which now hasNEW LED BLINKING CODE).If the controller losses power...whileAis storingNEW LED BLINKING LEDinB, it will just retry when power is restored, allowingAto remain in tactwhen thebootloaderis writingBtoA, thenAwould be incomplete. I assume the controller would always have to always start inbootloaderto prevent this from being a problem. So if the controller starts and seesAis invalid andBis (I assume it ignores the fm version in this case), it writesBtoA.Draw Backs:If theGETNEWFWfunction (that will be located in the currently running app) is implemented wrong on a new image, the device can not longer get new fw and will be bricked (maybe I could make that function modular and place it in RAM so it never gets changed)Double of my flash will be used (A+B) but only half of that will be in use at any given time (A)Pros:thebootloaderwill be pretty robust and smallbricking controller will be less likelycan update with both older and new fw versionsIs this how this scenario would go?

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 14, 2023, 02:30:13 pm
content: Quote from: Psi on June 08, 2023, 10:29:33 pmWouldn't I still waste 1/2 the space if I went with the A/B switching scheme since I would need to account for APP and APP2 even though one app is used at at time? I am leaning towards the 'intermediate option'. My IC (STM32F030K6T6) has a max of 256kb of flash (according to the data sheet ) and my APP is 24kb so I think I should definitely have enough

author: Siwastaja
author_rank: Super Contributor
timestamp: June 14, 2023, 03:06:45 pm
content: I think you got it right. The minor difference between what's proposed by me vs. nctnico is the copying step in bootloader. Copying B->A allows you to use single app binary linked to single address space, but the cost is somewhat increased complexity in bootloader (it has to erase and write flash, not bad at all but still something), halved write endurance of flash (probably not a big deal at all), and significant boot latency after the update (erasing is slow, often in seconds) - this might be a real problem.IMHO having to link two binaries is a non-issue - you just run the linker twice with slightly different linker scripts -, but this is also matter of taste. If you do that, you can just choose whether to run A or B from the bootloader. Downside is having to either communicate, from the app, to the updater, which of the two files the MCU app wants - easy if you have some kind of periodic status message anyway; or always providing the two files concatenated each time, doubling the file transfer length through your update link.If you want 100% robust flashing against power failures, and I think you should in $current_year, there is no way around effectively halving the flash, it happens in any scenario except the non-robust one.

author: Tripplecheckuser
author_rank: Newbie
timestamp: June 14, 2023, 04:40:08 pm
content: Do you know any good resources that have the model of bootloader for the STM32boards? I plan on searching but would like a good place to start.

author: krho
author_rank: Regular Contributor
timestamp: June 15, 2023, 07:29:58 am
content: Why halving the flash space available in MCU. Add SPI flash and have a backup copy there. In most cases it's cheaper than getting MCU with twice the flash. Especially if you need 1MiB+

author: nctnico
author_rank: Super Contributor
timestamp: June 15, 2023, 08:19:15 am
content: Quote from: Tripplecheckuser on June 14, 2023, 02:10:12 pmYes. And indeed, testing new firmware where it comes to receiving new firmware images is essential. OTOH: testing new firmware for bugs that affect existing firmware features is always necessary.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 15, 2023, 09:48:09 am
content: Quote from: krho on June 15, 2023, 07:29:58 amIt's still halving what you have available. You are adding more and halving that total. I don't agree with the general principle being good, because it adds cost and complexity. In some rare cases it might end up cheaper, but most microcontrollers I have ever worked with have had more than double the flash for the application, or if not, a next up model costs only a little bit more. (BOM line, PCB real estate, routing work, and loss of available IO also has an equivalent cost which can be more than the cost of a $0.50 flash chip.)One notable exception is STM32H750 which is a stripped-down-flash version of STM32H743, and only has a single sector, so you definitely want external flash, not only due to the relatively small amount of flash, but also it being in one erase unit only, preventing failsafe firmware update. And the cost difference of these two parts is more than just $1 or $2.

author: Psi
author_rank: Super Contributor
timestamp: June 17, 2023, 04:31:34 am
content: Quote from: Tripplecheckuser on June 14, 2023, 02:30:13 pmYes, the only way to not waste that space is to use the more traditional bootloader method.Where you have a bootloader space and an one APP space. The bootloader runs and talks to the outside world to receive new FW data and write them to the APP area.The disadvantage with this is that the bootloader is more complicated and larger in size.It needs to understand RS232/SPI/TCP/BLE or whatever method you use for FW updates, and include libraries for them. So there is more risk of bugs.Because your bootloader needs the full communication stack you are now duplicated the stack in two places, in your app and in your bootloader.Not a problem if you are using UART, but if your using TCP or BLE it gets annoying and wasteful. If in the future you update the app you may end up with a newer communication stack in your bootloader than in your app. Which is fine but you need to support both, so has legacy support issues.Check out this bootloader video that mikeselectricstuff did.It's quite good at explaining things.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 17, 2023, 07:07:09 am
content: Quote from: Psi on June 17, 2023, 04:31:34 amIMHO, this is only really suitable for simple UART based updaters which are so simple they can be fully tested with 100% coverage, and interface locked at design time. You can alleviate the size issue by exposing the bootloader functions for application use, but then you are locked down to these implementations forever, in both bootloader and app. It's a total no-go IMHO, things like TLS implementations will have critical security bugs which need updating.

author: darkspr1te
author_rank: Frequent Contributor
timestamp: June 17, 2023, 08:34:56 am
content: As a user who has fallen prey to badly implemented bootloaders with little to no fail safe's , I can say its not a fun rabbit hole to go down.Most implement a bootloader in it's own right with app being compiled ass separate entity ,here is a examplehttps://github.com/darkspr1te/stm32f407_vet_bootloader/blob/master/platformio.iniit for stm32f407 but the main code principle is the same for most stm, it looks for a bin file on sd-card and writes it to flash @0x4000 and then resets into it, (that just mean i load the vector table into ram and the reset into the new vector table)or this one which has more complicated setup,https://github.com/darkspr1te/mkstft35_bl_vet_fsmcthis one runs a small crc check as it reads the data , if crc fails it wont write, if it passes it's decrypted then written (it's open sources version of the nasty and stupid encrypted MKS robin bootloader )I have many more code examples of bootloaders and apps in platformio / stm32cube base if needed, you will note most have a encrypt/decrypt option as i've found the need to hack quite a few badly done bootloaders in order to repair the device which has since bricked and more care was put into encrypting the IP than ensuring correct updates.You will also find it hard to do the A/B flash space on many stm's you look at, even though I have implemented such myself on stm's and others I have found it works best on spi flash based devices offering more /swappable memory space.While most of my loaders work with sd-card/usb flash/uart over usb there are much more modern ones like the new st example which make the device appear as a flash card and you just drag and drop a bin file(a check is implemented that the first few bytes point into a proper vector table or it wont write) , or the older DFU version (buggy as heck even in silicon based DFU)The main thing i have learned in all this re-writing bootloaders is it's key to allow bootloader activation even if firmware corrupts, one device i have always goes to bootloader if powered via USB yet it was not implemented ,it was there in the code but it was skipped by a prior instruction , bug or left test code i dont know but once i NOP'ed out that it allowed me to write correct firmware and it lived again.another lesson is it's key to have a decent check method in place, for MKS they simply used RSA128 code to act as a form of crc but they made a mistake in the code in that it erased first, then checked the incoming packets for correct rsa sequence decrypt and if they passed were written, if failed it would error out but has already started the erase procedure rendering the device bricked.How you provide the firmware to the bootloader can also effect your implementation, a example would be if delivered via sd-card I have in the past used headers and footers added to the binary file that include crc data as as the complete file is available to the mcu then a crc check can be run on the entire file prior to writing, however if you sending via another method then you wont have the memory space normally to receive the entire file and then run a crc on it, this is where some implement transfer systems like zmodem/xmodem which have inbuilt packet based crc's and were common in modem days and are popular among some as the code is small and is the same for host/client making deployment a little easier.That leads me onto the last item, the app that delivers the update service, also important. Again i have run foul of many badly done updaters that either fail due to age/windows support, driver support, dodgy chinese dll's requiring admin privilege's to just write to serial port ? no valid file checking , no valid servers anymore my fav android issue no BT device seen !. I have to maintain a windows 7 VM just for a few devices with PL2303 chips because windows has banned the driver and my OBD device has chinese spyware clear as day in it but it's the only way to clear out the saved reports, almost like they did that on purpose.so if you can eliminate the update APP requirement you will remove one large headache.darkspr1te

author: peter-h
author_rank: Super Contributor
timestamp: June 17, 2023, 10:19:48 am
content: QuoteAnyone exposing an embedded box to the outside world is gonna have much bigger problemsAnd if the embedded box is not exposed (is aclientonly, usually behind NAT) then security is a non-issue.But we've done all this beforeAs in the other threadhttps://www.eevblog.com/forum/microcontrollers/upgrade-firmware-by-end-user/msg4912849/#msg4912849the choice will be according to other requirements.Also whether and how firmware needs updating is a heavily political decision - as posted in the other thread.

author: nctnico
author_rank: Super Contributor
timestamp: June 18, 2023, 08:39:51 am
content: Quote from: Siwastaja on June 17, 2023, 07:07:09 amTrue. With the firmware update mechanism inside the firmware itself, you can always replace things like encryption algorithms and change the ways the firmware image is delivered / formatted.

author: peter-h
author_rank: Super Contributor
timestamp: June 18, 2023, 01:32:08 pm
content: And brick some large volume of your product if it goes wrongAs I posted above, if it is on an open port, it'sgoingto get trashed if somebody finds it (subject to an adequate attack surface e.g. the ability to access a filesystem and either DoS that or plant some executable in there, etc) regardless of some esoteric debate about MbedTLS vulnerabilities, and if it isn't on an open port then the crypto never needs to be updated because it will work "for ever".Well, until the (public facing) server (to the back of which it is connecting to as a client) is forced to update. An example might be your box is using TLS 1.1 and 10 years down the road the server needs an upgrade and the lowest it supports is TLS 1.3 and then you are forced to update your box, with all the risk involved. I am facing this with my product. It should all work but the disaster potential is not wholly avoidable. Only extensive testing will minimise the risk. If it was me I would implement a TLS 1.1 interface on the said server and to hell with upgrading the client crypto. Nobody will find out.

author: Georgy.Moshkin
author_rank: Frequent Contributor
timestamp: June 18, 2023, 02:15:23 pm
content: Bootloader rules I came up with during few years1. write protected bootloader sector2. readout protection is on3. crc32 and retry mechanism4. aes encrypted in software, easy to migrate5. can enter to bootloader on reboot during 500ms window6. a way to reboot without powering cycling7. dma bridging between two usarts for flashing other MCU, e.g. esp82668. no swdio swdclk pins routed on final pcb

author: peter-h
author_rank: Super Contributor
timestamp: June 18, 2023, 09:19:39 pm
content: QuoteThat's an interesting option. The 32F4 boot loader can flash the flash directly from serial uart data; whether it uses dma I don't know but there is no need to because programming is quite slow.I remember someone telling me that having two CPUs is the only provably non-brickable way. I guess it is true. One of them is just a boot loader and feeds the data to the main CPU, with its serial boot loader activated.

author: Georgy.Moshkin
author_rank: Frequent Contributor
timestamp: June 20, 2023, 04:47:31 am
content: Quote from: peter-h on June 18, 2023, 09:19:39 pmYes, probably DMA is not so important for bridging. I have commands for write protecting of first sector and for setting RDP level. Theoretically, write protected bootloader may be overwritten, but this never happened to me.Two CPU way is interesting too, maybe cheaper is to use some flash IC with bootloader pre-flashed, and route with WE pin tied to "write disabled" voltage level. After it is soldered, write protection is always on.