ST 32F417 veneer code (or something quite different) - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: August 20, 2022, 09:57:35 am
content: I am having some weird problems where I am calling a function in a boot block (bottom 32k i.e. 0x8000000 to 0x8008000) from a boot loader that's been relocated into RAM at these addressesSometimes it runs, sometimes it just gets skipped. The function calls do nothing. I even see this when stepping in Cube, although if I step in the disassembly mode then all the code is correctly stepped. Weird!I can see what the veneer is e.g.https://developer.arm.com/documentation/dui0489/i/arm-and-thumb-instructions/blbut how can it fail? Some interesting comments herehttps://stackoverflow.com/questions/39477993/arm-cortex-a7-avoid-memory-veneersand it is what I am seeing. Literally, the code being called with the "long call" is as if it wasn't getting called.And I can't move the functions being called to the boot loader because it would make it too big.I discovered this only when doing some testing, uploading new firmware using my http server and then using a reboot function there, which calls this well known codeCode:static inline void L_reboot(void){// Ensure all outstanding memory accesses including buffered write are completed before reset__ASM volatile ("dsb 0xF":::"memory");// Keep priority group unchangedSCB->AIRCR = (uint32_t)((0x5FAUL << SCB_AIRCR_VECTKEY_Pos) |(SCB->AIRCR & SCB_AIRCR_PRIGROUP_Msk) |SCB_AIRCR_SYSRESETREQ_Msk );__ASM volatile ("dsb 0xF":::"memory");// wait until resetfor(;;){__NOP();}}The issue happens very reliably after the board is restarted with the above, but not if it is restarted with debugger (presumably that does a hardware reset). I don't know if that is any kind of clue.Lots of stuff is simply broken, like the functions for turning a LED on/off (see above screenshot) which are just GPIO stuff. The LED is not changing. But the codeisthere, in the disassembly.Thank you for any ideas. Spent a day on this already, and I am down to watching GPIO pulses with a scope, and (mostly) can't see any. Basically long calls to functions are getting skipped, but otherwise the code is not crashing.

author: wek
author_rank: Frequent Contributor
timestamp: August 20, 2022, 10:09:33 am
content: QuoteYou did not show us calls to 0x0800'xxxx, but calls to RAM (bl 0x2001f688).JW

author: peter-h
author_rank: Super Contributor
timestamp: August 20, 2022, 10:33:00 am
content: It turns out that I have been down this rabbit hole a while ago, but in the other direction: calling a RAM-resident loader from the boot blockI can't see addresses in the calls to FLASH. The disassembly of the veneers looks like thisIf I step through it I can see the function in FLASH:I tried this format of long_call (this function is in the boot block) but looking at the disassembly it does not prevent it getting a veneer. Maybe the attribute syntax is wrong?The problem is that if long calls were failing, this doesn't explain why this code has been running for over a year, with the loader calling lots of boot block functions...Could there be something in the CPU which makes it support or not support long calls, according to how it has been reset?After much trial and error and trying out various code samples on e.g. stackexchange (which were obviously never tested, or were for some other compiler) I found that the way to get rid of veneers is to put the long_call attributein the prototypes(in the .h file):

author: wek
author_rank: Frequent Contributor
timestamp: August 20, 2022, 11:07:07 am
content: ldr.w pc, picks the word immediately after this instruction and stores it into PC, in other words, jumps to address given by that word. So, the two "instructions" after ldr.w pc, are not instructions, and if you'd look at the values at that place in RAM, you'd see the target address.IMO you are after a red herring and your problem is elsewhere. Try to describe the symptoms without making premature conclusions.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 20, 2022, 11:27:19 am
content: Indeed; removing the veneers did not solve the problem.It is a mystery; some calls from the ram resident boot loader are just not taking place, although if I step them in the debuggerin the disassemblymode, they happen ok.Statistically, the problem depends on how the CPU was reset.I don't know how to troubleshoot this.

author: wek
author_rank: Frequent Contributor
timestamp: August 20, 2022, 12:01:16 pm
content: > It is a mystery; some calls from the ram resident boot loader are just not taking placeThat's already a conclusion which may be false; the reason for what you observe may be something else.> Statistically, the problem depends on how the CPU was reset.That may influence what's mapped at address 0, among other things. What's the value of VTOR register, btw.?> I don't know how to troubleshoot this.As in any other case: observe behaviour, formulate hypotheses, construct experiments to prove or disprove them.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 20, 2022, 12:46:50 pm
content: At this point, VTOR is not at the default base; it is at 0x200 (512-aligned as required) above the boot block i.e. at 0x8008200. But should VTOR matter? Interrupts, RTOS are disabled.I did actually wonder whether I have interrupts enabled. They should not be because this stuff is running after a reset (a reset carried out with the code above, and a reset should reset VTOR, disable interrupts, etc). But maybe that software reset doesn't?I have addedCode:// This should not be neededRCC->CIR = 0x00000000;// disable interruptsSCB->VTOR = FLASH_BASE;// set VT location to defaultto the software reset, and before the entry to the boot loader.But it makes no difference. It does change the nature of how it hangs. Instead of crashing with an invalid opcode (which is directly the result of a function which writes to the FLASH not getting called) it hangs in a delay routine which uses CYCCNT. It now looks like the CPU is running very very slowly.

author: wek
author_rank: Frequent Contributor
timestamp: August 20, 2022, 01:19:10 pm
content: > RCC->CIR = 0x00000000; // disable interruptsThis disables only the particular interrupts generated by RCC, is this what you intended.Nonetheless, this does not matter as software reset resets among other things NVIC so effectively disables all interrupts which can be disabled. It also resets VTOR so it does not matter how do you set it before software reset.Software reset is as good as "pushbutton" reset, both cause so called system reset (read first part of RCC chapter). Only a select few things are not affected by this reset - the backup domain (which is not germane here probably), some status bits in RCC, some sleep-related bits, some (most?) of the Debug stuff. These (except backup domain) are reset by power-up reset only.SYSCFG_MEMRMP.MEM_MODE (i.e. mapping of memory at address 0) *is* affected, but it has its own rules (a.k.a. bootloader entry, AN2606 etc. - read Boot configuration subchapter).The debugger may do whatever it is pleased to do, and rarely is it concisely documented.> It now looks like the CPU is running very very slowly.Then investigate this. Output sysclk to MCO. Check RCC settings (this means "read out the RCC registers content and check", not "look at CubeMX's clock tree picture"). If you've clicked clock setup in CubeMX (because it's so easy and fast), make sure you've handled every case where it returns with failure.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 20, 2022, 09:29:47 pm
content: The slow clock was not the case... It is ok now. I am not using MX. A lot of code was originally generated with MX (by the other guy who was doing it part-time) but not much of that left now. Also I rarely use HAL code myself.Am I right that the veneer code is always ok, with no side effects or issues? I am pretty damn sure I had to use the long call attribute on that long call/jump to the boot loader. I spent ages on that one.I still have the issue that sometimes it all runs perfectly while at other times, and always after that software reset, it doesn't. The reason it doesn't is because a particular function (in the boot block, but called from the boot loader) which does some data manipulation, is simply not executed. So the data written to the CPU FLASH is garbage, 100% consistent garbage... It is exactly as if that function call was commented out.

author: peter-h
author_rank: Super Contributor
timestamp: August 21, 2022, 01:53:21 pm
content: After many more hours I am finding that the soft reset definitely does not work like a hard reset.It is hard to narrow down the way it differs though. One manifestation is that SPI2 is not running properly; wrong data is fetched from a serial FLASH. That could be all kinds of things of course.I looked up SYSCFG_MEMRMP.MEM_MODE but I am not using ST's boot loader. BOOT0 & BOOT1 are both 0.Maybe I should have implemented a spare GPIO to drive NRST (with suitable safeguards of course, but GPIOs are supposed to power up as inputs so this should work). Unfortunately it needs >20us so would need a hardware one-shot.

author: langwadt
author_rank: Super Contributor
timestamp: August 21, 2022, 04:51:26 pm
content: Quote from: peter-h on August 21, 2022, 01:53:21 pmis the flash also connected to HW reset, so it is actually the flash that needs a reset?instead of doing SW reset you try to use could use the watchdog

author: wek
author_rank: Frequent Contributor
timestamp: August 21, 2022, 05:58:48 pm
content: What do you mean by hardware reset, exactly?Software reset - and WD reset, too - generates a pulse on NRST, that's why it's in effect identical to pushbutton reset. Except if you prevent that pulse e.g. by using a push-pull output reset generator to NRST.Power-on reset (generated by the internal POR circuit) is different, I discussed that already above.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 21, 2022, 08:15:18 pm
content: This is turning out to be something weird.The issue is betweena) code loaded into the target via the STLINK V2/V3 debuggerb) code loaded into the target via a firmware update featureAnd it is not the obvious thing (item b) being duff). The firmware update route has been exhaustively tested over more than a year, but not since Cube was updated to v10, and the GCC tools updated with it.This is the post build batch file I am using, to generate the firmware update file:Code:arm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary yyy.elf zzz.datI know with 100% certainty that zzz.dat used to be identical to the content of the CPU FLASH. There is/was ~350k of code running fine.But now, not quite, and this is why it took so many hours to home in on it. The following examples are: cpu flash and then the fileIt starts off nicelyand starts to break around here (line 1, halfway along)In terms of the resulting code, it is still about 80% there. Bloody confusing...Does anyone have any ideas?I haven't done a "diff" properly because I don't actually have a file of the CPU FLASH. Yes, I could write a bit of code on my board and write one out. But fairly clearly the first difference is at offset 488 hex which is 110 decimal.The flash size (before the FFs start) is 377964 whereas the above zzz.dat file is 372748.I have no idea of what command Cube is using to drive the debugger but I see thisCode:make allarm-none-eabi-size KDE485.elftextdatabssdechexfilename3546966948057584481760759e0yyy.elfFinished building: default.size.stdoutThe reset issue may still be there, but all the time I have different data in the firmware-update file (which is where the problem is confined to) I don't have a hope of a working system, obviously.QuoteThat's useful to know!UPDATE: I wrote out a file of the actual CPU FLASH (the board has a 2MB FatFS filesystem) and treating that as a firmware update does work!So what has changed in the bloody GCC objcopy, making the above command line produce a small % of garbage?Code:arm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary out.elf out.datI can extract a firmware update file out of a running target by writing out the CPU FLASH into a file, but it is a pretty weird way to go about it.

author: wek
author_rank: Frequent Contributor
timestamp: August 22, 2022, 06:50:09 am
content: Look at the mapfile, what's around the address in question.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 22, 2022, 08:09:06 am
content: At exactly 0x488 is thiswhich is thisThe question is what could have changed? The batch file is invoking the GCC executable which is in the project path, set up by Cube. It isn't some old version which might be incompatible with a new compiler.I have written code to readily generate the firmware upgrade file on the target, which works 100%, but it is a strange way to go about it. It would be useful to find what command Cube uses to produce the binary which goes to the debugger, but I can't see the command anywhere.

author: wek
author_rank: Frequent Contributor
timestamp: August 22, 2022, 08:26:07 am
content: Are you sure the binary comes from the same elf the mapfile comes from?Try to disassemble the elf using objdump -h -S> It would be useful to find what command Cube uses to produce the binary which goes to the debugger, but I can't see the command anywhere.Debugger (gdb) uses elf.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 22, 2022, 09:44:02 am
content: The two files actually agree; at 0x488isthe _init_switches function.This is the -S dumpThe get_hex_switch is a misread; that function is the next one below, at 0x4c4.QuoteI am sure it does, since .elf is the only output file, but the .elf is 4.6MB, so 4.3MB is somehow stripped out on the way to the CPU FLASH. After various discussions here a long time ago I had been successfully using the commandCode:arm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary out.elf out.datbut bizzarely this no longer works. It still produces a file of very close to the right size.

author: wek
author_rank: Frequent Contributor
timestamp: August 22, 2022, 10:54:14 am
content: QuoteThat's mostly debug information (dwarf).If the binary matches objdump, then you objcopy/objdump from a different .elf than you debug/program from.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 22, 2022, 01:04:56 pm
content: Sure, plus Cube runs thisCode:Finished building target: xxx.elfarm-none-eabi-size xxx.elftextdatabssdechexfilename352540694805758447960475174xxx.elfFinished building: default.size.stdoutarm-none-eabi-objdump -h -S xxx.elf > "xxx.list"Finished building: xxx.listarm-none-eabi-objcopy -O binary xxx.elf "xxx.bin"Finished building: xxx.binwhere the xxx.bin (131MB) doesn't appear to get used for anything. The .list is a disassembly listing, as you mentioned.I did indeed have an orphaned .elf file, from about a month ago, caused by me having renamed the project, and the batch process producing the firmware update was picking up the old .elf file.But that doesn't explain the corrupted data. That orphaned .elf would have still contained running code. Especially in the boot loader which I had not touched for ages and where the corruption starts. That corrupted code could not have been produced by any compiler+linker.All I can think is that an updated objcopy.exe was not compatible with an older .elf.Thank you for your helpRegarding the veneers, I won't go over this again (takes hours of testing) but I spent some hours a year ago on how to transfer control to the boot loader (which is in RAM) and there was certainly no way to get a veneer or anything else to work automatically. The long-call attribute was the only way I found. Yes, some function callsinthe boot loader have been apparently successfully calling functions in the boot blockwithout the long-callbut I don't think thedebuggerhandles stepping into these correctly. Only in assembler stepping mode does that work right. So I have given all boot block functions (which for obvious reasons are exported only to the boot loader, not to the rest of the product which may get updated without the boot loader getting updated) the long-call attrib in their prototypes.This stuff is all easy once you know it, and it takes years to learn it.

author: wek
author_rank: Frequent Contributor
timestamp: August 22, 2022, 01:24:17 pm
content: An easy and quick test is to delete the elf from which you generate the binary. Recompilation will regenerate it. There may be another thing: if the bootloader loads the binary from the serial FLASH into the mcu FLASH, then regardless of how do you overwrite it in mcu FLASH, it gets reloaded from the serial FLASH once the bootloader runs. Timestamps/CRC inside the binary would reveal the difference.JW

author: peter-h
author_rank: Super Contributor
timestamp: August 22, 2022, 02:25:30 pm
content: QuoteI deleted the old elf, and edited the batch file which generates the firmware upgrade file to use the new elf. That was easy and obvious. A silly mistake.The current elf was always being generated (at every debugger run) but it wasn't getting picked up for the firmware upgrade file.My project has a firmware upgrade facility, via a file upload.QuoteThe process used to process the firmware upgrade file and eventually end up with it in the CPU FLASH is a bit long. There is CRC checking involved (a post build step appends a crc32 to the file) and other stuff. But this wasn't the problem. The file which came out ofobjcopyhad the duff bytes in it.So this is nothing to do with my board.So the CPU FLASH also had the duff bytes in it.All I can think of is the GCC toolkit update to v10.something which came with Cube v9 or 10. Also Cube got a very fast update from 10.0 to 10.1 which is unusual...The lesson is to retesteverythingafter anything major, which I normally do, but one can't retest really absolutely everything. I suppose the one benefit of spending a lot of time on a product is that by the time it is released, it should be fairly solid.