GCC Embedded Linker Issue Stops uSupply Development - Page 1
Forum section: electronics / general

author: EEVblog
author_rank: Unknown
timestamp: December 09, 2018, 11:48:08 pm
content: David found a linker bug in the GCC 7.2 ARM-EABI (embedded) while working on the µSupply firmware. An example of how your project development can completely stop when you encounter errors like this.

author: darik
author_rank: Regular Contributor
timestamp: December 10, 2018, 12:00:36 am
content: Reduce the resource load on the linker by reducing the number of symbols it has to deal with.Go into your code and make as many functions and variables as you can local in scope. Maybe that'll fix it.

author: ataradov
author_rank: Super Contributor
timestamp: December 10, 2018, 12:07:08 am
content: Where does this version of the GCC come from? Even official launchpad version is newer, so it definitely worth trying. There is no real point on reporting bugs against old versions.

author: FrankBuss
author_rank: Unknown
timestamp: December 10, 2018, 12:12:54 am
content: Right, first try the latest GCC version, maybe the bug is already fixed.If not, good luck finding the bug in GCC. If you have lots of time, you can compile GCC with debug information and then debug the compiler and linker. If you don't have time, compile it with Keil µVision, they use a different compiler, written by ARM. It is even free, if you need less than 32 kB.If you want to stick with GCC, and the latest version has still the bug, just comment out everything step by step until the linker error doesn't happen anymore. Or rollback to older versions of your code that works (I assume it is all version controlled and you commit often) and apply the changes to the current version step by step. You can then see exactly what causes the problem and probably find a workaround for it, like different link order etc., or the GCC guys can help then.PS: GCC code quality sucks. By fuzzing the compiler, someone foundfound 100 bugs. The same fuzzing found 8 bugs in the Rust compiler and 9 bugs in LLVM/clang. Maybe it is possible to use LLVM/clang as the compiler and linker. But might need some work for writing new linker files, Makefiles etc.

author: MSM
author_rank: Newbie
timestamp: December 10, 2018, 12:19:47 am
content: Is it possible to dump several source files into one translation unit to decrease the number of the objects and avoid the need for linking too many of them? You may be able to pinpoint where the shunt hits the fan!

author: lucazader
author_rank: Regular Contributor
timestamp: December 10, 2018, 12:34:19 am
content: (posted on youtube, but realised this is probably the better place ot have a discussion about this)There is an open bug with linking with arm-none-eabi-gcc to do with link time optimisation -flto. Not sure if this is what you are using here however.If it is:Seems to only pop up with STM32 cores that I have seen reported.Has to do with the order of the files that are being linked to, and specifically the weakly defined functions defined in the start.s assembly file being discarded.This happens if you pass the assembly script to the linker after all the c and c++ files.The temporary fix for this bug is to pass the assembly files to the linker first.﻿Otherwise it might be worth trying to see if this is a specific issue of the compiler on windows?If the code is in VCS somehwere, maybe ask someone on the forums who has an stm32f0 dev environment setup to see if it is some random setting?As some have mentioned you could try compiling on linux. If you havent got any of that setup, or have little interest in setting it up, perhaps ask one of the forum members to give it a crack to see if it can be linked there?Compiling quite large codebases for stm32 products at work we have only run into the afore-mentioned actual bug in the compiler/linker.There have been instances where using an eclipse based IDE, the java VM gets messed up and starts giving super weird compiler/linker errors. but these issues never followed the code when someone else tried to compile.

author: nctnico
author_rank: Super Contributor
timestamp: December 10, 2018, 12:38:14 am
content: Quote from: ataradov on December 10, 2018, 12:07:08 amUsing an older, more stable version is probably a better choice. I'm never using the latest GCC version. Always an older version (previous major version number) with a lot of bugs fixed. When it comes to software remember 'newer is always worse'.

author: IanB
author_rank: Super Contributor
timestamp: December 10, 2018, 12:43:39 am
content: Here's just a thought. It was hard to tell from the screen captures, but it looks like you might be passing lots of object files to the linker in the link command?Have you tried putting all your object files into a library first, and then passing the library to the linker so it can resolve symbols that way? It might be more efficient for the linker than forcing it to consume all the object files directly.

author: lucazader
author_rank: Regular Contributor
timestamp: December 10, 2018, 12:45:01 am
content: I think ataradov is more questioning that the current official release of the arm-none-eabi toolchain is the Q2 release, which on linux at least is 7.3.1.

author: hwti
author_rank: Contributor
timestamp: December 10, 2018, 12:57:44 am
content: You should at least try with the latest 7-2018-q2 release fromhttps://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads(even only the ld.exe if you have issues with the other changes).As their binary releases aren't built with debug information, you won't get a callstack unless you rebuild the toolchain, which should be done on Ubuntu if you follow the official instructions (and adding "--build_type=native,debug" argument to their build-toolchain.sh, else they strip the binaries).So I think it would be easier if you could reproduce the problem on Linux (you don't need to build the whole project, you can reuse the object files and the custom linker script if there is one).Also, you are using ARM's toolchain, not official GCC/binutils releases, so the bugs should be reported athttps://bugs.launchpad.net/gcc-arm-embedded/(maybe the bug doesn't even exist on binutils master branch, only on ARM ones).

author: modrobert
author_rank: Regular Contributor
timestamp: December 10, 2018, 10:44:51 am
content: Compile on Linux, if that fails do 'strace' in build script for linker on Linux.

author: Kilrah
author_rank: Unknown
timestamp: December 10, 2018, 11:45:05 am
content: May not solve this particular issue, but as people pointed out on Youtube NEVER build code in a OneDrive/Dropbox folder. The build process creates/modifies/deletes hundreds/thousands of small files during build and when in such a shared folder it frequently causes things to break due to the sync program locking files while it tries to transfer them online.If you need to put the work online use a proper repo system made for code but not an automated sync system.The issue is very unlikely to be the linker itself, but something in your code causing it to barf. As someone also suggested on YT, I imagine that there was a point where it still built. Should look very closely, maybe post what you changed when it started failing.

author: mywave
author_rank: Newbie
timestamp: December 10, 2018, 11:57:41 am
content: It is not gcc that needs to be compiled with debug-information, but binutils (binutils includes all the low-level utilities).

author: mywave
author_rank: Newbie
timestamp: December 10, 2018, 12:06:01 pm
content: David, do you have tar-ball / .zip of all the files you are trying to link, and the ld command that is failing?

author: thm_w
author_rank: Super Contributor
timestamp: December 10, 2018, 07:58:40 pm
content: Quote from: Kilrah on December 10, 2018, 11:45:05 amyep, development will only "completely stop" if you decide to let it block you. Go back to the last working build, and either figure out what change broke it, or work on other functionality.

author: BierOrk
author_rank: Newbie
timestamp: December 10, 2018, 09:22:16 pm
content: Quote from: Kilrah on December 10, 2018, 11:45:05 amUsing OneDrive/Dropbox can definitly mess up your build process. Make uses timestamps to decide wether a file has been modified since the last time it has been compiled. Even a network mount can be upsetting make.An easy way to debug is to copy the source code and compile it with make -BMaintainig your source code with git can help you to get back to a working version.

author: nopjmp
author_rank: Newbie
timestamp: December 11, 2018, 02:41:01 am
content: Made a forum account since most people might not realize this.GCC was designed around large stack sizes. The easiest way to get this to work is modify the binaries stack size to a the same as Linux.Default stack sizes:Windows: 1MBLinux: 8MiBCode:editbin /STACK:reserve[,commit]So I would probably try playing around with this. You will need to be in "Developer command Prompt for VS" which should be in the Visual Studio menu.For example:Code:editbin /STACK:8388608 ld.exeYou can be more conservative with 2097152 (2MB) but since this is a developer tool, I don't see a problem with using 8MB.Note: I probably have the MiB/MB wrong since I'm finding conflicting material. I apologize if it's confusing.

author: modrobert
author_rank: Regular Contributor
timestamp: December 11, 2018, 04:09:43 am
content: Would be great if David replied, assuming he has a forum account, if nothing else just to confirm he reads this thread.

author: maginnovision
author_rank: Super Contributor
timestamp: December 11, 2018, 06:55:03 am
content: I've had plenty of issues like this it's usually easiest to undo what you did that made it break and carefully analyze that code to see if anything is odd. If that doesn't work you can start commenting out blocks. If you haven't been using any version control, even if just your memory, then you sort of just need to go through everything. Do you compile with -wall? Without seeing the code I can only suggest the basic diagnostic method.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: December 11, 2018, 07:24:45 am
content: ..but it's Open Source, so you can fix issues like this yourself...Seems you get what you pay for.BTW there are code-limited free versions of IAR and other professional tools, which ought to be enough for something a simple as a power supply...I'll get my coat.

author: ataradov
author_rank: Super Contributor
timestamp: December 11, 2018, 07:27:21 am
content: When I was working on a big project (ZigBee stack, ~150K binary), we have reported numerous bugs to IAR, it is no better. The first few were painful, since we had to go through the support people. At some point we've just got direct contact with the engineers and things got better.

author: nctnico
author_rank: Super Contributor
timestamp: December 11, 2018, 10:30:41 am
content: Quote from: ataradov on December 11, 2018, 07:27:21 amNot just that but ARM GCC is maintained by ARM themselves so saying it is free isn't exactly right.

author: Gabri74
author_rank: Regular Contributor
timestamp: December 11, 2018, 01:52:48 pm
content: From the video it seems you are using Eclipse/CDT + gnu arm toolchain.Have you tried building the same project using Atollic True Studio?It ships with modified versions of gcc/gdb.Since it's based on Eclipse and officially supported by ST it could work.https://atollic.com/truestudio/

author: Gabri74
author_rank: Regular Contributor
timestamp: December 11, 2018, 02:00:24 pm
content: Quote from: ataradov on December 11, 2018, 07:27:21 amThis ^^^Same with Keil. Despite having a full-payed license once had to download a pirated copy of their RTX RTOS to fix a bug they weren't so inclined to fix...Last week spent one day debugging a code I swear on my life was correct but didn't work as expected (a simple FIFO queue).Know what? Changed compiler revision and magically worked. Spent 3 hours digging trough change logs and found thisQuoteThis was the (pseudo-)code that triggered the bug (not kidding...)Code:do {if (queue->buf == MAX)do_something();} while (cnt--)So paid does not always means better...

author: janoc
author_rank: Super Contributor
timestamp: December 11, 2018, 04:16:43 pm
content: Quote from: nctnico on December 11, 2018, 10:30:41 amIn what sense does it make it "not free"? That's like saying that since RedHat maintains a lot of GCC/employs many maintainers "saying GCC is free isn't exactly right"."Free-ness" of software doesn't depend on who maintains the code but on the license (when talking free as in freedom) and on price (when talking free as in beer). In GCC case the license is still GPL and it is still a free download, both the "normal" and the ARM versions.

author: nctnico
author_rank: Super Contributor
timestamp: December 11, 2018, 04:58:08 pm
content: Quote from: janoc on December 11, 2018, 04:16:43 pmNot free as in: with every ARM device you buy you pay for the development of gcc for the ARM platform.It is kind of like (commercial) TV and radio aren't free even though you don't have to pay anything to the TV station. With every product you buy you pay for TV and radio.

author: Kilrah
author_rank: Unknown
timestamp: December 11, 2018, 05:01:53 pm
content: He's probably twisting it in the sense that "we all pay a little something for it when we buy something with an ARM core".

author: T3sl4co1l
author_rank: Super Contributor
timestamp: December 11, 2018, 05:05:21 pm
content: Quote from: mikeselectricstuff on December 11, 2018, 07:24:45 amHey Mike, why don't you go PIC your nose...(...No, I don't have anything to contribute to this thread. I use avr-gcc and have found zero problems with it.)Tim

author: ataradov
author_rank: Super Contributor
timestamp: December 11, 2018, 05:15:25 pm
content: I think the point of the video was not to solve the problem, but to show just how horrible tools are, since the most obvious solution or at least path to a workaround was mentioned here multiple times.AVR-GCC toolchain is infamous with its "relocation truncated to fit: R_AVR_13_PCREL" liner errors that pop up for multiple reasons, so they get fixed and pop up again. This happens from time to time, not a huge deal, just rearrange some stuff in the code and it goes away.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: December 11, 2018, 05:31:33 pm
content: Quote from: ataradov on December 11, 2018, 05:15:25 pmHmm, so a discrepancy between where the compiler expected the data to be, and where the linker wanted/had to put it? I can see that being a problem with the >128k AVRs, yes. Or more generally, any system with segmented memory, or even more generally, a platform with different sizes of pointers.And yeah, everything is horrible. Anything only ever gets fixed just enough to be usable, for some arbitrary (market-driven?*) degree of "usable". Any sufficiently large project, is too large to ever find all the bugs, or unexpected quirks, before release.*Hey, you guys all like capitalism, right? Right?...Tim

author: ataradov
author_rank: Super Contributor
timestamp: December 11, 2018, 05:39:45 pm
content: Quote from: T3sl4co1l on December 11, 2018, 05:31:33 pmNo, linker is just not able to place the code. LD does not "shuffle" the functions around, it just places things in a linear order and once placed, a function can no longer be taken out and placed somewhere else. And two or more functions may be located in a way that the distance between them is bigger than 13-bit relative offset allows.This can happen on smaller memory devices too. 13-bits it +/- 8K words, so any device with memory size over this number may be affected.And that just describes the legitimate case. This particular linker error happened in a few cases because of compiler bugs and had nothing to do with the linker itself.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: December 11, 2018, 07:04:18 pm
content: Quote from: ataradov on December 11, 2018, 05:39:45 pmThat error ought to be sanely reported, and fixed by selecting an appropriate memory model.

author: ataradov
author_rank: Super Contributor
timestamp: December 11, 2018, 07:09:24 pm
content: Quote from: mikeselectricstuff on December 11, 2018, 07:04:18 pmI'm not really sure how to report it any better short of writing a complete explanation in the error message. Given the error message it is not all that hard to find documentation explaining the error in details.You can say that compiler can try harder to avoid the issue, but it is still possible to create code that can't be properly placed in the device no matter what.Also IAR shows very similar message in this situation as well.

author: janoc
author_rank: Super Contributor
timestamp: December 11, 2018, 10:13:41 pm
content: Quote from: nctnico on December 11, 2018, 04:58:08 pmThat's a rather oddball definition, but okay. Let's not pollute the thread with more philosophical disputes.

author: janoc
author_rank: Super Contributor
timestamp: December 11, 2018, 10:29:27 pm
content: I have seen similar things happen with STM32 code as well - sometimes simply changing the optimization level for the compiler will make this go away because the compiler will inline some functions, remove unused code and suddenly the linker won't blow up anymore. My impression is that the ARM port of binutils could use a bit more love.And re Mike and the suggestion to use IAR - given that you are mostly a PIC developer, you are probably intimately familiar with the shitshow the various PIC compilers, especially for the smaller devices (PIC18 and such) are. And they were (still are?) paid. So much for the "you get what you pay for" in this case. I will use GCC or Clang (that supports ARM as well, AFAIK) any day over those things.

author: Jeroen3
author_rank: Super Contributor
timestamp: December 13, 2018, 09:39:37 pm
content: https://www.mail-archive.com/bug-binutils@gnu.org/msg29920.htmlQuoteSo, not really a bug of the linker, aside from it not reporting the stackoverflow correctly?I did see that you're running your working dir in OneDrive. I must discourage that. It has caused me strange issues before running working dirs inside cloud synced directories.

author: Brumby
author_rank: Unknown
timestamp: December 16, 2018, 02:56:02 am
content: Quote from: modrobert on December 11, 2018, 04:09:43 amYes he does. His forum name isSeppy.

author: permal
author_rank: Regular Contributor
timestamp: January 16, 2019, 11:34:37 pm
content: David,At 2:14 in the followup, you say the std::vector has massive disadvantages - what are you basing that statement on? Any particular context? Perhaps it was just a slip of the tongue, but the vector itself is allocated on the stack, its memory buffer is however allocated from the heap so not all is kept on the stack.

author: FrankBuss
author_rank: Unknown
timestamp: January 17, 2019, 07:59:12 am
content: Looks like the right way for GNU would be to undefine CP_DYNAMIC_ARRAYS, if they are still using this in the latest version, so that it uses the heap allocation part.