RISC-V Compiling and Debugging - Page 1
Forum section: electronics / general

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 04, 2023, 06:52:28 pm
content: I'm having some issues when debugging a firmware running on a RISC-V microcontroller (GigaDevice GD32VF103 or WinChipHead CH32V103)When the code is compiled with a GCC 8 Compiler (binary fromMounRiver's toolchain), all debugging appears to work as expected.However, when I compile my code with a GCC 12 Compiler (binary xPack toolchain), even though the firmware appears to operate fine, when debugging, GDB acts like the stack is corrupted.A working firmware compiled with GCC 8 gives the following backtrace.Code:(gdb) bt#0 u8g2_font_decode_len (u8g2=0x20000014 <m_u8g2>, len=0 '\000', is_foreground=0 '\000') at ../../..//ext/u8g2/csrc/u8g2_font.c:400#1 0x00006078 in u8g2_font_decode_glyph (u8g2=0x20000014 <m_u8g2>,glyph_data=0x9f6f <u8g2_font_5x8_tf+399> "\263b\263bkJ\n4b73\310 \225\021K\t4b\023U\222\251\063L\f4b\223A\006\031d\220\301\bM\t4b\023\307", <incomplete sequence \315>)at ../../..//ext/u8g2/csrc/u8g2_font.c:605#2 0x000062aa in u8g2_font_draw_glyph (u8g2=0x20000014 <m_u8g2>, x=15 '\017', y=16 '\020', encoding=73) at ../../..//ext/u8g2/csrc/u8g2_font.c:724#3 0x0000638c in u8g2_DrawGlyph (u8g2=0x20000014 <m_u8g2>, x=15 '\017', y=16 '\020', encoding=73) at ../../..//ext/u8g2/csrc/u8g2_font.c:789#4 0x00006418 in u8g2_draw_string (u8g2=0x20000014 <m_u8g2>, x=15 '\017', y=16 '\020', str=0x9ae0 "²C DEMO") at ../../..//ext/u8g2/csrc/u8g2_font.c:807#5 0x000064ec in u8g2_DrawUTF8 (u8g2=0x20000014 <m_u8g2>, x=0 '\000', y=16 '\020', str=0x9adc " I²C DEMO") at ../../..//ext/u8g2/csrc/u8g2_font.c:861#6 0x00001074 in print (str=0x9adc " I²C DEMO", line=1) at dis2.c:50#7 0x00000366 in main () at main.c:174A working firmware compiled with GCC 12 gives the following backtrace.Code:(gdb) bt#0 u8g2_font_decode_len (u8g2=0x2000006c <m_u8g2+88>, len=4 '\004', is_foreground=0 '\000') at ../../..//ext/u8g2/csrc/u8g2_font.c:400#1 0x000060ba in u8g2_font_decode_glyph (u8g2=0x2000006c <m_u8g2+88>, glyph_data=0x4004db0 '\377' <repeats 200 times>...) at ../../..//ext/u8g2/csrc/u8g2_font.c:605Backtrace stopped: frame did not save the PC(gdb)As the firmware compiled with GCC 12 appears to work fine, it is my assumption GDB has troubles interpreting the stack, rather then the stack actually being corrupted. One note regarding the elf files generated by the toolchains, it appears the GCC8 version does not contain architecture specific information, while the GCC12 version does contain architecture specific information, stating the stack alignment is 16 byte.Code:$ readelf -A mounriver-studio-toolchain-riscv/i2c_demo.elf$ readelf -A riscv-none-elf/i2c_demo.elfAttribute Section: riscvFile AttributesTag_RISCV_stack_align: 16-bytesTag_RISCV_arch: "rv32i2p1_m2p0_a2p1_c2p0"Tag_RISCV_priv_spec: 1Tag_RISCV_priv_spec_minor: 11Could this possibly be the cause of these debugging issues? Do I have to give GDB extra commands for it to work? Or should I pass some more flags to GCC? Or adjust my LD script?I have uploaded the binaries and map files for thesehttp://www.blaatschaap.be/tmp/tests/

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 04, 2023, 10:16:21 pm
content: I have removed the architecture specific infoCode:$ riscv-none-elf-objcopy --remove-section .riscv.attributes i2c_demo.elfThis does not change the behaviour of gdb.

author: brucehoult
author_rank: Super Contributor
timestamp: February 04, 2023, 10:56:16 pm
content: Did you also update gdb when you uipdated gcc?There is not enough information in RISC-V binary code to be able to reliably form a backtrace (there is no frame pointer, you don't necessarily know the stack frame size, and also the return address could be anywhere in it). Metadata is needed, and that could well change in format between gcc 8 and gcc 12.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 05, 2023, 10:31:07 am
content: Using riscv-none-elf-gdb which came with the xPack version of the gcc 12 compiler gives this behaviour.But this brings another question. Could this relate to the ISA specification version? gcc 8 defaults to ISA 2.2, but for gcc 12, I need to specify ISA 2.2 manually, or it will default to a newer ISA. --- Still, in that case the resulting binary for both would be in ISA 2.2.

author: brucehoult
author_rank: Super Contributor
timestamp: February 05, 2023, 11:43:55 am
content: As far as I know 2.2 is the newest version of the User-mode instructions, and even then only floating point had changes in 2.2, last changes to RV32I and RV64I were in 2.1.There will NEVER be incompatible changes.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 05, 2023, 12:01:29 pm
content: It's my makefiles that require some updating in that regard.The zicsr extension was split of from the i extension, so it would require replacing "-march=rv32imac" with "-march=rv32imac_zicsr" when building.This leads to an issue with the riscv64-elf-gcc from the Arch repo, as the linker recognise it should link the rv32imac libgcc and tries to link a 64 bit version. Building with "-march=rv32imac_zicsr" and linking with ""-march=rv32imac" appears to work. But those are unrelated issues.

author: brucehoult
author_rank: Super Contributor
timestamp: February 05, 2023, 01:10:10 pm
content: Quote from: GromBeestje on February 05, 2023, 12:01:29 pmOh, right.That change was pre-ratification in July 2019, so it's not formally an incompatibility. History starts from ratification, everything before that was just something experimental that resembled RISC-V to a greater or lesser extent.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 05, 2023, 08:04:32 pm
content: GCC 8 is pretty old at this point. I wouldn't use it for any RISC-V development unless I had a gun held to my head.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 06, 2023, 06:52:06 pm
content: It is... that's why I wish my GDB would play nice with newer GCC versions... I've been breaking my head over this issue.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 06, 2023, 09:53:42 pm
content: Symptom or cause?Code:Call Frame Instruction op 36 in vendor extension space is not handled on this architecture.

author: brucehoult
author_rank: Super Contributor
timestamp: February 06, 2023, 11:00:20 pm
content: Quote from: GromBeestje on February 06, 2023, 09:53:42 pmThat message relates to ARM Aarch64, not to RISC-V.Try using a RISC-V gdb version.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 07, 2023, 08:27:46 pm
content: It looks they are riscv, not arm. So unless something is wrong with the gdb build scripts... these are riscv builds.And even if this were the problem, it wouldn't explain why code compiled with gcc 8 (MounRiver) and code compiled with gcc 12 (Arch, xPack) doesn't.Code:[andre@mortar ~]$ riscv32-elf-gdb --configurationThis GDB was configured as follows:configure --host=x86_64-pc-linux-gnu --target=riscv32-elf--with-auto-load-dir=$debugdir:$datadir/auto-load--with-auto-load-safe-path=$debugdir:$datadir/auto-load--with-expat--with-gdb-datadir=/usr/share/gdb (relocatable)--with-jit-reader-dir=/usr/lib/gdb (relocatable)--without-libunwind-ia64--with-lzma--without-babeltrace--without-intel-pt--with-mpfr--without-xxhash--with-python=/usr (relocatable)--with-python-libdir=/usr/lib (relocatable)--with-debuginfod--with-guile--enable-source-highlight--with-separate-debug-dir=/usr/lib/debug (relocatable)--with-system-gdbinit=/etc/gdb/gdbinit("Relocatable" means the directory can be moved with the GDB installationtree, and GDB will still find it.)Code:[andre@mortar ~]$ riscv-none-elf-gdb --configurationThis GDB was configured as follows:configure --host=x86_64-pc-linux-gnu --target=riscv-none-elf--with-auto-load-dir=$debugdir:$datadir/auto-load--with-auto-load-safe-path=$debugdir:$datadir/auto-load--with-expat--with-gdb-datadir=/Host/home/ilg/Work/riscv-none-elf-gcc-12.2.0-1/linux-x64/install/riscv-none-elf-gcc/riscv-none-elf/share/gdb (relocatable)--with-jit-reader-dir=/Host/home/ilg/Work/riscv-none-elf-gcc-12.2.0-1/linux-x64/install/riscv-none-elf-gcc/lib/gdb (relocatable)--without-libunwind-ia64--without-lzma--without-babeltrace--without-intel-pt--with-mpfr--without-xxhash--without-python--without-python-libdir--without-debuginfod--without-guile--disable-source-highlight--with-separate-debug-dir=/Host/home/ilg/Work/riscv-none-elf-gcc-12.2.0-1/linux-x64/install/riscv-none-elf-gcc/lib/debug (relocatable)--with-system-gdbinit=/Host/home/ilg/Work/riscv-none-elf-gcc-12.2.0-1/linux-x64/install/riscv-none-elf-gcc/riscv-none-elf/lib/gdbinit (relocatable)("Relocatable" means the directory can be moved with the GDB installationtree, and GDB will still find it.)Edit:formatting

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 10, 2023, 07:18:41 pm
content: After some more testing, it seems when all source files are given to GCC at once, the debugging works fine,When I compile all source files to object files and then link them together, the debugging fails.When building everything in one go, debugging succeeds (paths shorted to shrink the length)Code:riscv-none-elf-gcc ch32.S gd32.S rv32f103.S irq.S gd32_intexc.S gd32_intexc.c ch32.c gd32.c common.c main.c -I NMSIS/Core/Include/ -o test.gcc12.elf -T rv32f103.ld --specs=nosys.specs --specs=nano.specs -nostartfiles -Wl,--no-relax -Wl,--gc-sections -march=rv32imac -mabi=ilp32 -mcmodel=medlow -fdata-sections -ffunction-sections -misa-spec=2.2 -O0 -g3 -Wl,-Map=test.gcc12.mapWhen building every file into its own object, then linking it all together, debugging fails (paths shorted to shrink the length)Code:riscv-none-elf-gcc -c -INMSIS/Core/Include/ -Iucdev/lib/libhalglue/common/ -Iucdev/lib/libhalglue/bshal/ -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -DNONE -DNONE -I -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/main.c.d" -DDEBUG -Wa,-a,-ad,-alms=debug/none/build/main.lst ../main.c -o debug/none/build/main.c.oriscv-none-elf-gcc -c -INMSIS/Core/Include/ -Iucdev/lib/libhalglue/common/ -Iucdev/lib/libhalglue/bshal/ -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -DNONE -DNONE -I -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/gd32_intexc.c.d" -DDEBUG -Wa,-a,-ad,-alms=debug/none/build/gd32_intexc.lst ucdev/lib/libhalglue/compat/rv32f103/gd32_intexc.c -o debug/none/build/gd32_intexc.c.oriscv-none-elf-gcc -c -INMSIS/Core/Include/ -Iucdev/lib/libhalglue/common/ -Iucdev/lib/libhalglue/bshal/ -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -DNONE -DNONE -I -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/ch32.c.d" -DDEBUG -Wa,-a,-ad,-alms=debug/none/build/ch32.lst ucdev/lib/libhalglue/compat/rv32f103/ch32.c -o debug/none/build/ch32.c.oriscv-none-elf-gcc -c -INMSIS/Core/Include/ -Iucdev/lib/libhalglue/common/ -Iucdev/lib/libhalglue/bshal/ -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -DNONE -DNONE -I -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/gd32.c.d" -DDEBUG -Wa,-a,-ad,-alms=debug/none/build/gd32.lst ucdev/lib/libhalglue/compat/rv32f103/gd32.c -o debug/none/build/gd32.c.oriscv-none-elf-gcc -c -INMSIS/Core/Include/ -Iucdev/lib/libhalglue/common/ -Iucdev/lib/libhalglue/bshal/ -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -DNONE -DNONE -I -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/common.c.d" -DDEBUG -Wa,-a,-ad,-alms=debug/none/build/common.lst ucdev/lib/libhalglue/compat/rv32f103/common.c -o debug/none/build/common.c.oriscv-none-elf-gcc -x assembler-with-cpp -c -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/ch32.S.d" ucdev/lib/libhalglue/compat/rv32f103/ch32.S -o debug/none/build/ch32.S.oriscv-none-elf-gcc -x assembler-with-cpp -c -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/gd32.S.d" ucdev/lib/libhalglue/compat/rv32f103/gd32.S -o debug/none/build/gd32.S.oriscv-none-elf-gcc -x assembler-with-cpp -c -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/rv32f103.S.d" ucdev/lib/libhalglue/compat/rv32f103/rv32f103.S -o debug/none/build/rv32f103.S.oriscv-none-elf-gcc -x assembler-with-cpp -c -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/irq.S.d" ucdev/lib/libhalglue/compat/rv32f103/irq.S -o debug/none/build/irq.S.oriscv-none-elf-gcc -x assembler-with-cpp -c -INMSIS/Core/Include/ -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 -O0 -g3 -Wall -fdata-sections -ffunction-sections -MMD -MP -MF"debug/none/build/gd32_intexc.S.d" ucdev/lib/libhalglue/compat/rv32f103/gd32_intexc.S -o debug/none/build/gd32_intexc.S.oriscv-none-elf-gcc debug/none/build/main.c.o debug/none/build/gd32_intexc.c.o debug/none/build/ch32.c.o debug/none/build/gd32.c.o debug/none/build/common.c.o debug/none/build/ch32.S.o debug/none/build/gd32.S.o debug/none/build/rv32f103.S.o debug/none/build/irq.S.o debug/none/build/gd32_intexc.S.o -nostartfiles -Wl,--no-relax -march=rv32imac -mabi=ilp32 -mcmodel=medlow -misa-spec=2.2 --specs=nosys.specs --specs=nano.specs -Trv32f103.ld -L.. -L.. -Wl,--gc-sections -Wl,-Map=debug/none/rv32test.map,--cref -o debug/none/rv32test.elf

author: eutectique
author_rank: Frequent Contributor
timestamp: February 10, 2023, 07:28:07 pm
content: You do not pass-goption to the linker. Could it be the reason? Do you see.debug_*sections when you do*-*-objdump -h your.elf?(Speaking from my ARM and x86 experience, never used RISC-V)

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 10, 2023, 09:06:06 pm
content: For both files, compiled on the same line, or file by file, I've got debug sections present.I'm coming from ARM as well. Got my Makefiles and ld files from there...I've been looking at these files, on first glance, disassembly seems to be the same, but the order of functions is different. (as the order the objects are listed differs) -- this could affect weak function linkage... but that would be a different problem then this.Still linking order might be the next thing to investigate.Edit: making the order I give the .o files the same as the source files in the working build makes no difference regarding the issue.

author: GromBeestje
author_rank: Frequent Contributor
timestamp: February 11, 2023, 09:43:28 am
content: Some more testing... now, I can remove all my startup code again and revert to the minimal startup to reduce the complexity of the output.The conclusion is, the produced binary contains the same code. That isCode:riscv-none-elf-objdump -dproduces the same output (except the filename of course)This means, the problem is not in the code produces. As the problem occurs while debugging, my suspect is the produced DWARF.Looking into the DWARF, I've used dwarfdump on the elf files,The bad elf file gives me some errors in the output, while the good elf file doesn'tCode:ERROR: on getting fde details for fde row for address 0x0000006aERROR: Printing fde 0 fails. Error DW_DLE_DF_FRAME_DECODING_ERROR: instr opcode 0x18 unknownERROR: printing fdes fails. DW_DLE_DF_FRAME_DECODING_ERROR: instr opcode 0x18 unknown Attempting to continue.There were 3 DWARF errors reported: see ERROR above.This looks to me like I've hit a bug ingccbinutils.Edit 1: test case on github:https://github.com/a-v-s/riscv_gcc_debug_issueEdit 2: I think I have found the trigger. Generating the assembly listing when compiling the C file seems to result in the DWARF error.Edit 3: As the request to generate assembly listing is passed to the assembler, I'd say the problem is located in binutils