[SOLVED] building a metalbare cross compiler from scratch (custom configured) - Page 1
Forum section: electronics / general

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 03:42:06 pm
content: hi guysI need to compile gcc,target=mips-be(host=arm-le, I can native compile under Qemu/Arm)I am a bit confused by the GNU documentation, the following is my current-build-scriptCode:VERSION=v4.4.3export TARGET=mips-idp-elfexport PREFIX=/usr/$TARGETexport PATH=$PATH:$PREFIX/bintarget=build/$TARGET-gcc-$VERSIONecho "building gcc"rm -fr $targetmkdir -p $targetcd $target../../src/gcc-$VERSION/configure \--target=$TARGET \--prefix=$PREFIX \--disable-libssp \--without-headers \--with-float=soft \--without-newlib \--with-gnu-as \--with-gnu-ld\--disable-nls \--disable-threads \--disable-shared \--disable-libssp \--enable-languages=cmake allmake installI need to compilea crosscompilerC language onlytarget=mips big endianprofiled for metalbare (no OS)I do not need any newlibc (by Cygnus/RedHat) or glibc(by GNU) or whatever (1)(1) I do not need any newlibc or glibc or whatever, because i have to use my C runtime (crt0) and my BSP, which already includes the full support for libc* * * My questions * * *1) is the configuration correct? Do I miss something?2) is there a way to reduce the time needed to complete the build-phase? I have seen the compiler is rebuild 3 times3) I do not like the "install-path" (see the following), I'd like to have3.1) { cc1, collect2 } installed in "/usr/mips-idp-elf/bin2" instead of "/usr/mips-idp-elf/libexec/gcc/mips-idp-elf/4.4.3/", how to do so ?3.2) I do not need "/usr/mips-idp-elf/lib/*", and I'd like to have it installed in "/usr/mips-idp-elf/lib" instead of "/usr/mips-idp-elf/lib/gcc/mips-idp-elf/4.4.3/", how to do so ?3.3) I'd like to have include-path installed in "/usr/mips-idp-elf/include", how to do so ?Code:# ls /usr/mips-idp-elf/bin/ include/ info/ lib/ libexec/ man/ mips-idp-elf/ share/# ls /usr/mips-idp-elf/libexec/gcc/mips-idp-elf/4.4.3/cc1 collect2 install-tools/# ls /usr/mips-idp-elf/lib/gcc/mips-idp-elf/4.4.3/crtbegin.o crti.o el/ include-fixed/ libgcc.a soft-float/crtend.o crtn.o include/ install-tools/ libgcov.athank you in advanceedit:I forgot: mips-binutils already compiled and installed in "usr/mips-idp-elf"no problem with them

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 03:45:42 pm
content: I am studyinggnu dot org configure, but …

author: nctnico
author_rank: Super Contributor
timestamp: November 03, 2015, 04:05:30 pm
content: AFAIK compiling GCC from scratch = world of painI'd look for a known-good build script or pre-build toolchain and go from there. There are options to make gcc/ld ignore the standard libraries and startup files. I recommend to use a prebuild GCC and use those options for bare metal work.

author: andersm
author_rank: Super Contributor
timestamp: November 03, 2015, 04:17:07 pm
content: Whatever reasons you think you have for breaking the toolchain are not valid.

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 04:50:30 pm
content: Quote from: nctnico on November 03, 2015, 04:05:30 pmunfortunately it's not already existing, and what exists doesn't satisfy my needs, so I have to recompile(I am not a GNU fan, so you can understand I am not happy to have to deal with their sources, but I have to)

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 04:54:03 pm
content: Quote from: andersm on November 03, 2015, 04:17:07 pmcould You answer my specific questions instead?

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 05:06:36 pm
content: ( specifically host=armv5tejl-linux-gnueabi )

author: janoc
author_rank: Super Contributor
timestamp: November 03, 2015, 05:12:42 pm
content: Hello,I haven't built GCC for mips, but I did compile it for AVR and ARM before. So take my opinion with a grain of salt.Quote from: legacy on November 03, 2015, 03:42:06 pmAFAIK, the config above will build a native compiler (i.e. something that will run on a MIPS machine), not a crosscompiler. You need to add the --host option for that.You can look here for instructions and cherrypick the relevant bits (they are compiling a toolchain targetting Linux on MIPS, though - different TARGET):https://www.linux-mips.org/wiki/ToolchainsQuoteThen just don't compile newlib/glibc.QuoteIf I remember right, GCC builds first a bootstrap compiler, then it uses that to compile the gcc version you want to build and finally uses the self-compiled gcc to rebuild itself to ensure correctness. So no, from this point of view there isn't a way to speed it up.What you can play with is to ensure that flags such as -pipe are set, that will avoid using temporary files between preprocessor and compiler and compiler and assembler, using pipes instead. I wouldn't recommend doing parallel build (make -j X, where X is the number of parallel builds you want) unless the GCC doc says this is OK - the GCC build is very complex and things could go wrong if it is not adapted for parallel compilation.QuoteNo idea how to do this - you may have to hack on the makefiles. I suggest that you ask on the GCC mailing list, because GCC expects these files in certain places and if you change the path it could break stuff.

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 05:13:41 pm
content: Code:PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND2544 root 40 0 52112 35812 272 R 99.2 58.4 18:53.71 genattrtabalso I am not happy with "genattrtab", because it eats more than 180Mbyte of ram during the gcc-buildup( I add to add 1Gbyte of ram to Qemu/arm, in order to reduce the swap size, which is handled to the virtual hard drive ---> too slow)

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 05:19:48 pm
content: Quote from: janoc on November 03, 2015, 05:12:42 pmit works perfectly!as I wrote above the whole build up is invoked underQemu/Arm!so host=arm by default, host=arm, target=mips ---->it's a cross compiler!!!Quote from: janoc on November 03, 2015, 05:12:42 pmYes, but you have to specify if you need support (new lib? I have forced --no-xxx), and I also want to avoid libgloss to be compiles: I do not need it!Quote from: janoc on November 03, 2015, 05:12:42 pmso --disable-bootstrap isn't working?Quote from: janoc on November 03, 2015, 05:12:42 pmOK

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 05:24:03 pm
content: Quote from: legacy on November 03, 2015, 05:19:48 pmsee the logCode:building gccchecking build system type... armv5tejl-linux-gnueabichecking host system type... armv5tejl-linux-gnueabichecking target system type... mips-idp-elf

author: Lukas
author_rank: Frequent Contributor
timestamp: November 03, 2015, 09:10:57 pm
content: Have you tried buildroot? I managed to make it produce a working mips gcc some time ago with little effort.

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 09:59:58 pm
content: Quote from: Lukas on November 03, 2015, 09:10:57 pmdoes it solve my specific questions ?

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 10:09:14 pm
content: subscribed to - gcc-digest at gcc dot gnu dot org -

author: dgtl
author_rank: Regular Contributor
timestamp: November 03, 2015, 10:29:28 pm
content: Why not using crosstool-ng? It helps with a kernel-style menuconfig menu configuring all of the options you might otherwise not think of.

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 10:58:03 pm
content: Quote from: dgtl on November 03, 2015, 10:29:28 pmbecause it does not matter with my purpose!

author: janoc
author_rank: Super Contributor
timestamp: November 03, 2015, 11:00:29 pm
content: Quote from: legacy on November 03, 2015, 05:19:48 pmAh okay, my bad. You are right, of course.Quote=> GCC mailing list, that is a fairly non-standard setup.QuoteYou didn't have --disable-bootstrap in your configure options if I see correctly. Furthermore, I wouldn't recommend that, because the compiler you may have on your host may not be able to build the gcc - e.g. some commercial Unix systems come with a really ancient compiler and the bootstrap was designed to overcome this.If you are building on Linux with a reasonably recent gcc it will probably work, but it is better to be safe than chase weird bugs because some obscure code got optimized wrong.However, if I understand this page correctly:https://gcc.gnu.org/install/build.htmlthe bootstrapping is used only for building native compilers, not a crosscompiler. So this could be a moot point.On the other hand, they say that parallel building is possible, so you may want to try make -j X build. However, if you are building in QEmu, it will likely not help much - there the bottleneck is the emulated CPU and amount of RAM and a parallel build will only make things go even slower.

author: janoc
author_rank: Super Contributor
timestamp: November 03, 2015, 11:05:49 pm
content: Quote from: legacy on November 03, 2015, 10:58:03 pmBefore you yell at people I suggest you actually look at those tools - both buildroot and crosstool-ng. Both are designed to do what you are attempting to do and it is possible that the things you are looking for are configurable there. Worth checking them out, IMHO, instead of trying to reinvent the wheel.People are trying to help you here, mate.

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 11:52:06 pm
content: Quote from: janoc on November 03, 2015, 11:00:29 pmyes, it's slower, no help from the j-flag when you are under QEmu (and see how much memory it eats)I am manually trying different options (I have asked the right combination to GNU email list)the GNU documentation is not so clear to me too, their sources appears too bloated, which is confusingI am really afraid I have to hack the Gcc's Makefile to force thingsor probably I'd better hack the binary "gcc" in order to force the "collector" to point to the wanted-path"cc1" is the C compiler, "gcc" invokes it, so I could, eventually trash "gcc" and build my-own version of the "collector"I already have my builder (already tested), so a project work-flow will look this way{ list of file.s } ----> GNU.as ( my FLAGs ) ----> { list of file.o }{ list of file.c } ----> GNU.cpp (1) --> GNU.cc1 ( my FLAGs ) ----> {{ list of file.o } + { list of file.S }} ( I want to see the assembly level ){ { list of file.o } + myBSP + myLIBC.o + myCRT0.o } ---> GNU.ld ( my flags + linker script + memory model ) ---> binary (elf+debug symbols)binary.elf ---> GNU.objdump (strip) ---> motorola/s19no doubt that it's a bit crude, but it looks like a practical solution to have everything under control ( including paths )and it avoids me to waste too much time understanding whatever is GNU-skilled-only(1) GCC has a number of phases to its compilation, and it uses different internal commands to do each phase.C in particular is first preprocessed with cpp, then is compiled into assembly, assembled into machine language, and then linked together.cc1is the internal command which takes preprocessed C-language files and converts them to assembly.It's the actual part that compiles CInterestingreadabout Gcc's Architecture

author: legacy
author_rank: Super Contributor
timestamp: November 03, 2015, 11:59:17 pm
content: Quote from: janoc on November 03, 2015, 11:00:29 pmups, I forgot to tell you about thatCode:QEmu-arm # gcc -vUsing built-in specs.COLLECT_GCC=/usr/armv5tel-softfloat-linux-gnueabi/gcc-bin/4.8.4/gccCOLLECT_LTO_WRAPPER=/usr/libexec/gcc/armv5tel-softfloat-linux-gnueabi/4.8.4/lto-wrapperTarget: armv5tel-softfloat-linux-gnueabiConfigured with: /var/tmp/portage/sys-devel/gcc-4.8.4/work/gcc-4.8.4/configure --host=armv5tel-softfloat-linux-gnueabi --build=armv5tel-softfloat-rThread model: posixgcc version 4.8.4 (Gentoo 4.8.4 p1.6, pie-0.6.1)

author: janoc
author_rank: Super Contributor
timestamp: November 04, 2015, 09:13:13 am
content: Silly question, but do you need to host the compiler on that particular (and ancient - ARM5? Whoa - are you planning to compile on a Sharp Zaurus?) architecture? If you could compile on real hw (e.g. in native Linux or at least in a fast Linux VM like VirtualBox), it would certainly be a lot faster.Even if you need to host on ARM, a newer system running on something like Raspberry Pi or BananaPi (or one of the many ARM Linux SBCs available now) would be probably much faster than the emulation in Qemu.BTW, why are you fighting with the paths? The ones that GCC uses by default are semi-standard and other tools may actually depends on them. I wouldn't move things around unless I had a very good reason for that. The scheme GCC uses is there to permit having multiple compiler versions installed side-by-side. Wouldn't a bunch of symbolic links post install solve your problem? (I assume you only want the path changed for convenience). That's probably a lot easier and safer than hacking binaries or the build system where one doesn't understand all the dependencies.

author: legacy
author_rank: Super Contributor
timestamp: November 04, 2015, 09:47:49 am
content: Quote from: janoc on November 04, 2015, 09:13:13 amfaster than QEmu/ARM on multi core i7 machine?

author: legacy
author_rank: Super Contributor
timestamp: November 04, 2015, 10:06:22 am
content: Solved !!!it works great onQEmu/ARMtwo solutions, two branches, mutual exclusivehacked "gcc.c", forcingall the pathsI need (and I need it this way for my reasons, no time to say why/what/where)removed "gcc", swapped it with my "collector", it's a piece of software I have developed, it does the same job of "gcc" (as collector)edit: lesson learnt from GNU.gcc's configureprefixgcc_exex_prefixC_include_pathin the theory they allow you place the compiler pieces in any locationpractically(1) they get embedded within "gcc.c", in the "collector" partIn order to be sure that things get built as I expected:I had better change and modify "gcc.c" immediatelythis is my conclusion!(1) note these can be used as environment variables to move things aroundeven on a already built and working compiler, so …… they might cause troubles to the host's toolchain (it's not clear WHY?)

author: janoc
author_rank: Super Contributor
timestamp: November 04, 2015, 11:32:48 pm
content: Quote from: legacy on November 04, 2015, 09:47:49 amDid you benchmark it? If you are hitting swap at 1GB of RAM in the emulator, I think it will be faster if you take a physical, not emulated HW and 2GB of RAM. Qemu is very slow compared to native code. I would be very surprised if even multicore i7 was able to run it at near native speeds.

author: legacy
author_rank: Super Contributor
timestamp: November 05, 2015, 12:43:19 am
content: I do not have any RPI/BananaPI/whatever on hand, and no time to buy/reallocate one of them in the labwhile the customer wants a copy of the whole building system (which should run on PC), so it's not the pointabout what I could to to improve my results: I simply assigned less than 256Mbyte of ram to QEmu (my mistake)and due to the fact that gcc-build-up eats a lot of ram (more than 512Mbyte if you enable "pipe" instead of "temp files")the kernel was swapping on /dev/sda1, which was a bottleneck, and assigning 1Gbyte of ram to QEmu improved results

author: janoc
author_rank: Super Contributor
timestamp: November 07, 2015, 04:38:44 pm
content: Quote from: legacy on November 05, 2015, 12:43:19 amOK. Well, you could always deliver the build system on an SD card and with its own computerQuote from: legacy on November 05, 2015, 12:43:19 amYeah, 256MB, that's no good - gcc is a huge amount of code, even on a PC with 4GB of RAM the machine will struggle. RAM is by far the most important thing there.