[Sony Xperia Z Ultra] dbolivar:
I've recently been doing some experiments compiling ROMs with some tweaks, like changing the toolchain and applying compiler optimizations, and thought about sharing them with you, as more and more people are doing unofficial ROM builds. This is something "quick-n-dirty", there are more complete guides here on XDA if you want to understand the whole process.

I'm assuming you already know how to build a ROM, and is familiar with Linux. I won't answer questions about that: you will find everything you need using XDA and Google.

1) Replacing the default toolchain by SaberMod

I tried also Linaro, and upgrading the toolchain version (e.g. 4.7 -> 4.8), but both didn't work without some heavy changes in code. So, for something more straightforward, replace the default toolchain with the same version of SaberMod. I'm not sure if SaberMod is as fast / faster than Linaro, searched a bit but found nothing conclusive, but it should be faster than Google's toolchain.

The links below are for SaberMod 4.7:

sm_arm-linux-androideabi-4.7

sm_arm-eabi-4.7

The first one is for the ROM itself, the second one is only for the kernel (AFAIK). Now replace the default toolchain in the ROM's prebuilts with SaberMod's toolchain you've just downloaded. I usually symlink:

mv ./prebuilts/gcc/linux-x86/arm/arm-eabi-4.7 ./prebuilts/gcc/linux-x86/arm/arm-eabi-4.7.bak
mv ./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.7 ./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.7.bak
ln -s /sabermod_dir/sm_arm-eabi-4.7 ./prebuilts/gcc/linux-x86/arm/arm-eabi-4.7
ln -s /sabermod_dir/sm_arm-linux-androideabi-4.7 ./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.7

2) Changing default optimization level

I simply replace all occurences of -O2 and -Os for -O3 in the makefiles, and remove debugging symbols (-g). Note that some things may not work as intended with -O3, specially the kernel (some people recommend using only -O2 for it; anyway these changes shouldn't affect the kernel, which uses a different makefile), and without debugging symbols it can be harder to debug some issues. But you should get a faster ROM (and slightly bigger, but that's not an issue with current hardware).

find . -type f -name "*.mk" -exec grep -q "\-g\ " {} \; -print > out_g.txt; find . -type f -name "*.mk" -exec grep -q "\-O2\ " {} \; -print > out_o2.txt; find . -type f -name "*.mk" -exec grep -q "\-Os\ " {} \; -print > out_os.txt
for i in `cat out_g.txt`; do cp $i $i.bak; sed -i 's/-g //g' $i; done
for i in `cat out_o2.txt`; do sed -i 's/-O2 /-O3 /g' $i; done
for i in `cat out_os.txt`; do sed -i 's/-Os /-O3 /g' $i; done

The first replacement will create backup files, if you need to return something to default.

3) Adding architecture optimization flags

It depends on the hardware, and you can try different flags. The ones I use are:

vi build/core/combo/arch/arm/armv7-a-neon.mk

Add the line below AFTER the last "arch_variant_cflags":

arch_variant_cflags := -march=armv7-a -mtune=cortex-a15 -mfpu=neon-vfpv4 -mfloat-abi=softfp

This way, your own flags will replace whatever has been set previously in the file.

4) Adjusting the makefiles if the build breaks

With these tweaks, the build can break at different parts, depending on the ROM you are building. Usually it's just a matter of adjusting the corresponding makefile, adding / remove flags (usually disabling warnings).

For example, one build once broke with the following error:

hardware/qcom/display-caf-new/libhwcomposer/hwc_dump_layers.cpp:435:42: error: argument to 'sizeof' (...)

I solved it by doing:

vi hardware/qcom/display-caf-new/libhwcomposer/Android.mk

Add the following to the LOCAL_CFLAGS variable:

-Wno-sizeof-pointer-memaccess

Another example: when building a vanilla CM11 with SaberMod, I had to remove the flag "-fuse-ld=gold" from the makefiles, because SaberMod's toolchain doesn't have the ld.gold linker. The approach is similar to the other changes:

find . -type f -name "*.mk" -exec grep -q "\-fuse\-ld\=gold" {} \; -print > out_fuseld.txt;
for i in `cat out_fuseld.txt`; do sed -i 's/-fuse-ld=gold//g' $i; done

That's it, good luck with your now optimized builds! And feel free to share your own optimization tricks!