yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] t4n017:
Yep, could be that.

I always used Mint 16 to build and that error never happened. But recently I switched to elementary OS Luna (which is based on Ubuntu 12.04) and I had OOM once.
[Android Software/Hacking General [Developers Only]] sidneyk:
Just did a build of Omnirom 4.4.3_r1.1 for Asus TF101 on my Archlinux build box, using Sabermod 4.8.4 prebuilt tool chains and all of your v.3 optimizations and the ART patch. Happy to report no graphics issues on latest Play store and no storage visibility issues. All is butter smooth.
[Android Software/Hacking General [Developers Only]] JustArchi:
Minimum of 8 GB memory (RAM + Swap in total) is required.
[Android Software/Hacking General [Developers Only]] Balino:
I'm about to buy a i7 2 GHz and a 8 gb RAM .... Is it enough to build with this optimizations?
[Android Software/Hacking General [Developers Only]] spanish33:
I have a laptop with an i5 with 8gigs assigned to virtual machine and it takes about two and half hours to do a clean build with ccache enabled after first build. First build with ccache is about 7 hours.
[Android Software/Hacking General [Developers Only]] JustArchi:
Make sure you have all CPUs assigned. I'm building with 4 GB, 4 assigned CPUs from i7-2600k, about 90 minutes clean build with empty ccache.
[Android Software/Hacking General [Developers Only]] spanish33:
Yes all CPUs assigned, I believe my bottle neck is my 5400rpm hard drive . At the time I bought the laptop for school and I didn't realize I would start compiling roms for my phone.
[Android Software/Hacking General [Developers Only]] r0gger:
Hello all,

Regarding this:

Important changes:

- Added missing Cortex-A9 CPU variant (-mcpu=cortex-a9)

Can we use something similar, to enable cortex-a15 optimizations? I know that SGS3 is cortex-a9, but most of the phones after, are a15 based.

Regards

---------- Post added at 09:11 PM ---------- Previous post was at 08:51 PM ----------

It can be related with almost everything.

First check your cpu score at http://cpubenchmark.net/cpu_list.php (my Q6600 is with score - 2978 (at 2.4, but currently it is running at 2.8Ghz). The laptop CPU is I5 is with score 3978. So my compile with Debian GNU/Linux testing using the i5 machine, finishing for 2-3 hours with 1.5GB of ram and 4GB swap, with make -j2. I am talking about this, because some of the new laptops are with CPU with score something like 700-1000 .. so it will be veeery slow to build android with such cpu.

Currently the vm mem is increased to 3G and I'll try the same. If you enable all the cores of your CPU to the vm, you will receive a warning from virtualbox (if this is your VM software), that your speed will be reduced because of that. So I enabled only 3 of my CPUs (Mine cpu is dual core with hyper threading). Using the Quad core machine was the same, if you enable all the cpu cores, the speed is reduced somehow .. So for the old Q6600 machine, I just used native booted Debian to compile the source, with make -j8 ..

But as Winnie Pooh says - "The more - the more" ..

Try to enable as much as possible CPU cores (save some for your host os if you running a VM), give as much as you can RAM (with 8GB machine, you can use 4 or even 6GB for the VM ..
[Android Software/Hacking General [Developers Only]] JustArchi:
The core-word is "missing", A15 is already present.
[Android Software/Hacking General [Developers Only]] r0gger:
Hey JustArchi, thanks for the reply.

So in my case, I am building for mako (N4), and I applied your customizations V3 to my local repo. How and where A15 optimizations are selected/enabled?. (using A9 opts are doing this?) or lunch build-mako probably?. Or just the basic/default google (without apply your customizations) make files are selecting these?

Regards!
[Android Software/Hacking General [Developers Only]] JustArchi:
CPU Architecture is specified in target device, usually in device/vendor/model, such as device/samsung/i9300.

https://github.com/omnirom/android_device_samsung_i9300/blob/android-4.4/BoardConfig.mk#L20

https://github.com/omnirom/android_...mon/blob/android-4.4/BoardCommonConfig.mk#L29

Cortex A9 was missing in main android_build repo as a valid target, that's all.
[Android Software/Hacking General [Developers Only]] r0gger:
Definitely makes sense. In this case, I'll ask another noobie question (yes I know, I know read the RTFM .. :D), are we talking about now, that the optimizations of yours, are related mainly only for SGS3 like devices with A9 CPU and build type?

Regards
[Android Software/Hacking General [Developers Only]] JustArchi:
No! This is only one point added in latest V3 to the others, check commit itself. My optimizations apply to all devices and all ROMs (built from source).
[Android Software/Hacking General [Developers Only]] dragonnn:
I have standalone kernel, well I did this changes to the Makefile:

diff --git a/Makefile b/Makefile
index 91f73f5..1f8a465 100644
--- a/Makefile
+++ b/Makefile
@@ -369,7 +369,29 @@ KBUILD_CFLAGS := -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-Werror-implicit-function-declaration \
-Wno-format-security \
-fno-delete-null-pointer-checks \
-	-ffast-math
+	-ffast-math \
+	-DNDEBUG -funsafe-loop-optimizations \
+	-fsection-anchors \
+	-fivopts \
+	-ftree-loop-im \
+	-ftree-loop-ivcanon \
+	-ffunction-sections \
+	-fdata-sections \
+	-funswitch-loops \
+	-frename-registers \
+	-frerun-cse-after-loop \
+	-fomit-frame-pointer \
+	-fgcse-after-reload \
+	-fgcse-sm \
+	-fgcse-las \
+	-fweb \
+	-ftracer \
+	-fstrict-aliasing \
+	-Wstrict-aliasing=3 \
+	-Wno-error=strict-aliasing \
+	-Wno-error=unused-parameter \
+	-Wno-error=unused-but-set-variable \
+	-Wno-error=maybe-uninitialized
KBUILD_AFLAGS_KERNEL :=
KBUILD_CFLAGS_KERNEL :=
KBUILD_AFLAGS := -D__ASSEMBLY__
@@ -661,9 +683,9 @@ endif

# Use --build-id when available.
LDFLAGS_BUILD_ID = $(patsubst -Wl$(comma)%,%,\
-	$(call cc-ldoption, -Wl$(comma)--build-id,))
+	$(call cc-ldoption, -Wl$(comma)--build-id,)) -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
KBUILD_LDFLAGS_MODULE += $(LDFLAGS_BUILD_ID)
-LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID)
+LDFLAGS_vmlinux += $(LDFLAGS_BUILD_ID)

ifeq ($(CONFIG_STRIP_ASM_SYMS),y)
LDFLAGS_vmlinux	+= $(call ld-option, -X,)

Is that enough and good?
[Android Software/Hacking General [Developers Only]] JustArchi:
Looks good but you've missed -O3 :).
[Android Software/Hacking General [Developers Only]] dragonnn:
I set -03 some lines under this already long time ago so it is ok ;).
[Android Software/Hacking General [Developers Only]] helicopter88:
By the way,looks like you missed graphite(Loop optimisations) optimisations:

-ftree-loop-linear

-floop-interchange

-floop-strip-mine

-floop-block

Those do not work on every GCC Toolchain you might find as they need to be compiled with CLooG

You might want to address the aliasing violations by removing the -Wno-error,otherwise it doesn't make any sense to set the warning to level 3 and -fstrict-aliasing.

-funswitch-loops,-frerun-cse-after-loop,-fgcse-after-reload are not needed as O3 turns them on already.

You might want to separate warning options and use a variable for the flags to make the file more readable
[Android Software/Hacking General [Developers Only]] JustArchi:
Thank you :).

Actually I knew about graphite, but haven't tried it yet. It's even mentioned in the commit itself. Perhaps we can make it for "V3".

I wanted to include some useful O3 flags, such as three mentioned by you in case somebody will want to go back to O2 without needing of disabling them.

About strict aliasing warnings, thanks, I'll take a look if I can disable warnings. However -Wno-error is required because only this way GCC is allowed to ignore fstrict aliasing rules for non-valid targets while keeping for the rest. Otherwise you'd have to set no strict aliasing per each invalid target.
[Android Software/Hacking General [Developers Only]] metalspring:
FYI, if you mean graphite for the rom, that's a bad idea since bionic cannot be built with graphite optimizations and I don't think there is any real easy way around it (I tried before). For the kernel they work fine, however you also forgot one of the graphite cflags, -fgraphite-identity. You can also consider adding LTO to the kernel as well as arch and vpf optimizations.

There are also some other potentially useful cflags for the kernel, I'll post some when I get to my PC.

Also I'm not entirety sure changing the kernel makefile cflags to the same as archi has for the rom, kernels can behave very differently than the rom but at the same time there are some extra optimizations that are difficult to make work in a rom but are very easy in kernels.
[Android Software/Hacking General [Developers Only]] helicopter88:
Graphite on kernels gave a nice boost so definately try it.

You should address those violations so there is no need for the -Wno at all (android gerrit contains some fixes for those)

Now that I remember,-flto shouldn't be turned on by default,LTO is link-time-optimization,which at the expense of compile time reduces size and increases performance.

By the way,google already includes GCC 4.9 and Clang/LLVM 3.5 in their repositories,so there might be more flags to play with in the near future.

I reccomend you to switch some projects(I used to do so on projects such as libm,libnativehelper and frameworks/native) to clang and do some benchmarks (adding LOCAL_CLANG := true before target definition is enough),usually you get a nice speed boost and a lower compile time.
[Android Software/Hacking General [Developers Only]] metalspring:
You can always try using -flto=n to make the linker run in parallel threads, defined by n (I.e. -flto=4 to run the linker in 4 parallel threads) or even -flto=jobserver to run as many linker threads as defined by the jobserver (make -j"number").

I also know a couple kernel commits to add in both graphite and link time optimization in a way that's easy to enable/disable
[Android Software/Hacking General [Developers Only]] helicopter88:
-fgraphite-identity isn't really an optimisation flag,but more a flag to store the results of the optimisations,however I'm not sure how to make GCC aware of those results,

I built fine openmaster bionic with graphite some time ago,it's really possible that 4.4 (or previous bionics) do not compile at all with it

Beware with loop and math optimisation in kernel,those can make everything weird quickly(-ffast-math on my evo 3d drove battery meter mad).
[Android Software/Hacking General [Developers Only]] JustArchi:
Just a note, don't even try to compile whole ROM with -ffast-math, things like SQLite3 are not compatible and will throw errors.

Also, yes, optimizing kernel is far easier thing that optimizing whole ROM, but only optimizing whole ROM can "show" the pure speed, optimizing a kernel is a bit silly because there's very little to optimize :).
[Android Software/Hacking General [Developers Only]] metalspring:
It might depend on what bionic source, whether you have Qualcomm, etc. but I know when I tried before the bionic repo threw out a very specific error saying graphite cflags cannot be used on it (it even listed all the graphite cflags), and for ffast-math, I tried that too and the chromium repo threw out errors saying it cant be built with ffast-math

Btw [MENTION=5161352]JustArchi[/MENTION] here are the commits I use to build the kernel with graphite and LTO in case you want to reference them since they are easy to enable/disable the optimizations with just a config change

This actually changed the way graphite was enabled in the source I had but you can get the idea-

https://github.com/metalspring/andr...mmit/ea23296b9f085a62add77b3469133bb327b0abe6

https://github.com/metalspring/andr...mmit/0bf644fec2c74db6adb7ea129cee01d80c1e2b0f

Link Time Optimization-

https://github.com/metalspring/andr...mmit/ea23296b9f085a62add77b3469133bb327b0abe6

And this one was just adding in the actual config options to enable them

https://github.com/metalspring/andr...mmit/3da7b38e7b708bd0fa74538352a73333b872a012

And you can notice differences with optimizing the kernel as well, especially -march= -mtune= , mfpu= -mfloat-abi= but they really need to modified to each specific device
[Android Software/Hacking General [Developers Only]] JustArchi:
When you build kernel from whole AOSP branch (eg CM) then you don't need to worry about march mtune mfpu and mfloat-abi :).
[Android Software/Hacking General [Developers Only]] helicopter88:
No,those errors aren't due to bionic,but due to toolchain missing CLooG and/or ISL which is a requirement for graphite.

Linaro usually doesn't build toolchains with CLooG and ISL,but you can build them by yourself easily.
[Android Software/Hacking General [Developers Only]] metalspring:
I was able to build with link time optimization which also relies upon CLooG and ISL though?
[Android Software/Hacking General [Developers Only]] helicopter88:
I don't think lto relies upon cloog, anyways linaro toolchains lack graphite optimizations by default
[Android Software/Hacking General [Developers Only]] Snoop05:
:good: I like compiling CyanogenMod at my own, with -O3 and 4.8 toolchain IT WILL ROCK!

Thanks for best tutorial about optimization flahs

Cant wait when i get 256GB SSD and set up build enviroment :D

Q: Do you use "bison" or "bison++" ?
[Android Software/Hacking General [Developers Only]] desalesouche:
build error

hi sir. i followed your tutorial, i am compiling for huawei honor (u8860) mokee KK rom.

i could fix the unknow cpu error following commit you did.

now i am facing that error...

http://pastebin.com/m5xyQBVb

may i ask you if you could provide me some way to fix that?

i am using linaro 4.8 from your github...

thank you.
[Android Software/Hacking General [Developers Only]] JustArchi:
Try using Google Toolchain, if it doesn't help fallback to GCC 4.7.

https://github.com/JustArchi/ArchiDroid/blob/2.X-EXPERIMENTAL/__dont_include/roomservice.xml#L7
[Android Software/Hacking General [Developers Only]] manuel1992:
Hi, this is a great work. Can you make a video to see the difference?

How come Google never thought of that? or Samsung or HTC, LG, Sony......?
[Android Software/Hacking General [Developers Only]] JustArchi:
Too lazy to make video :p. Maybe someone else will.
[Android Software/Hacking General [Developers Only]] r0gger:
Thanks, this answers completely my question! I will reread carefully again the commit!

Regards
[Android Software/Hacking General [Developers Only]] metalspring:
FYI, just so everyone knows, the very minimal you really need to build roms with any optimizations are-

At least dual core cpu and running at least 2 GHz (more is better)

At least 4gb ram (again, more is better)

If you do only use 4gb ram, be sure to have enough swap space- its always recommended to have slightly more swap space than you have ram (although you rarely come close to using it all). I.e. I have 3920 mb of usable ram and my swap partition is 4109 mb

and lastly a recent linux distro either on a separate partition or in a vm (anything based off ubuntu 12.04 or newer is recommended since its the easiest to set up, I myself use elementary os).

Also keep in mind most newer intel cpus are hyperthreaded and each cpu core can actually run 2 threads (vs one thread on regular cpus), this means you can allocate more cpus to a vm and not worry, and that you can also double the number of jobs when building (i.e. make -j<number of jobs>)

At the minimum specs it will take around 4 hours or more for each full rom build (longer if its the first build or without ccache)

Hope that clears things up :P
[Android Software/Hacking General [Developers Only]] JustArchi:
I agree with everything apart from one thing. Intel's hyperthreading is not that nice as you describe it. It's just another "pipe" to the same core. Running VM on more virtual CPUs than physical cores decreases significantly overall performance - VM as well as main OS. FYI I'm getting best results with setting only 4 virtual CPUs to my VM, even if I have 8 threads total in my i7 2600k. For some reason both my VM and OS work awesome in terms of performance, there is no overhead whatsoever to the main OS, I can compile my AOSP while playing mid-high game in the same time. So if somebody is looking for my advice, always set VM number of physical cores you have (even Oracle suggests it) instead of setting it to a number of every available thread on your machine. This may be connected with the fact that VM can't properly detect HT technology, or simply have no use of that and every additional job only slows down overall process.
[Android Software/Hacking General [Developers Only]] sysKin:
Hi all,

First of all, THANKS for all this, a simple -O3 on "thumb" flags made my vanilla CM 10.2.1 score ~15% faster on average in various 0xbench tests (and one test is six times faster lol; I hereby validate this thread's title ;p).

Looking at armv7-a-neon.mk, I have two questions:

1. I see it's not trying to optimise for any cpu if cpu is "krait". In the CM 10.2 tree, it uses -mcpu=cortex-a9 for "krait", which I also think is weird because krait is supposed to be like a15.

I'm currently building CM with -mcpu=cortex-a15, if it works I'll let you know.

2. I see that it's using -mfpu=neon unconditionally. However, some CPUs here (krait, a15...) have vfpv4, so -mfpu=neon-vfpv4 sounds like something that should work. What do you think?

:)
[Android Software/Hacking General [Developers Only]] xboxfanj:
Krait is more like A15. I compile with A15 flags and it is a bit smoother, but CM will not accept that in the global flags, so just set it locally in BoardConfig. Same for Neon-vfpv4.
[Android Software/Hacking General [Developers Only]] metalspring:
1- Arch variant is set in your device's boardconfig.mk file, and for krait, you are right that it should be cortex-a15. That said however cyanogenmod feels that the difference between cortex-a9 and cortex-a15 is a placebo effect (see this commit for the m8 which is krait also http://review.cyanogenmod.org/#/c/63538/). I personally disagree because even if it doesnt make a difference, they should at least have the proper setting

2- You certainly can try using neon-vfpf4 but its probably not going to make a huge difference either, but again it would be right :P
[Android Software/Hacking General [Developers Only]] sysKin:
Hi, I'm a "little bit" of a n00b in how Android build system is set up (mildly speaking). I can't see any compiler flags in my BoardConfig.mk (nor in stuff it includes), but I do see it set in armv7-a-neon.mk.

Does that mean that, for me, arch is set in armv7-a-neon.mk? Or not set at all? Or something else?

They might be right of course ;) but then again, if they build with -Os, just exactly how many optimisations are disabled in the first place lol.

Good to see krait is a15 compatible; if it wasn't, I'd try -march=cortex-a9 -mtune=cortex-a15 instead :D
[Android Software/Hacking General [Developers Only]] metalspring:
Try adding something like this-

http://vaniraosp.goo.im/#/c/11726/3/BoardConfig.mk

Also cortex-15 for krait is really only useful for the m8 from what I've heard, I think different krait optimizations in bionic rely on cortex-a9 being set
[Android Software/Hacking General [Developers Only]] takota6:
Can we have a request page for those of us like "Me" that have legacy devices with low devs , but have the need to see this sweetness in Action?

HTC DNA(dlx)

Æaaa..\../
[Android Software/Hacking General [Developers Only]] JustArchi:
You'd have more luck asking your local dev for your device to take a look at this and experiment a bit. I'm building only for i9300, and in my small community everybody "feels" the difference. While it's "easy" to apply this commit when you know how to compile AOSP, your device may not work right out of box, especially kernel. Thus, it requires some self-knowledge and some hours of free time to apply this commit in proper optimized way. However, after you put time and your knowledge into this, you'll eventually benefit from that.
[Android Software/Hacking General [Developers Only]] kantjer:
You are absolutely correct. Until recently I never compiled a ROM in my life. I then started to build SlimKat for i9505 with stock settings for personal use. Inspired by this thread I spend a week applying your optimizations and solving a few device specific issues (google is your friend here)

Like you said, I learned so much from the days applying the commits and solving issues.

I now have my own ROM thread and user are calling the V3 compiled SlimKat ROM ridiculously fast and smooth :D
[Android Software/Hacking General [Developers Only]] r0gger:
Any ideas?

In file included from drivers/usb/gadget/android.c:48:0:

drivers/usb/gadget/f_qdss.c: In function 'qdss_bind_config':

drivers/usb/gadget/f_qdss.c:586:38: warning: argument to 'sizeof' in 'strncmp' call is the same expression as the second source; did you mean to provide an explicit length? [-Wsizeof-pointer-memaccess]

error, forbidden warning: f_qdss.c:586

make[3]: *** [drivers/usb/gadget/android.o] Error 1

make[2]: *** [drivers/usb/gadget] Error 2

make[2]: *** Waiting for unfinished jobs....

I am using linaro 4.8. The idea is to use for the system build and kernel build the same compiler/toolchain.So I do not want to revert to linaro 4.7 or standard google gcc (which looks like is an option too, if I use google 4.8 ..)...

Regards
[Android Software/Hacking General [Developers Only]] metalspring:
Try adding something like this to the local cflags in the makefile closest to where its erroring-

$(call cc-disable-warning,sizeof-pointer-memaccess,)

That disables the warning for sizeof-pointer-memaccess and its the warning itself that is causing the error (some gcc versions dont support different warnings and error when they encounter them)
[Android Software/Hacking General [Developers Only]] sysKin:
Is this the general consensus for working around those problems? I had to "fix" several of those, but I never know if I should keep the bug (the sizeof(pointer) is absolutely wrong and will lead to strings being cut at three characters) but make it compile, or try to find the actual buffer size (often not available, but at least once it was), or use non-n version of string functions and risk buffer overruns...

For example, this particular code in usb attempts to find whether a usb channel name (whatever that is) is already on a list. However, because of this sizeof bug, it only checks the first four characters of channel name. Channel names are equal if first four characters are equal. I have no idea what kind of strange behaviour this produces, but I changed this particular code to use strcmp (non-n function)...
[Android Software/Hacking General [Developers Only]] metalspring:
It would depend on the warning really, in this case the warning isnt causing the error, its the gcc version not recognizing that warning that is causing the error (error, forbidden warning: f_qdss.c:586), whereas if the warning actually caused the error itself it cause something like-

error: argument to 'sizeof' in 'strncmp' call is the same expression as the second source; did you mean to provide an explicit length? [-Werror=sizeof-pointer-memaccess]

cc1: some warnings being treated as errors

Usually if warnings arent severe enough to be turned into errors like above, they are safe to ignore
[Android Software/Hacking General [Developers Only]] sysKin:
I don't really mean the warning itself, I mean the absolutely wrong code that the warning caught. Do you usually compile the buggy code, or try to fix the bug and compile something correct?

Samsung-copyrighted driver code for my device is full of those, and it made me really wonder if someone who wrote it was missing some basics of C.
[Android Software/Hacking General [Developers Only]] metalspring:
You mean fix all the warnings while compiling? Its always a good idea, but in general they are ignored just due to how many of them there usually are lol