yashsingh1/xdaforums huggingface dataset:
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Since I've been waiting to hear if there were any problems with the latest test build (again, no news is good news!), I've been performing some other optimisation tests.

I've seen a few devs applying the a57 flag in the makefile on the basis that this is the closest match to the kryo architecture. I thought I've give it a try too, so I built a personal test version and ran the same benchmarks as before (see the google sheet linked in the quote above).

The results show a marginal decrease in performance. I wondered if it could be down to luck (with some of the tests having a fairly high variability) so I discounted all of the results where the coefficient of variation was above 2%, but got the same result, a marginal decrease in performance.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] habylab:
Interesting. The last test was brilliant by the way. I thought I'd give Nougat a go, so can't help now.

Have you a view on overclocking? It completes tasks quicker and on our processor doesn't cost anymore battery drain, so in affect makes it more battery efficient.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
I know regular followers of my thread appreciate facts and testing. I doubt anyone comes here for misleading hype that is often found in other threads all over XDA. It was a careful and deliberate choice give the kernel an old fashioned name and an image of a man who appears to seem pretty serious about his career choice as an ice-cream parlour barista. It's the complete opposite of 'super-conducting laser-guided carbon-fibre optimisations.'

I'd like to invite [MENTION=3637908]kairi_zeroblade[/MENTION] to the thread to lay out the facts about a57/kryo compiler optimisations, and OC.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
As far as i know A57 is the closest to the Kryos in S820, I remember i read somewhere that the Kryos were called as A57 in some scripts last time I was on OOS.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] kairi_zeroblade:
Qualcomm's latest SOC is not a fully Cortex based Architecture..so any flags used on Cortex CPU's (such as A57/A53) won't do good..based on my observations adding those flags could hit IPC performance huge, moreover, Kryo is not based 100% on A57, its more of a hybrid A57 and A72 (but not completely), it just borrows instruction sets from those architectures mentioned but Qualcomm treated them differently..

A few more observations I had was the GCC compiler that Kryo must use. Based on the linux mainline's recommended GCC compiler for Linux 3.18.y tree, it is best advised to use at least a GCC5 compiler, but to my surprise it was a miss on every figure I see when its compiled, there were setbacks/regressions when using the latest and the mightiest GCC. what was more dumbfounding was when I tried to build it with Google's 4.9.x Toolchain and it was hitting the high notes expected from it, It may have been really best to stick with Google's arrangements for this, and I concurred that the changes brought about by Android on top of Linux 3.18.y may have caused some irregularities.

Also, when I was trying to use my recipe on the Kryo, it was really hitting a miss (some GCC optimization flags tested one at a time) and enabling O2 levels of compiler optimization can get more performance regressions than you benefit from it..

Currently I'm using mild GCC flags on my project which I tested personally and strictly follows Google's GCC toolchain build..so far, its still early since Kryo was just a few month's old and I'm trying all possible stuff we can feed the kernel at a GCC compiler level..

About Overclocking on Kryo, it is possible just as long as you compute the PLL Frequency, furthermore TZ allows an extra 36.4mhz allowance on clock rates. (I have observed this on my SD410 phone and was able to push it beyond 1.8ghz with a performance jump off course) PLL rate for SD820 is 76.8mhz, that is the allowance(could be a tad more as I explained above) between clock frequencies so if you follow the rule book and stick to the computation TZ will just think its a valid frequency that the chip can run..AFAIK I do remember when Kryo was being previewed it was said to run at a max of 3Ghz, but off course for thermal and power drawback reasons we don't see it run that high..as we technically don't need that high as Qualcomm really did well on the chip architecture..

If you have any further questions you can hit me up and I'll gladly reply and explain as far as I can..

And also a huge thanks to [MENTION=6295391]bedalus[/MENTION], he has been one of those who triggered the spirit in me during the Nexus S days, I can say he was a "Pioneer" of what Ingenuity and Innovation really mean.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
I hope I can have the opportunity to test this kernel real soon, but you know sometimes life interferes and tie your flashaholic hands for a while :o

Keep up the good work :highfive:
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Seems like a good time to release v9! Download here

Changes:

- Merge linux 3.18.46

- Brought in a large number of kgsl updates

- Update toolchain to latest linaro (6.2.1)

- Merged in latest wifi module code (LA.HB.1.3.2)

In light of the information provided by [MENTION=3637908]kairi_zeroblade[/MENTION], the toolchain itself could introduce performance problems, so again, more testing is required. I'll likely run a similar benchmark to the others using the kernel compiled to the same commit, just with a variety of toolchains.

We're still waiting for the kernel source for N to drop, so no news there! This version is still Marshmallow.

Thanks to kairi for his insights, and again, much appreciation to those who have helped test this release :)
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
No kernel drop, so still on Marshmallow!

Using the data in my previous tests: google sheets

...I've devised a way of testing differences (if any) that can be provided by toochain choice. The data shows which tests have a low coefficient of variance, but a relatively high difference in the average results. These tests are the ones to focus on. I found four available in Geekbench alone, which runs so quickly that it'll be easy for me to run a lot more passes. This will give better averages.

If there are gains to be had from using alternate toolchains, I now have a way to demonstrate it.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
[MENTION=6295391]bedalus[/MENTION] hello :)
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
[MENTION=5060769]Mostafa Wael[/MENTION]

I've never heard of this app before, so thanks for introducing it to me. If it's measuring app loading times this will largely be affected by the biggest bottleneck of all systems, the non-volatile storage. Things here can be seen to improve with choice of IO scheduler. There might be scope for running some refreshed IO tests with different schedulers, but that's not something I'm interested in delving into again right now. CFQ has proven itself in every way I've tried testing it in the past, and it's set as default on most systems for a good reason.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
You're welcome buddy, and yeah it is DiscoMark; an app used to measure app opening times via an accessibility service. Yes CFQ is quite good actually, I have seen a lot of I/O sched tests and CFQ managed ace them all, specially when there is a background write process done while launching apps. Still, it is quite impressive here tho.

Anyways, speaking of I/O schedulers, I found a weird I/O scheduler option in your kernel besides CFQ and the others, it is listed as "test-iosched", last time i saw that was in Sultanxda's CM13 and touching that would be simply asking for a ride to hell (phone instantly went in a random reboot and a freeze), so any idea what is that ?

looking forward! :)
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Oops, that's supposed to be disabled. Thanks again!
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
Now I have been using v9 for a while and it seems quite good, still not assessing it finally but it is good :good:

however, I see you mention that I may suffer "performance problems", I am not sure what kind of problems should I be expecting ?

Also I would like to note that benchmarks are just benchmarks, Geekbench for instance has been way too much inconsistent for me, and sometimes the scores are irrelevant to what I tweak for. DiscoMark was the only one that I would call reliable (If we have the I/O sched settings constant)

Anyways, till kairi releases his next build, I will be crushing in here and see what's what :)

Man, Marshmallow will still be missed even after I went to Nougat :cyclops:
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
The toolchain takes the code which is pretty much human readable and makes it machine readable, and there's various ways it can go about doing this, there's a good overview on stack overflow.

The toolchains are constantly evolving, getting better at spotting errors in the code and providing more useful debug info, and also improving the generated code, taking advantage of the new features of the new architectures as they are released.

From https://releases.linaro.org/components/toolchain/binaries/latest-5

When ARM released their 8.2 architecture, they provided the schematics for chip manufactures to build the hardware, and software engineers to leverage the hardware in order to perform certain types of processing on the chip that weren't available before. To be able to utilise these features, the required code must be introduced into the kernel, which happens in the newer kernel releases (and if essential for users of older kernel releases, someone might go to the effort of backporting the code too, which can pose a very large challenge). For the new code to work on the new chips, the compiler must be capable of generating the new instructions.

This is where there is potential for performance regressions. In order to support the newer instructions, the toolchain developers build the new support into the toolchain. Now a code optimisation that previously worked to reduce some code to its smallest possible form generates instead a slightly longer piece of code and results in a cache miss. Or perhaps ARM may have provided some assembly instructions to support the new hardware, but the rush to market meant older, better tested code has been replaced with something that works but lacks refinement (here is a good example of a problem introduced by ARM that required an inconvenient workaround and introduced a performance hit). The reasons why the new compiler features have affected code generation are complex, and the regression may not be easy to fix, or impossible to fix without disabling support for the new hardware.

In these cases there is always the option to use the newer toolchains to help identify code issues, then rebuild with an older toolchain without compromises introduced to support the newer hardware.

If you can handle it, there's an article I found which goes into great detail here but it may be slightly biased because the author is advocating a proprietary method for testing for these kinds of regressions. However, it does lay out the issues in greater depth.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Mostafa Wael:
[MENTION=6295391]bedalus[/MENTION] so it is down to refinement and well-tested for a toolchain to be quite good right?
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
It's extremely complex, and the best method I've found is to attempt an optimisation then test it to ensure that it actually helped with thorough benchmarks. Of course coders are always looking to improve the performance of the code, and at the kernel level it's fun to bring in these optimisations and test them out. At the toolchain level, I just obtain whatever binary is the latest. However, as [MENTION=3637908]kairi_zeroblade[/MENTION] was saying a few days ago (here) there might be regressions introduced in these newer toolchains that are hurting performance. This is something I'd never considered. This is why I'm planning to rebuild the kernel using some earlier toolchains to see if there is a measurable difference.

Building multiple variants of the kernel and benchmarking them using tests that have been proven to produce reliable statistics may not sound exciting, but it's necessary work to ensure that claims of improved performance are actually valid.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] saayuj98:
I'm still on 3.2.8(will switch to OOS4 when kernel sources for it are released) with your Moxie Kernel. It is running super fine. No issues whatsoever. DOPE stuff indeed.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] habylab:
Source is out, in case you did not know:

https://github.com/OnePlusOSS/android_vendor_qcom_thulium/commits/oneplus/QC8996_N
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] mixtapes08:
Yeah z3 compact was a solid device. Solid battery afaik, my friend got one then decided to switch to nexus 5x that had mediocre battery. Lol. Anyway op3 is the best choice today. I think it's the fastest phone up to date. I'm very satisfied with it. :)

Sent from my ONEPLUS A3000
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Okay, as promised, I'll go into some detail about what I've achieved in this update. First I want to talk about this file: drivers/gpu/msm/adreno_cp_parser.c

See Github commit: 9464eba

I just want to go over the methodology first. Since the kernel selects CC_OPTIMIZE_FOR_SIZE in the def_config, warnings about potentially uninitialised variables are disabled in the Makefile:

ifdef CONFIG_CC_OPTIMIZE_FOR_SIZE
KBUILD_CFLAGS	+= -Os $(call cc-disable-warning,maybe-uninitialized,)

This has been standard for a while I think, since it produces a lot of false positives. I think I had to sift through around 70 files, picking up a few hundred uninitialised variables. However, it seemed to me that there were a few functions that appeared to have genuine issues. The worst offender appears to be this file adreno_cp_parser.c

for (i = 0; i < ib_obj_list->num_objs; i++)

There are four instances of this loop in this file, each following a similar pattern of looping through items in an array, however, the original code appears to be lacking a scope (in C a scope is a section of code that is contained within curly braces {}) for this one loop. This has the effect of failing to perform any of the checks in the following if statement:

if ((SNAPSHOT_GPU_OBJECT_IB == ib_obj->snapshot_obj_type) &&
(gpuaddr >= ib_obj->gpuaddr) &&
(gpuaddr + dwords * sizeof(unsigned int) <=
ib_obj->gpuaddr + ib_obj->size))
return 0;

...for all objects excepting for the final one in the array.

The rest of the code changes (the other files in the same commit) were more trivial, just a bit of code shuffling really. These minor changes merely make it absolutely certain that the variables must have been initialised before being used. However, it's perfectly reasonable to assume that they could never execute without being uninitialised because of the logical flow of the functions, but there's nothing wrong with making certain.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] kentexcitebot:
Wow, I'm actually learning a lot by reading this thread! I think developers on XDA should write these detailed posts, definitely interesting [MENTION=6295391]bedalus[/MENTION] :)
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] DontPushButtons:
Well now, I was putzing around here checking out development as motivation to buy an OP3, and I find [MENTION=6295391]bedalus[/MENTION] has come out of hiding again. How you been man, haven't seen you since moob on mako.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Well, as you can see I've re-joined XDA... I still enjoy kernel development, and I like to share my work (and get help testing!)

I think since mako I've radially altered career path, from maths teacher to reporting analyst. I'm making fair money, but I enjoy the work so it feels like I'm robbing them! The thing about teaching maths was (sigh) you never actually get to do any! But my current job has problem solving at its core. And I'm basically burning the candle at both ends because after work I'm coming home and slamming straight into git and C to build this project.

I just need to go off on a tangent about numbers, since I've been listening to Feist on a heavy rotation, and I just noticed my thanks meter has hit 303: Feist, flight #303 (youtube)

...just a weird coincidence!

What about you? Seen any amazing development? Tempted by the OP3?
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] AlkaliV2:
I've been running your test kernel for about 13 hours uptime and there have been no random reboots or instability. I checked the kernel logs for errors and other than some Socks callbacks (probably due to the school wifi I'm using) the only error I was seeing at kernel level was the synaptics call-out in the screenshot below. Don't know if it has any effect, but overall the kernel is running very well. I haven't been able to play with it much because of classes but this weekend I'll try using it the normal amount and see if I have any issues. Thanks for sharing the test build, it is doing very well so far! :good:
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
What app is that please? I like how the colour scheme highlights the log info. Looking at the messages, they are just ordinary debug information rather than errors. It's normal for the kernel to log benign messages. Often kernel hackers disable debug logging as they see it as an unacceptable overhead (even though it is really tiny!). If it looks like these messages are appearing too much. I'll remove the code. Here's the function in question. You can see the TPD_DEBUG lines are just logging suspends/resumes/screen blanks. Nothing to worry about!

I'm very grateful to have another tester actually! Thanks for posting your insights.

If you expect a certain amount of battery consumption, would you be able to recognise if it was less or more than usual?
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] nevermorenorless:
It looks to be 3c toolbox
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] AlkaliV2:
He's correct, it is 3C Toolbox (I have pro) and it does a really good job with logs and general tweaks for our OS. I used it exclusively for the longest time.

I'm glad to do testing, I'm learning Java programming at the moment in my classes and you posting your code snippets and changes is awesome because I get to see the real-world application of the logic. I am able to tell if there are battery changes but more in idle lossage based on how I treat my device. If I see any drastic changes I ensure it isn't my apps causing the issue because earlier I was experiencing lag but it was LastPass causing the issue.

Anyway, I'll keep testing and let you know if anything changes. So far it's been up for the entire day so stability is not an issue. Again, thank you for posting your knowledge and development steps. I get a lot out of it.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] XeloOw:
Yeah this is very cool that the kernel dev shows to us what he changed in the code .

I really appreciate that !
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
I've set up an online sheet that shows the stats from the two big patch sets in more detail

here

I've had a look through all the commits to see what the most interesting changes are. The majority are simple patches to optimise code or provide extra safety checks. These four patches are the most interesting because they are a bit more radical and pertinent to our device due to its limited battery capacity:

8b9a0bb

6c2f52a

aa5c87c

63fe3af

These patches improve the accuracy of the CPU load calculations, which is a crucial factor in determining the need to scale the CPU frequency up or down. CPU frequency is one of the biggest factors in power consumption.

Back in the Nexus S days I did a lot of work measuring battery drain and if you're interested in what is actually effective in saving power you can still get a lot of useful info from reading the first four posts in the old thread here. In short, the biggest drain after the screen is the CPU (and if you didn't know, power can be saved in OLED screens by using black and red themes, the more black the better).

Incidentally, under-volting the CPU essentially does nothing, except make it likely that your device will crash and reboot, ironically causing an enormous drain on the battery. But the CPU can save a significant amount of power by keeping the frequency low. This doesn't mean you deliberately hamstring your device, but there are other factors at work, like the bus speed, and what the GPU can handle.

One of the most effective mods I developed on the Nexus 4 was to allow higher bus speeds at lower CPU frequencies. This meant the GPU could work at its full capacity while the CPU was using less power. This will be the next stage of development if the current testing version proves itself to be stable.

Edit: For my own records, https://source.codeaurora.org/quic/la/platform/frameworks/native/refs/tags?h=LA.AF.1.1.1 recently pushed LA.HB.1.3.2-28400-8x96.0 (wifi)
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Stroky:
Wow this sounds pretty awesome! Looking forward for you results!

Really interesting.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] mady51:
Damn getting to know many things from these posts which is awesome. Loving it bro. Keep going and rock on
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] Antsu:
Test kernel flashed on stock 3.2.7... It works :). And to be precise, I've encountered zero problems yet. Thanks. I'll be using this until your next test release and report if something unusual happens. I'm not a pro tester or even novice, but I'll try my best to produce some logs, if there's found any bugs.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] bedalus:
Another one to test please: http://d-h.st/h25y
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] AlkaliV2:
Just got it installed, wiped cache/dalvik, and I'm giving it a full charge before putting it through its paces. I'll let you know if the logs tell of any issues after I've had some time to play with it. Never had any issues with the last one over the past two days.

I read your post about allowing the GPU to work at full speed by raising the bus speeds while keeping the CPU clock low, but I wanted to ask how much of the rendering is handled by the GPU. When battery saving mode kicks in, it turns off haptic feedback, animations/transitions, and brightness is lowered (I assume CPU frequencies are governed as well). I suppose my question is: Does the GPU and CPU handle the UI draws and smoothness of the interface or is it biased one way or another? I know touch boost exists which boosts the CPU quite a bit, but I assumed that was for passing commands to the OS not necessarily for scrolling purposes.
[OnePlus 3 ROMs, Kernels, Recoveries, & Other Devel] AlkaliV2:
There's been a lot more feedback in this build from the kernel logs. I filtered it to show only warnings and errors so that is what is attached.

Warning and errors aside the performance of this kernel for me was insanely good. With my transitions and animations set to .5x my phone is extremely smooth on this test kernel release. My first reboot it was like a brand new install it was so fast. Idle loss was VERY low (less than 2%/hr) but screen on loss was higher on this release. I suspect this has to do with Allo because it was at the top of my battery stats using a little more than 3x the mAh of the screen.

Overall performance is astounding with me applying zero tweaks to the kernel. Hopefully the logs only lead to improvements.