yashsingh1/xdaforums huggingface dataset:
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
i'm unfamiliar with pixel dust, i wonder if they are patching their kernel and that is breaking things for custom kernels?

i can look and see, when i have time - you could also ask them...?

i only really test on Stock rom. this is because it tends to be more optimized, i can be sure that it's signature key signing has been done correctly, etc...

i still have to run a handul of apps to tweak it like a custom rom, but it's not like when i had my galaxy s5 - I had no choice but to roll my own rom based on Cyanogenmod. (can't stand Samsung's builds.... but CM was pretty rough too; i had to remove a lot of crap and had a delta of like 60-70 patches against cm-12/14.1. ... pain in the butt...lol)
[Google Pixel ROMs, Kernels, Recoveries, & Other De] グリッチ:
Love how active you are, I'm watching those commits and starred the repo :D

Might I suggest something like this?
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Thanks, but no - I'm not disabling debugging that is potentially useful/important.

Nick probably disables CONFIG_HAVE_DEBUG_BUGVERBOSE because he doesn't know what any of that information means anyway (and sure it might reduce overhead slightly)... But I need to be able to have some verbosity.... On top of that, locally I enable a lot of the lock debugging, I just don't push it to my github.... It's kind of similar to people who just mask over gcc-5+ compiler warnings in their builds => i do that on github, but locally; I've spent some time actually fixing warnings and incorrect code (still lots more to do)...

Nick is a bit of a noob (we've chatted) and while exNoShadez-eas may be new, in terms of being publicly available / android kernel builds / me hacking on Marlin sources... (a couple of weeks ago)

Here's some background on me;

- I maintained Archlinux Linux kernel packages (PREEMPT_RT_FULL/linux-rt with some additional patchwork)

- hacked on nvidia-rt linux drivers (tracked down the source of latency spikes on x86/x86_64)

- packaged linux-xenomai (another flavour of realtime linux / RTOS stuff)

- introduced and packaged plethera of rt tuning apps from Redhat enterprise linux for PREEMPT_TR_FULL for Archlinux.

- I've been maintaining my own linux kernel builds (for desktop x86/64 linux for a decade)....

Android-wise -> I also contributed to Cyanogenmod KLTE (s5 kernel) helped getting all of the needed security patches in place (seccomp support); which allowed it to be one of the 1st devices to be green-lighted to run Nougat.... I also pushed some patches to AOSP to have the Mac Address show in the WIFI advanced configuration, I never heard a peep back on them... but in Oreo, they seemed to have come to the same conclusion. lol (ie: it's there).

None of this makes me an expert or authority or anything like that. I just thought explaining a bit about where I'm coming from might give some perspective on my decisions and direction. (like not supporting TWRP / CONFIG_CC_OPTIMIZE_FOR_SIZE=y or not including a zillion user facing features that I don't have confidence in, etc)... There is a method to my madness. lol
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
hmmm... I may have to revert my pulls from linux-4.2 (lockless wakup stuff) OR figure out how to make it play nice with the BatteryStatsService. (or rather the kernel code that it hooks into....). With those changes; it does seem to improve battery life (like draws nothing when sleeping now), the problem is that the Battery Stats don't appear to be factoring in / accounting correctly. Example:

- I play some music for a bit in my youtube player. The stats change (based on my usage), letting me know that I will be able to do continue on this path for about 12hrs. Sounds about right....? BUT, if I put the device into deep sleep ~ It doesn't seem to be taking that into account. ie: it's not correcting itself / accounting for what my usage will be based over time, in deep sleep too.... (like switching to something like "1d 9hrs" or something like that).... In some sense, I prefer knowing how much juice/hrs I have based on my actual usage (but that's not the expected behavior).

Maybe some of you using the latest build can verify what I'm seeing???

It would seem that I've bumped into this issue before, when I pulled in some franco patches that I had to revert. They did reduce battery consumption too, but caused a similar issue.... I can explore adapting the affected drivers to workaround the issue too - but it's not obvious to me how to get them to do what I would like (may have to revert the change on my next build, even if temperary).
[Google Pixel ROMs, Kernels, Recoveries, & Other De] siheals:
Is your latest kennel the one in the op
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Updated:

- I reverted the 'Introduce: helpers around the i_mmap_mutex" patchset from linux-4.2

- the "futex: Implement lockless wakeups" is included/intact

- all previous bug fixes are included.

I will likely re-introduce the i_mmap_mutex/rwsem patchset, once I have fixed the issue. No breaking userspace beyond Dynamic Stune Boost doing away with Power Hal.... For now, i Have stashed the patchset/commits in another branch - so I don't need to redo the merge and can quickly cherry-pick for testing, when I have another stab at resolving the issue (it's fixable).

I've got some other bug fixes queued in my local branch (hasn't hit github or my zip yet), I'll probably merge in Maple scheduler soonish too.

Also, got more patches for my local LSE branch, but it's going to suck trying to merge them :(
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
always, yes - and it was just updated to fix the batterystats regression.

the only time it will be different, is if I post a tester in the thread.... but even the Nov3 build I posted in the thread yesterday, is now the download in the OP (with the batterystats stuff fixed).
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
[MENTION=5502224]siheals[/MENTION]

There seems to have a been a delay in my upload. it's fixed now and I also added a backup for todays build....

but I had went away from my machine, switched tabs in my terminal and noticed it 'stuck' and didn't finish uploading to github. So I tried again and it pushed right up. no idea why that was...?

anyway, worst that could happen is you grabbed the kernel with the batterystats bug - there's nothing from with it beyond that. (as in, I would personally use it... I kind of kind the whole batterystats and lack of good prediction to be annoying. lol)
[Google Pixel ROMs, Kernels, Recoveries, & Other De] siheals:
I use accubattery for my stats, so I will see how it goes.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Cool. Never used that App.

Why is that your particular choice?

...and I suppose I will take a look! .... I just like to make sure I'm not breaking userspace, so the BatteryStatsService needs to work as expected.... I broke that Rule with EAS + Dynamic Stune Boost (by breaking pwer.marlin.so / userspace control and awapped it for in-kernel), but in that case - It's an area of interest and personally; for me it's been working quite well.

anyhoo, yes. definitely report back any useful information or insights that you may have! thanks.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] almightysiman:
979，It is good to have an expert like u working on Pixel！
[Google Pixel ROMs, Kernels, Recoveries, & Other De] Nick80835:
Oof lol time to change things for 8.1 :eek:
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Lol. definitely NOT an expert, but thanks ;)

I've interacted with the REAL experts (upstream linux developers), I don't have that kind of knowledge or expertise. I'm fluent in using git, managing patchwork, fixing compiler warnings/bugs... and have experience with linux, a variety of compilers/toolchains, bug trackers, etc... But I'm not a great programmer or anything like that. lol

Anything that I am working on is me standing on the back of giants. I'm just trying to improve the kernel because;

1). Google hasn't even bothered to keep up with LTS (or doing anything substantial to the kernel, really).

2). there are a ton of bug fixes floating around that haven't landed in Marlin (spread across tons of msm-8996x repos).

- Most come from Code Aurora, linaro, etc... I plan to scoop up as many as possible.

3). I want to push the kernel forward, enable/integrate the newest features and enhancements

and on that subject, I've merged a number of fixes/commits today; https://github.com/nine7nine/marlin_exnoshadez/commits/EX-NoShadez-eas

- most are minor fixes to mdss and kgsl (race conditions, memory leaks, etc)

- a fix for cpufreq

- another boot time tweak (early boot frequency to reduce boot up time... even though it's speedy. lol)

- a fix for the energy aware driver (should decrease wake ups)

I have a crap ton more fixes to apply, in queue. but it's a process.... I have my eyes on some other more substantial code to merge in too - but I'm focusing on code aurora, arm and upstream fixes just now.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Haha. Hey Nick! :) I wasn't knocking ya, I just remember it happening to you! ;)

Ya, I won't touch 8.1 with a ten foot pole... It won't be released for a while yet, I'm surprised my kernel booted at all on it.

anyway, 8.0 is plenty good.... Well, once you ditch their kernel, root it / install Magisk, ditch have of google's apps, replace Pixel launcher, change a bunch of settings and download some apps to be tweak, manage and modify it (and get rid of their awful white them. bhahahaha)...

(but I would have to do the same to 8.1 - so there you go!)
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
So just as an experiment and because merging in fixes gets boring (and I've done enough for one day. lol).

I decided to play around with optimization levels in GCC.... After a bit of tinkering and merging in some changes, I successfully have built a kernel using gcc optimization level -O3 .... The only thing that broke (after ensuring that things wouldn't), was again BatteryStatsService from properly updating. lol... Makes me think that that code is flaky in general.... (there is some irony here; given that I've been fighting with this already today.... they must be related, ya think?).

so I think I will enable a bunch of warnings in GCC and see what trips up in the power and thermal related drivers...

In benchmarking (hardly real world use) my scores are bit lower using -O3 flag for optimization, than -O2 (not entirely unexpected, as more doesn't always mean better, in fact. More optimizations can often lead to code behaving not as expected ( strict-aliasing + the linux kernel being a prime example )...

But I did notice a few things that seemed MUCH faster; like google earth seemed to be quite a bit quicker when rendering. Google play seemed to be much smoother too... Anyway,

I wouldn't dare merge this into my kernel, as there are bound to be bugs; even if my next build fixes the obvious stuff.... but it may not have been a useless thing to have done, as mentioned; it did point back to the BatteryStatsService issue....

fun stuff for anyone interested
[Google Pixel ROMs, Kernels, Recoveries, & Other De] siheals:
Very good indeed as I play 3d games
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Beautiful! ... Thanks a ton for posting that :)

AccuBattery actually seems very good for gathering useful information - I installed it lastnight.

I do have some other patches that might improve your gaming experience a bit too (watching high def videos / high frame rate - noticeable improvement), ***but in the huge round of merging bug fixes, I have hit a regression - thus, this morning I am in the process of moving all of the commits into a separate branch (from which I can cherry-pick later) and then I will roll back my main branch to the commit of the kernel that has been released...

Then I will go through - one by one, cherry-picking until I find the culprit. joy. lol
[Google Pixel ROMs, Kernels, Recoveries, & Other De] siheals:
Excellent, that sounds like a lot of work.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
It is and it isn't.

Now, that I have loads of patches enqueue (and already merged in a branch - it's really just a matter of going through them... a few slipped in that I probably don't want - like a couple of downstream / cyanogenmod/LineageOS commits or one-offs like that - but the verified one's aka; Code Aurora, Upstream, Linaro, etc are always of interest).... So I'll go back through them, no immediate rush....

This morning I'm spending a bit of time going through my kernel configs and adjusting a few things, as well as testing RCU with a few different configurations... I would like it if our config was a bit closer to the Pixel 2 / OP5-eas kernel and I would like to sort out Input Boosting, so I'm trying to iron out that...

I'm also tinkering with qcom polling interval in relation to temp. I'd like if it polled a bit less (it causes unnecessary wakeups) - I have a patch from Code Aurora to test this out.

I'm pretty comfortable not merging all of the fixes in yet and working on this - because the kernel seems to be working well AND the lack of being able to enable input boost (and the method that I used to determine it should be disabled until I sort it out) means I should really focus on getting it working properly (and better match our kernel against the kernel from which I pulled the code)...

The RCU and Input boost stuff is more work than the bug fixes. ;)
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
sooo...

Just in the process of migrating Marlin Pie kernel sources/repo to gitlab (something I can do, while not home). Then when I am home, I'll get all of the rebase stuff merged/ready on android-msm-marlin-3.18-pie branches (both of my kernels), followed by grabbing some bits from v3.7 over to v2.9...

So hopefully, late-late tonight (if the last bit goes smoothly) I will have updated builds or possibly tomorrow (after work, worse case scenario), *if* I run out of time today.

in the meantime, the v2.x should be fine (while v3.x shouldn't be used ~ as there is a "showstopping" bug currently; but I do know the cause of it - so will be addressed / fixed in the next/official P build).
[Google Pixel ROMs, Kernels, Recoveries, & Other De] sl1v1k:
Will it work with new Android P?
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
It should (it did for me a couple builds back)

Also nine, I used to use gitlab and in my experience it was a lot slower than GitHub for some reason. Do you experience this? I have really good internet speed too, it seemed like their servers were slow or something.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Hey Shag,

Gitlab IS slower than Github. However, as Gitlab becomes more popular ~ I'm sure that it's infrastructure will improve. I don't actually mind the speed trade-off, being as Gitlab offers some other features that are paid-features on Github (like having private/non-public repos, for example. which is something I've wanted for a long time.).

I also think I like Gitlab's UX/layout better too. Although, it doesn't look quite as refined as Github, everything is quicker to access. So, speed aside - I'm happy to use Gitlab and ditch Github.

Then there's the Microsoft part.... so yeah, github is done for dinner. I won't delete my github or anything, but I won't be using it either....
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
HEADS UP: I won't post builds until tomorrow, so sorry for any inconvenience, but here's why; (and the state of things)

#1 - I've rebased all of my patchwork for v2.9, from scratch (tonight) on the vanilla Marlin/P sources (1000's of commits to go through)

#2 - I had to make a few small changes, as there were some differences in a few places (not unexpected, but I do want to test)

#3 - I've pulled in all of the changes to /drivers/staging/android/* drivers (binder, ion, etc) from v3.x => 2.9 + made a few small changes, reverting a couple of specific changes not applicable to v2.9 kernel. (differences in MM subsystem in v3.x).

#4 - I've added back && squashed in all LTS (3.18.100 => 3.18.117).

#5 - compiler fixes, bug fixes, etc.

While v2.9 is running on my device, I do need to give it a good run, due to the above (although, it does seems to be working great, thus far!) and also finish up the last few bits.... I've been at it since just after 5pm and its now almost 11pm... Sooooo! ~ i'm taking a bit of a break, then I'll came back and try to smash all of v3.7 off, before I go to bed tonight ~ it will be a lot less work, but still enough that it'll take some time (and I want it running on my device over night, during day).

* then publish builds and kernel sources when I get home tomorrow.

I could've probably just grabbed/merged in just the P changes, However, I was long overdue to just redo everything. I did do this like a month ago (and I actually used that as a reference), but I still did everything from scratch. ~ on the bright side, it should be dead simple to maintain now; 1000s of commits boiled down into 16-18 commits (upto 2v.9), not sure what v2.9=>3.7 will add upto - but probably a similar amount, a bit less.

anyway, no worries, there will be new builds tomorrow.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
So, I just flashed the rebased v3.7 to my device ~ finished up the rebase this morning and on my lunch.

Assuming, it holds up (it should now that the show-stopper bug is fixed), I'll be able to post builds an hour or so after I get home.

The only todo's left are to clean up a few commits, pull a commit from v3.7 back to v2.9 that fixes a bug, rebuild both kernels and make my new kernel sources publicly available....

Frickity-frock; v3.7 turned out to be as much or more work than v2.9 ~ I think I forgot just how much of the kernel has changed. LMFAO.

and on that note (read next post),
[Google Pixel ROMs, Kernels, Recoveries, & Other De] duniecool:
Exciting stuff! Thanks for your work, looking forward to trying it out. Which kernel do you recommend between the v2.x and v3.x? Might be a stupid question, but I am not familiar with a lot of the features you mentioned in your most recent post.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
It's not a stupid question at all (asking questions rarely is!)... Well, I use v3.7 and it's where most of my energy goes into ~ the v2.x kernel series is basically in maintenance mode, beyond a few changes that needed to be made for Android Pie (and also to reduce my workload, by reducing a few differences between kernels)... I've maintained 2 kernels, for development purposes / as to not overly disrupt end-user experiences... There were also a few people that had issues with v3.x, but I think that stuff should be resolved now && I know in at least one case; Magisk was likely the cause of bootloop issues. (which i didn't know at the time, but hit myself recently).

* Ideally, I would like to drop v2.x kernel at some point ~ it's possible that now that everyone is likely starting off "fresh" on P, I may be able to drop it by 9.1 ... Therefore;

I would recommend trying v3.7, but if you run into problems - v2.9 is there to use....

It's unsurprising that you wouldn't be familiar with a lot of the "features", it's all low-level kernel stuff, for the most part. Meaning, stuff used/consumed internally by the kernel, as opposed to "end-user features" that are exposed to userspace; like say KCAL/color control or wireguard VPN, etc .... I'm referring to the former type of "kernel features", not the latter "end-user features", sorry for any miscommunication there...

a lot of v3.x is about modernizing the kernel code base, improving performance (not so much "raw throughput", more about improving responsiveness and better task placement, prioritization, etc.). Continuing on with EAS integration and also making wider use of new APIs/functionality that I've introduced or came with upstream linux backports.... But in modernizing the code base, there's lots of improvements (that tend to add up) and I can more easily integrate features that were never written for or backported to 3.18. (it's a form of future proofing / reducing headaches).

As far as userspace features; a few will be coming, probably in a month, maybe less... and I will have details on that stuff, when it's ready. ;)
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
ok, so I've hit a bit of a snag, unfortunately. ~ and I am _way_too_tired_ (only a couple hours of sleep!) and I'm over-worked ~ I need sleep before I can even try to debug and resolve the issue...

You all will just have to use v2.8.2 until I get some sleep and can resolve the issue... maybe late tonight (if I wake up) I'll have it sorted, but most likely not until tomorrow, idk.

off to bed
[Google Pixel ROMs, Kernels, Recoveries, & Other De] BarackMitzvah:
Should we only hold off on flashing v2.9? Or are both branches having issues? For now I'm going to assume both.

Thanks again for your work buddy!
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
just use v2.8.2 ~ I haven't made any new builds available because; yes, I'm having an issue and it

is not easy to resolve... you're going to have to be patient, as I'm having some difficulties...

It's also very problematic, because I have *maybe* 48hrs to work on this / get it resolved, otherwise I likely won't be able to work on it for a while (big project at work starting, so i won't have time ~ and I can't prioritize my kernel over work, obviously)....

what makes this even worse is that; In general, I'm not happy with Android 9.0 Pie (at all), which is very de-motivating... I've found numerous bugs (android userspace bugs; rendering issues, slow-downs, where 8.1 worked fine... Hell, after installing P in the first place Chrome wasn't working properly, it is now ~ but I have no clue what even happened and that was pure "stock" / up-to-date, no other changes...), I've also had a random reboot on stock kernel ~ which isn't comforting...

Regardless, I'm going to have another stab at it, when I get home tonight. Not sure what the outcome will be, yet. we'll see.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] BarackMitzvah:
Yikes, I was going to hold off for a little bit regardless since I rely on Xposed atm, but now maybe ill just wait until 9.1.

And as always, no rush. Appreciate the effort you're putting in and sharing with us.
[Google Pixel ROMs, Kernels, Recoveries, & Other De] :
Yeah, new major version releases of android usually have an issue or three. (just in general, although, my kernel complicates things for sure).

That said, I am confident that i'll get my own kernel related stuff sorted out ~ my issue is having the time to do it (right now), mostly. I do know where/what the problematic changes in P's kernel are that are causing me grief ~ but it'll take time to figure out how to deal with it, as I have to read through a lot of code and some related docs, etc ~ I laos need to review the related changesets in Pixel 2/Wahoo's P kernel ~ as i suspect I need a few commits from there...

not the first time I've bumped into this kind of thing, though. I'll get it resolved.