yashsingh1/xdaforums huggingface dataset:
[Nexus 5X Android Development] jugoslavpetrovic:
Thanks.

But, by asking those in public thread I wanted to learn something and so others could learn as well.
[Nexus 5X Android Development] zurik:
I flashed, seems to be that fingerprint scanner it a bit slow maybe the vibration is at 49%. In stock it's reduced and it feels a bit faster. So when I flashed the test build, after a minute, it rebooted and then bootlooped and I reverted back to stock kernel and now phone works fine.
[Nexus 5X Android Development] mr.ace:
No problem for me, flash stock kernel before test build
[Nexus 5X Android Development] zurik:
I used stock kernel before flash the test build. I just scarf a lot, I receive my phone 2 days ago from service and then I see that freeze and reboot (when phone booted up)
[Nexus 5X Android Development] jugoslavpetrovic:
This build for me is more responsive then 2.0 and 2.1.

I lower write entropy to 128 and even disabled KMS but even with on it is better in performance. Still cant tell anything about life of the battery yet.
[Nexus 5X Android Development] mr.ace:
Thanks for feedbacks it's what I need for this project.

Don't touch entropy, higher write entropy is necessary due the backported random driver from 4.1.39
[Nexus 5X Android Development] buffer overflow:
128 and 256 is okay.
[Nexus 5X Android Development] mr.ace:
As I said there's some different between this and another kernel in random driver keep it at 896 128(default value) is better, I'm sure this can't increase battery drain.
[Nexus 5X Android Development] jugoslavpetrovic:
I remember this same value was in flash kernel. Also there was swapiness at 15.
[Nexus 5X Android Development] mr.ace:
15 swapiness is very low for our device while we don't have enough memory
[Nexus 5X Android Development] jugoslavpetrovic:
Never enough, lol
[Nexus 5X Android Development] Nicktheprofessor:
I don't get this reply... What do you think reducing swapiness actually does?

On a real time system with low memory you'd want to keep it low not to swap out apps needlessly and on a high memory system server you'd like it higher to prepare for future large tasks.

15 is plenty on a phone, 100 would be insane since you'd have the memory free and yet still fill up the ZRAM and then spend CPU power bringing it back.

On a production device with fairly fast memory 10-15 is quite enough and you won't run into any slowdowns since it WILL be swapped when needed but not before (this operation is very swift and since it will tax everything for a thousandth of a second once in a blue moon it will be handled fast and reduce battery consumption compared to constant compression and decompression).

Also, KSM isn't a bad idea at all and doesn't care about ZRAM but the question is whether you even need ZRAM and KSM.

I don't have the time to do this right now but a comparison on this device on using either of them should probably be done. I have a feeling that just using KSM may be a better idea.

[EDIT] Or perhaps using the frontswap/zswap solution to just catch paged out memory, I'm updating the kernel to 3.18 and there are a lot of interesting stuff that no one bothered with because "we don't use that". Well, maybe we should? Implementation of the latest lz4 and a frontswap/zswap that has an intelligent motor to determine relevance is probably going to be far better than the age old ZSWAP that we all used in the noughties. [/EDIT]
[Nexus 5X Android Development] bbs3223474:
Everything works just fine, but it seems that battery charging speed is very slow?

I had no issue when using other kernels except this one. Battery percentage can hardly increases, but I'm very sure I have nothing running on the background and device itself isn't hot at all.
[Nexus 5X Android Development] jugoslavpetrovic:
I think I noticed that too but wasn't sure if it was on my side for some reason, I didn't think of the kernel might be the one to blame.
[Nexus 5X Android Development] mr.ace:
[MENTION=7732448]Nicktheprofessor[/MENTION] I didn't see any real improvement for zawp it's completely useless if you have any idea let me know

For swapiness this shouldn't hurt anything while I have latest lz4 and zram from 3.18
[Nexus 5X Android Development] bbs3223474:
Maybe it's about the charging current when screen on. I also noticed that charging speed can be fast when I turn off the screen. If I turn it on, it will back on lower speed.

Meanwhile, sound driver may have a issue too. Everytime I touch the screen with earphone pluged in, a pop sound will come after it, not so loud but still annoying.
[Nexus 5X Android Development] jugoslavpetrovic:
Yea, I know that but in this case I have to apologize, it's not from kernel. My data cable is old and it started to slow down about the time when I was trying this kernel, lol.
[Nexus 5X Android Development] bbs3223474:
Well, I guess I will switch to jolla kernel again to further test about it. I assume everything was fine with that too because my cable and charger is brand new.
[Nexus 5X Android Development] mr.ace:
I'm aware about pop sound I'll fix it in the next version.

For slow changing I don't have such issue, check out in the another kernel and let me know if it's related to my kernel
[Nexus 5X Android Development] buffer overflow:
Slow charging is not "available" for me, just a small problem with sound what is very low level but after reboot is ok.
[Nexus 5X Android Development] mr.ace:
I can't understand what you talk about, very low sound level?
[Nexus 5X Android Development] buffer overflow:
Yes, exactly ! First time after flash kernel sound was very low even if volume level is max. Like I said, after reboot is ok. This behavior was only after flash kernel. Is not end off the world. ?
[Nexus 5X Android Development] Lughnasadh:
Just wanted to report that I have no popping sound issue with headphones and touching screen on latest test version (09-02) running latest nightly (09-06) AICP 7.1.2 with A.R.I.S.E. (Magnum Opus 08-11). Music sounds real good :good:

No low level sound issues either.
[Nexus 5X Android Development] Nicktheprofessor:
You have to use the vnswap/frontswap/zswap combo and disable zram to use it. Vnswap acts like a fake disk swap that lives in memory and zswap is the compression engine.

The problem is that you'll have to choose between two versions and you'll want the newer one. Unfortunately this means you'll have to upgrade block to 3.18 and with that you'll have to complete the upgrade of the kernel to 3.18 which is about 100 000 commits in total.

I'm working on that but it's time consuming since googles 3.10 for bullhead is a mishmash of code so it's not a smooth upgrade.

The latest lz4 and zram tends to be a bit slower IME but probably more stable so good on you for adding that to your kernel. I also took a look at your github and it looks good.
[Nexus 5X Android Development] Nicktheprofessor:
I made a general setup of this that you can modify, I'll attach it to this post.

Note that vnswap works but is out of date, the problem is that the entire block structure needs to change to get it up to speed since it's a block device.

It's good for testing I guess and it brings the other stuff up to 3.18 so you won't have to do that to make it compatible with the new LZ4.

Also, it doesn't break anything so adding it doesn't actually do anything unless you config for it.
[Nexus 5X Android Development] mr.ace:
There's no way to use zswap+zram? Because i think vnswap is very old and not useful for our device

Check out 8.0.0-zswap branch on my source but it's better that make a GitHub account for yourself if you like to backport our device to 3.18 otherwise I think it's useless while we can backport some important parts of 3.18 and even 4.4 ( shadow already have patches from both:cool: )
[Nexus 5X Android Development] mr.ace:
Shadow Kernel 2.2

New build is Up!

Changelog :

Upstream updates from CAF and Linux

Upstream f2fs updates from 3.18

RCU updates

cpu-boost improvement

Faster Fingerprint recognition!

Reduce default vm_swappiness to 30

Revert "PDesireAudio 11.0" (caused audible click and pop while headphones connected)

Force enable high performance audio when frequently isn't 24bit/192Khz

Camera drivers fixes from Motorola and CAF

Video drivers fixes

Downloads :

https://www.androidfilehost.com/?fid=673368273298986403
[Nexus 5X Android Development] zurik:
I'll give it a try now. Let's see! I will report a couple of hours later.
[Nexus 5X Android Development] zurik:
I got first freeze and needed to force my phone to reboot. Dmesg attached
[Nexus 5X Android Development] Nicktheprofessor:
Vnswap is a feature of kernels as new as 4.2 but the version used in 3.10 is as old as most crap we have in 3.10.

3.18 is already 5 years old too and most of the "new" code you are upgrading to was in the main trials in 2010 and included in 2013-2014.

I have a bitbucket account but it's not used for this kernel yet, I merely provided what I added and compiled which I'm using right now and even was so kind that I adapted it to your changes in your own kernel so it would plug right in.

Upgrading the entire thing means that instead of hacking crap together that 9 times out of 10 will not work properly we will have a stable branch that we can then add MSM code from Marlin like EAS to without having to worry about our kernel not supporting some of it's functions.

4.2 is a bad version, it's like 3.1 and I wouldn't pick anything from that branch.

As an aside, why "CONFIG_IOMMU_FORCE_4K_MAPPINGS=y"?

Just getting rid of that will do more for battery life than enabling PEW's. It's not needed, it's a debug option for non-production kernels that is useless unless you are running an SSBI link to your device and you're not doing that.

It literally turns off all mapping optimizations and the end result is a very slow process that uses software rather than hardware to perform a task that should take one third of the time.

Sure, you could use Zswap and Zram but ... do you know what they do at all? Zswap isn't a backend to Zram and cannot be one since Zram is it's it's own backend. Zswap compresses cached pages in tmem, Zram acts as a swap device.

Also, you know how you said that a full upgrade to 3.18 would be useless, well vnswap from 3.18 is dated 2017, too old for you? ;)

Unfortunately the kernel needs to be upgraded to use it.
[Nexus 5X Android Development] :
Do you plan a full upgrade to 3.18 with the help of nick ?
[Nexus 5X Android Development] mr.ace:
I think it's already fixed

TEST BUILD : https://www.androidfilehost.com/?fid=745425885120763497
[Nexus 5X Android Development] mr.ace:
Nope, I don't take care.
[Nexus 5X Android Development] zurik:
I will try it ASAP .
[Nexus 5X Android Development] :
Tried, it work state notifier won't be activated.
[Nexus 5X Android Development] mr.ace:
Check out here

/sys/module/state_notifier/parameters/enabled

It's enabled by default, EXKM just show it incorrectly
[Nexus 5X Android Development] mr.ace:
[MENTION=7732448]Nicktheprofessor[/MENTION] I'm not sure for remove that config, I found in bullhead defconfig history that it's a debugging config and another from angler that tell me it's better for reduce memory fragmentation.

I'm just confused

https://github.com/nathanchance/angler/commit/695cf616c989dd5b0f56114e10174b2309c82f2e

https://github.com/mortezadadgar/shadow/commit/c6644fd6eb192e1cd0f447b24944eefe9648ebae
[Nexus 5X Android Development] zurik:
I think now it's fine! I will report and tomorrow to be sure.
[Nexus 5X Android Development] :
Yeah so it works thanks.

But I don't know if it's due to your kernel but my phone won't open after deep sleep.
[Nexus 5X Android Development] mr.ace:
There's no such issue for me at all maybe a rom-related issue.
[Nexus 5X Android Development] :
I'm on stock but I think that is due to modifying the conservative governor for more battery.
[Nexus 5X Android Development] :
I disabled all tweaks (Lspeed, ex kernel manager) and my phone is rebooting after 30minutes.

Can you help me please.

I'm on Oreo stock (september release) with the last version of your kernel.

I use magisk.

Ps: why flashing your kernel removes root ?

---------- Post added at 09:29 PM ---------- Previous post was at 08:36 PM ----------

I reinstalled the kernel and now it work so it's a compatibility problem with exkm or Lspeed.

I will do tests and tell you.
[Nexus 5X Android Development] mr.ace:
[MENTION=6013428]AFK269[/MENTION] [MENTION=6657029]francos21[/MENTION] still unstable? if it's give me dmesg log
[Nexus 5X Android Development] zurik:
For now, it runs without any issues. At the first time I saw unexpected rise of temperature while charging with enabled WiFi and then realized that is thermal throttling mod that I have installed and I installed stock configurations. Now no overheats and no issues. Works like charm!
[Nexus 5X Android Development] :
I had too much problems so I return on stock actually I'm waiting next update.
[Nexus 5X Android Development] zurik:
Send dmesg logs to developer to fix the issues you have.
[Nexus 5X Android Development] :
Will do it this week-end but I'm actually at the university.
[Nexus 5X Android Development] mr.ace:
Shadow-2.3

I should release it a long time ago due to blueborne but I should make sure that everything is stable, but now it's ready Enjoy!;)

Changelog

upstream update from Linux

blueborne security patch (still need rom-level security update)

Updated toolchain thanks to [MENTION=6842057]nathanchance[/MENTION] to keep it up-to-date

Updated anykernel

Turn off big cores when the screen is off

Make stock input boosting optional (disabled by default)

Updated Binder from 3.18 big thanks to [MENTION=3292224]franciscofranco[/MENTION]

Fixed abnormal heating and performance regression in the last version

Download :

https://www.androidfilehost.com/?fid=745425885120765203
[Nexus 5X Android Development] Nicktheprofessor:
Yes but the point would be to enable page mapping and remove that option to make use of QCOM hardware which would offload the CPU from doing memory searches for information a separate circuit is meant to provide.

You don't use advanced debug features that disable optimizations when you are building a debug kernel.

Now, as you probably know we can't run a 64KB mapping with this kernel which is why upgrading to 3.18 is vital but using hardware memory mappings that take advantage of 64KB pages where available is a good thing either way.

There is no reason what so ever not to use available circuits for things that we need to do anyway and in this case, we have a hardware component to do the mapping so we should use that, we should not enable the debug option because it breaks the 64KB path and ensures that the CPU takes over in every instance.

I've already tried the 64KB option with the upgraded MMU but there are tens of thousands of pages of commits left so while it holds great promise, we really need the full upgrade for it to work.

---------- Post added at 12:44 AM ---------- Previous post was at 12:39 AM ----------

It's coming along, I'll get there eventually but it's a 100K+ commits to be added and we have a half arsed kernel where some things have been added through patchwork and i don't want to do a simple merge since no one but me would get it then.

I want a commit sequence that anyone can see and understand and then can use as they see fit.

Floss is the key, baby.

---------- Post added at 12:51 AM ---------- Previous post was at 12:44 AM ----------

I rewrote his anykernel script which turns off msm_thermal (which is a horrid idea since the kernel doesn't have the code to merge general watchers and general thermal interface to do anything at all, if not for hardware protection circuitry that would have fried your chip).

It's really simple, just remove the line that turns off msm_thermal and do a write for any temperature below 44.

And yes, it is possible to fry your chip still, you can set the msm_thermal temperature higher than 44. I was asked how to do that by someone who wanted a replacement unit since he heard that they are now shipping Pixel XL's as replacements, they took one look at it and told him that he's a moron for even trying. Don't do that.
[Nexus 5X Android Development] mr.ace:
[MENTION=7732448]Nicktheprofessor[/MENTION] I don't think disable msm_thermal could hurt anything it's disabled by default even on the stock kernel
[Nexus 5X Android Development] mr.ace:
I'm using stock kernel in current days It's very stable and always give me great performance so I like to keep Shadow kernel slim as possible ,but it doesn't mean I'll drop gcc7.2.1 and upstream-3.10.107

for example, there's stuff named relaxed on the current kernel that should save power but it doesn't show any power saving in the real world and just hurt performance so it's not better to drop them?

I'll drop TEST builds here and users who using my kernel tell me how that works
[Nexus 5X Android Development] Nicktheprofessor:
msm_thermal is not disabled by default in stock kernel? It's compiled and called by the general thermal interface as it should be.

Also, I looked into the hardware more, we have a PARDIS circuit that runs hardware page mapping and if you have mapped pages fragmentation doesn't matter. Looking through the code we are currently only partly using it though. I'd say that before 3.18 it's a matter of battery vs performance, using the PARDIS with the current programming seems to be a half arsed solution so it might be better to just not use it at all since the kernel code is such a horrid mess.

I have to say that most of your updates have been targeted and you've done a good job with it but I also wonder where you are going to go from here? You can't impement EAS without upgrading memory and the storage stuff is limited by the updates to block which is a core function of the kernel. I've also noted that newer zram doesn't work all that well with the old MM configs or the old schedulers. Enabling SMT might help since there is support for running at least one thread for memory management and Oreo is MT per core function.

I don't know if this helps at all. I'm up to the 35000 commit and after 2600 it's flying. Just wish I had more time.

You know what happens next, right? Yeah, finding all that crap that needs to be rewritten to work with the new kernel, fortunately MSM past 805 has general code so it's a matter of adding commits, rewriting only a few thousand lines of code and then...

Well then people like you will have your day with it I suppose. ;)

---------- Post added at 01:00 AM ---------- Previous post was at 12:56 AM ----------

If you'd like I can give you an old commit that I wrote to implement relax throughout the kernel through rewriting the assembly base, I wrote it for 3.4 but it should be easy enough to implement for any kernel if you have some experience with assembly.

It's a great idea to have a testing team, let them finish before release though and don't give users the ability to pressure you into releasing early.

Release often and release early doesn't work on XDA, at all.

Edit: My relaxed patch.
[Nexus 5X Android Development] zurik:
Hello developer, which commits are you used for the 2% drop fix , the wakelock control and how to enable the wakelock restrict by default?

Thank you
[Nexus 5X Android Development] mr.ace:
Fix 2% battery drop : https://github.com/mortezadadgar/shadow/commit/ebe2cb1779b75e46248a6bc59e5beb0c56e294a3

But really this didn't fix anything for me I'll drop it in the next rebase

For wakelock control Franco squached all of them in an one commit : https://github.com/franciscofranco/bullhead/commit/8397cab170d2c7d2dc4dde3bbeae429ec70ddbdd
[Nexus 5X Android Development] zurik:
I am trying to make stock kernel with upstream patches but it causes me headaches with upstream from Nathan's guide.

May you help me a bit?
[Nexus 5X Android Development] mr.ace:
Sure, how can I help?

But I started kernel development from his guide
[Nexus 5X Android Development] zurik:
I will contact you via DM, ok?
[Nexus 5X Android Development] zurik:
Check DM
[Nexus 5X Android Development] Nicktheprofessor:
Simplest possible way to do that is to download this kernel, look up the last version update to the root Makefile and git --reset hard to that commit.

For this kernel it would be git --reset hard 05c03806b794239be413fbd74dc4fb707f8ab7a5

That'd leave you with a stock kernel with upstream updates (not counting the later upstream updates from android common which I do believe are more important) and then you merge it with the latest source from kernel_msm for our branch. Or the other way around, whichever you'd like. You can, of course, also git format patch from the first upstream patch to the last upstream patch and apply that to current kernel source.

There is no need to reinvent the wheel, so to speak.
[Nexus 5X Android Development] zurik:
Thanks for your answer, I will try first from the guide. After if there is a lot of conflicts, I will try to fix them. If something for me will be difficult and I cannot resolve it, I will redownload the source but I will leave it 3.10.73 but with some features that I want!
[Nexus 5X Android Development] Nicktheprofessor:
Why? Just download the shadow kernel, run the git reset --hard command to that commit I gave you and you have the stock upstreamed kernel, if there are new commits to the stock kernel you can simply merge them in or add them in by using git am (preferably git am --reject [commit] and you get a nice .rej file that you can find by using "find -name *.rej | xargs vim" and have them in your editor (replace vim with gedit or whatever you use) and then you use some comparison tool like meld or Kdiff to edit your file and make it work.

This is not complicated at all and if you are going to do anything with an Android kernel you need to learn how commits work.

If something fails you don't need to re-download the whole thing again, you can just git --reset hard to the last commit that works and you're back to where you started from.
[Nexus 5X Android Development] zurik:
Too much suggestions too much things I don't know where to start :laugh:
[Nexus 5X Android Development] mr.ace:
The whole process of this build took 3-days works without a break for me

I wish everyone here feels much better

TEST BUILD: https://www.androidfilehost.com/?fid=673791459329051128
[Nexus 5X Android Development] zurik:
Can you tell me how to find the commit to merge the upstream from a kernel? For example from this Kernel or others....
[Nexus 5X Android Development] Nicktheprofessor:
Assuming that all upstream commits are done in a row it's quite simple, just get the id of the first commit of the upstream process and format the patch from there to the last commit of the upstream process.

Like this: git format-patch b031f221432793456592e49528c233462d434cc4~1..05c03806b794239be413fbd74dc4fb707f8ab7a5 --stdout > upstream-107.patch

That will give you the whole patch in one file, to commit it you use: git am --reject upstream-107.patch

If it just goes through as is, which it may or may not, then all is well. If there is a need to manually edit you can find the problem in the .rej files that are created. When done with your fixing, remove the .rej files (find -name *.rej | xargs rm), add your changes with git add and then run git am --continue until the entire patch is passed.

You can also just run <git format-patch b031f221432793456592e49528c233462d434cc4~1..05c03806b794239be413fbd74dc4fb707f8ab7a5> which will give you a bunch of patches that you then apply, one by one, using git am --reject [name of patch file] but that is a lot more work.

[edit] Another way to do the same thing, which will work fine is to cherry pick the commit range, this saves you the trouble of downloading the remote kernel.

Same idea as above but using cherry-pick from an online repo: git fetch git@github.com:mortezadadgar/shadow.git 8.0.0 && git cherry-pick b031f221432793456592e49528c233462d434cc4~1..05c03806b794239be413fbd74dc4fb707f8ab7a5

To merge in new stuff from your original source you run git pull origin [branch]. For example if you downloaded oreo-r6 and you want to merge in the new stuff from oreo-r4 you run git pull origin android-msm-bullhead-3.10-oreo-r4.

[/edit]