yashsingh1/xdaforums huggingface dataset:
[Xperia Z5 Compact Android Development] zacharias.maladroit:
well, need addr2line info on pc & lr to see what files were involved, that should help narrowing it down

oh - snap :eek:

see if reverting: d946b79ef3fe513ff412c32fa2b9849460791044

helps

perhaps something in drivers/slimbus is having trouble (or not)

disable slub_merging and see if that can help isolate the issue, it helped on my desktop with a faulty network driver (memory & data corruption):

https://github.com/zachariasmaladro...mmit/e07d2ad751946af2a45a02686c0b52cf010ce2db arch/arm/boot/dts/qcom/msm8994.dtsi: disable slub merging, this should make heap overflow attacks more difficult

Commit 40b7b4225666be537f5811dc2a22bac4ebc06c16 is a pretty big bump (Linux 3.10.108) - try reverting that and see if the issues still occur without it
[Xperia Z5 Compact Android Development] trax7:
Rats... Had two more reboots just now, both in YouTube, have only logs for the second one cause it rebooted so fast after I clicked YouTube again very quickly after it booted.
[Xperia Z5 Compact Android Development] GreyLeshy:
A bug with the camera resolved. https://github.com/GreyLeshy/androi...mits/e6b98cdc432af4c6fd7e9936bf9194aac9d32018
[Xperia Z5 Compact Android Development] Pandemicc:
I recorded many and important videos using this kernel, now none of them works. Please help me recover those videos. It is very important for me
[Xperia Z5 Compact Android Development] GreyLeshy:
What kernel was used? Replied to the PM.
[Xperia Z5 Compact Android Development] Pandemicc:
1.10.28 converting them did not work. You need to help me
[Xperia Z5 Compact Android Development] GreyLeshy:
I told you everything in PM about the restoration of the video.

Fix: post 480
[Xperia Z5 Compact Android Development] trax7:
Kernel version 1.11.90 + videocam hotfix:

DL LINK: https://www.dropbox.com/sh/8bso020n2xqyd28/AAAs1-gF-W71ZKokSCNbXPdka?dl=0&lst=
[Xperia Z5 Compact Android Development] trax7:
Had a couple of reboots these days with latest version here, logs3 is while watching YouTube, logs4 is while listening to music over bluetooth speaker and chatting on Messenger. :eek:
[Xperia Z5 Compact Android Development] Tingles:
Got myself a new Diabetic Contour Blood Tester. Can not pair it (Blue Tooth) with my Phone on the last couple kernels. Including last Kernel trax7 released "version 1.11.90 + videocam hotfix" I have no problem with other uses I have for BT, but this is something I would like to use, Blood Tester that is. I didn't grab any logs but if you would like me to try again I will without hesitation
[Xperia Z5 Compact Android Development] GreyLeshy:
On the stock kernel and with the BT stock driver, is the connection present with the device?

Unfortunately, I can not check the driver ANT+, I do not have devices with this protocol.
[Xperia Z5 Compact Android Development] GreyLeshy:
Posted the final kernel version 1.12.48/1.12.70.

End!
[Xperia Z5 Compact Android Development] ModyCZ:
Thanks for all the fish! ;)

The kernel is built and ready in my dropbox folder. Please [MENTION=4644172]trax7[/MENTION] to prepare the changelog.
[Xperia Z5 Compact Android Development] trax7:
The OP already has a changelog for the final version. Thank you for the fast build! :victory:
[Xperia Z5 Compact Android Development] Liquidator:
It's sad to see development on this great kernel stop, but thank you so much for all the work you put into this (as well as ModyCZ and trax7 for the extra support)!

Thanks a lot for all. It's a great daily driver. :)
[Xperia Z5 Compact Android Development] trax7:
There may be more to see in the future. Stay tuned! ;)
[Xperia Z5 Compact Android Development] Tingles:
Grabbed the new release (1.12..70) and Perfect. First thing I did when installed was my Pair my Contour Blood Meter. Since I didn't respond to your question GreyLeshy, although it works now, there was a recognition of the meter but no pairing on your last kernel (Kernels). Now all seems fine. Thanks to all for the work for those involved, GreyLeshy thank you specifically for getting this started. Amazing work!
[Xperia Z5 Compact Android Development] trax7:
[MENTION=8041676]GreyLeshy[/MENTION] I got two random reboots today with the kernel. While listening to music over bluetooth and now in YouTube :( logs attached
[Xperia Z5 Compact Android Development] GreyLeshy:
Support is closed, there will be no fixes!
[Xperia Z5 Compact Android Development] Pandemicc:
AVOID!

The kernel has many problems and the developer has abandoned it. This person did not and will not care if you have problems with this kernel. I say you must AVOID IF YOU WANT TO USE YOUR SMARTPHONE AS A SMARTPHONE. I wanted help from the developer about my lost videos, and the reason was the developer didn't even test the camera. Even though the developer told me that he was able recover one of his, despite my insisting, he refuses to help me and will probably refuse to help you on anything. A worthless kernel from a ruthless person.

AVOID!
[Xperia Z5 Compact Android Development] GreyLeshy:
[MENTION=8952061]Pandemicc[/MENTION] , You have been provided with all the necessary information to restore the video and if you do not understand it, then this is your problem.

Thoroughly I can't explain or even make a manual for You.

[MENTION=4644172]trax7[/MENTION] , at the moment, the kernel is much ahead of the operating system and therefore we see problems with the user space.

[ 6786.942432] PC is at __copy_to_user+0x18/0x70
[ 6786.942499] LR is at file_read_actor+0x128/0x168

[13442.006112] PC is at __copy_to_user+0x18/0x70
[13442.006179] LR is at memcpy_toiovec+0x84/0xc0

Further development is difficult because there is no free access to the necessary sources of QCOM.
[Xperia Z5 Compact Android Development] trax7:
I can see where you are coming from... This is really unfortunate as I and many other people really like and appreciate the work you are doing for our devices.

However, I can also see why people would be unhappy since a final release which has some random reboots is not reliable and, of course, nobody would want to go back to an earlier release in order for this to not happen. I would definitely like for development to continue or at least to have this final bug squashed so we can enjoy your work for the coming months or until someone else takes up the mantle.

Me and undoubtedly the community are very sorry to see you go :( but the decision about what to do has always been yours only... :):good:

Thank you!
[Xperia Z5 Compact Android Development] GreyLeshy:
I spoke about the termination of support long ago, but postponed.

The device is outdated, there is no support from the manufacturer.

My device works stably and it has never rebooted recently.

The fixes will continue to appear on DEV for a while.
[MENTION=4644172]trax7[/MENTION] , at this stage, I cannot fix bugs of the user space because of which at you the kernel crash.
[Xperia Z5 Compact Android Development] Tingles:
GreyLeshy, trax7 and ModyCZ, Thank you for all the work you have done. I have gone back to stock with recovery and Magisk. I will continue to visit this Kernel just like I have for my old Xperia X10 which stills runs great too.
[Xperia Z5 Compact Android Development] GreyLeshy:
VP9 HW does not work without updating the library.

I tested and I had artifacts when playing the video.
[Xperia Z5 Compact Android Development] trax7:
Userspace bugs were caused by SuperSu - confirmed.

Kernel is stable, no random reboots with Magisk 16.

Anyone that wants to use it (and you should, its way better than stock), do NOT use SuperSu. :)
[Xperia Z5 Compact Android Development] zacharias.maladroit:
wat :confused:

Haha - good timing - cause lots of users are seeing random reboots (me included) in using various different stock-based ROMs on Z5 and can't (100%) track the issue down [wiping of internal storage might help for now]

Thanks

which one is that ?
[Xperia Z5 Compact Android Development] trax7:
Flashing osmosis' UnSu zip did the job for me :) don't need to wipe internal storage.
[Xperia Z5 Compact Android Development] zacharias.maladroit:
Thanks for that,

in my cases it definitely helped - so there might have been some additional (potential) factors for instability.

Having a clean base is needed from time to time :silly:
[Xperia Z5 Compact Android Development] creager:
I get the infinite folders too. Haven't found anything useful on the web to help solve it. The directory location changes each time (usually an existing directory though).

The only way to delete I've found is to reformat the card. Tried file managers, and commands from terminal within android system and TWRP recovery

Could it be kernel related? .
[Xperia Z5 Compact Android Development] zacharias.maladroit:
yep, already got that in the past - most probably some FAT (file table) corruption,

doesn't look like it's fixable.

Well, I tried to fix it via linux tools and the program crashed :silly: - didn't have the chance to boot into Windows and fix it,

perhaps Windows is able to fix it.

Nevertheless I formatted the microSD ...

Afaik it happened after several crashes and heavy kernel development on LineageOS or Stock
[Xperia Z5 Compact Android Development] zacharias.maladroit:
Digging up the old discussion:

It's actually strange that the command in your script works:

find ./* | sudo cpio -o -H newc | sudo gzip -9 > ../../ramdisk_$devicename.cpio.gz

I used the exact same command several times in vain,

in any case - it's great to have a known "good" script.

So I can start looking elsewhere why repacking the extracted & modified ramdisk contents don't work
[Xperia Z5 Compact Android Development] zacharias.maladroit:
[MENTION=8041676]GreyLeshy[/MENTION]:

when you compile the Sony FIPS driver INTO the kernel you will NOT run into many problems that the kernel doesn't boot

upon changes in mm/ and other subsystems:

CONFIG_CRYPTO_DEV_SOMC_FIPS_KSCL=y
[Xperia Z5 Compact Android Development] trax7:
[MENTION=3037651]zacharias.maladroit[/MENTION] Hey, could you have a look at these? Those are logs from GreenKernel after disabling ZRAM, had a kernel panic but not sure if it's lack of ZRAM, voltage or smth else :silly:
[Xperia Z5 Compact Android Development] zacharias.maladroit:
HMP scheduling issues related to realtime tasks

kernel/sched/rt.c

find_lowest_rq & select_task_rq_rt

without addr2line output & other details it's rather hard to decipher though.

Was it immediately after disabling zram ?

edit:

keep swap config option in kernel build,

disable zsmalloc, z3fold and zram in config - and try how that works out
[Xperia Z5 Compact Android Development] GreyLeshy:
Yes, basically it really does not work, although occasionally the device was launched.

I use:

find ./* | sudo cpio -o -H newc | sudo gzip -c -9 > ../../ramdisk_$devicename.cpio.gz

Then I throw ramdisk_$devicename.cpio.gz in rootkernel and disable RIC. The device always starts.

Maybe the problem in Ubuntu (I use it), did not try other distributions.

I will add that there were no problems with WSL (Windows Subsystem for Linux).

No problems occurred, everything was stable.

All modules were loaded.

For stock ROM is better to use the module from Sony as it contains the keys.
[Xperia Z5 Compact Android Development] trax7:
I see. I will try that. Btw here's more logs where ZRAM should be untouched. Is it the same? Also I wanna get addr2line so I can be of maximum use and have all the info - how do I get it? :)
[Xperia Z5 Compact Android Development] zacharias.maladroit:
addr2line is part of binutils

so depending on linux distribution you might need to install development tools and other toolchain related packages
[Xperia Z5 Compact Android Development] zacharias.maladroit:
will take a look at that later,

it looks like a swapless kernel isn't that useful however, I didn't see much more consistent experience,

so zram & swap don't really seem to hurt that much ...

Also Google recommends it and that's the same conclusion I came to:

https://source.android.com/devices/tech/debug/jank_jitter#page-cache
[Xperia Z5 Compact Android Development] trax7:
Yes, after more testing I found that the performance improvement is there with and without zram. ZRAM enabled however is more stable, as predicted. I think we need a really low CPU consuming app to monitor CPU stats to find how it impacts CPU use, but apart from that there isnt much difference in my case too...
[Xperia Z5 Compact Android Development] zacharias.maladroit:
that error shouldn't happen (?):

cpuidle (low power mode) related + volts shown

quick guess:

so perhaps overclocking / undervolting (?)
[Xperia Z5 Compact Android Development] trax7:
I was undervolting a bit, yes. Though I've raised voltages by a bit several times due to instabilities... It used to be able to take lower ones in the past, but now not so much. The biggest delta was the big cores' low frequencies, where the default is 820mV and you could go down to 735 in the past.

Edit: 785 seems quite stable now and the voltage curve is okay. In the past it got screwed a lot, in the sense that big cores lower freq (like 1800) would have higher voltages than higher ones (like 2000, those were the usual ones).

Undervolting truly does miracles for this chip, the current thermal config isnt enough to keep it cool but throw in some undervolting of the big cores and bam - instant win, even decreasing them by a bit yields good results and that 735mV above was a godsend for battery life and the phone has even been what I would consider *cold* under low loads where only the little cluster is active (little cluster max overall delta is -5mV, never been able to go lower)...
[Xperia Z5 Compact Android Development] trax7:
[MENTION=3037651]zacharias.maladroit[/MENTION] here are some recent logs, will give debug files after I get home...
[Xperia Z5 Compact Android Development] zacharias.maladroit:
[MENTION=8041676]GreyLeshy[/MENTION] great work !

what effect does specifically the following commit have for you ?

ASoC: msm8994: Add support for 24 bit for session MultiMedia17 (Hi-Res Playback)

Thanks
[Xperia Z5 Compact Android Development] GreyLeshy:
Thank you very much!

In fact, the effect is zero. This stream seems to be not used.

Tested on Hi-Res 24bit/96khz.
[Xperia Z5 Compact Android Development] zacharias.maladroit:
[MENTION=8041676]GreyLeshy[/MENTION] sweet drivers/soc/qcom & drivers/slimbus updates ! :D

Many thanks :)
[Xperia Z5 Compact Android Development] ModyCZ:
Hi. Yesterday I had 3 reboots, all of them while calling. 5 minutes in call and bam! reboot. After phone booted up, I called again, and 5 minutes in call, bum! reboot. And then again. Otherwise the kernel is quite stable. Anyone with the same experience?

EDIT: Got some addr2line hints:

power_cost_at_freq
kernel/sched/fair.c:1854

sched_boost_handler
kernel/sched/fair.c:1750

<6>[ 7858.315493] Disabling non-boot CPUs ...
<1>[ 7858.327575] Unable to handle kernel NULL pointer dereference at virtual address 00000024
<1>[ 7858.327588] pgd = ffffffc00007d000
<1>[ 7858.327607] [00000024] *pgd=000000001020b003, *pmd=000000001020c003, *pte=00e00000f9000407
<0>[ 7858.327627] Internal error: Oops: 96000006 [#1] PREEMPT SMP
<6>[ 7858.327676] Modules linked in: mhl_sii8620_8061_drv kscl(O) ant_drv fm_drv brcm_bt_drv brcm_hci_ldisc qdrbg_module(O) qcrypto_module(O)
<6>[ 7858.327699] CPU: 3 PID: 10423 Comm: kworker/3:2H Tainted: G W O 3.10.108-perf-gb7f5281c9bb3-dirty #1
<6>[ 7858.327704] Hardware name: SoMC Suzuran-ROW (DT)
<6>[ 7858.327748] Workqueue: events_highpri wq_barrier_func
<6>[ 7858.327759] task: ffffffc034a94d00 ti: ffffffc02ab54000 task.ti: ffffffc02ab54000
<6>[ 7858.327780] PC is at power_cost_at_freq+0x78/0xa4
<6>[ 7858.327791] LR is at power_cost_at_freq+0x18/0xa4
<6>[ 7858.327800] pc : [<ffffffc0002652a0>] lr : [<ffffffc000265240>] pstate: 000001c5
<6>[ 7858.327804] sp : ffffffc02ab577a0
<6>[ 7858.327817] x29: ffffffc02ab577a0 x28: ffffffc034a94d00
<6>[ 7858.327830] x27: 00000000ffffffff x26: ffffffc0631a41c0
<6>[ 7858.327842] x25: 0000000000000000 x24: ffffffc0016121c1
<6>[ 7858.327853] x23: ffffffffffffffff x22: 0000000000000008
<6>[ 7858.327865] x21: 0000000000000001 x20: 0000000000000008
<6>[ 7858.327877] x19: 00000000ff00ffff x18: 0000000000000004
<6>[ 7858.327888] x17: 0000000000000000 x16: 0000000000000000
<6>[ 7858.327900] x15: 0000000000000003 x14: 00000719d11ea2e7
<6>[ 7858.327912] x13: ffffffc000e073e8 x12: 0000000000000004
<6>[ 7858.327923] x11: 0000000000000000 x10: 000000000000ba23
<6>[ 7858.327935] x9 : 0000000000000003 x8 : ffffffc001623000
<6>[ 7858.327947] x7 : 00000725a7a7a82d x6 : ffffffc02ab578b8
<6>[ 7858.327959] x5 : 0000000000000001 x4 : 0000000000000008
<6>[ 7858.327970] x3 : 0000000000000024 x2 : 0000000000000000
<6>[ 7858.327982] x1 : 0000000000000000 x0 : 0000000000000024

These comes from the latest sources on github. BUT I have been seeing the same behaviour since the last summer I think.
[Xperia Z5 Compact Android Development] trax7:
I second this. Here's more logs of this happening.

*log wall incoming* :D
[Xperia Z5 Compact Android Development] Pandemicc:
is there any way to install magisk on stock 5803 32.4.a.1.54 without using this crap?
[Xperia Z5 Compact Android Development] GreyLeshy:
Yeah, it's a long-standing problem.

I cannot fix this bug as I have not enough knowledge.
[Xperia Z5 Compact Android Development] GreyLeshy:
Go there!

https://xdaforums.com/xperia-z5/development/root-automatic-repack-stock-kernel-dm-t3301605
[Xperia Z5 Compact Android Development] trax7:
Dear Sir, I will ask you to please show respect towards the developer! He has done a lot for this and many other devices for free.

The fact that he can't fix something now does not mean that his work should be considered with that word that you wrote! He has the absolute right and control over his work and is doing this out of pure good will! We need to be patiend and encourage development if we want this community to stay alive! I hope you think a little about this now...

Good day! :)
[Xperia Z5 Compact Android Development] zacharias.maladroit:
git search turns up:

so try reverting:

sched: auto adjust the upmigrate and downmigrate thresholds [if possible, since that's a dependency of the timer commit]

timer: Queue timers on least power cpu

no sure if 'timer: Queue timers on least power cpu' was enabled on your kernel or not but I recall that there might have been issues in the past for me in connection with it
[Xperia Z5 Compact Android Development] zacharias.maladroit:
[MENTION=8041676]GreyLeshy[/MENTION]

were you able to internally test a sdcardfs-enabled kernel already, btw ?

https://xdaforums.com/apps/magisk/module-enable-sdcardfs-t3603375 & https://www.xda-developers.com/divi...les-fuse-replacement-will-reduce-io-overhead/

look really interesting,

the reduction is overhead is quite significant !

edit:

https://android.googlesource.com/platform/system/core/+/20ca9836b9a780c41a22850f478a29f29677553e

okay, probably close to impossible with Sony's Stock ROM :(

what a shame ...

edit.

I've ran into quite a few issues with the current FUSE (status quo) implementation and it seems: everything must be better :p

WOW - they're NUTS :eek:

and all this "thanks to" Microsoft and Windows to keep compatibility :rolleyes:
[Xperia Z5 Compact Android Development] GreyLeshy:
Was enabled, now disabled...

I build a kernel and checked it, sdcardfs works :):good:

Branch: DEV_v2

/data/media /mnt/runtime/default/emulated sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=1015,multiuser,mask=6 0 0

/data/media /mnt/runtime/read/emulated sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,multiuser,mask=23 0 0

/data/media /mnt/runtime/write/emulated sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,multiuser,mask=7 0 0

/mnt/media_rw/BEC8-B834 /mnt/runtime/default/BEC8-B834 sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=1015,mask=6 0 0

/mnt/media_rw/BEC8-B834 /mnt/runtime/read/BEC8-B834 sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,mask=18 0 0

/mnt/media_rw/BEC8-B834 /mnt/runtime/write/BEC8-B834 sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,mask=18 0 0

/data/media /storage/emulated sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,multiuser,mask=7 0 0

/mnt/media_rw/BEC8-B834 /storage/BEC8-B834 sdcardfs rw,nosuid,nodev,noexec,noatime,fsuid=1023,fsgid=1023,gid=9997,mask=18 0 0
[Xperia Z5 Compact Android Development] GreyLeshy:
I posted the test builds thanks to [MENTION=3037651]zacharias.maladroit[/MENTION].

timer: Queue timers on least power cpu:

disable by default

.

https://www.dropbox.com/s/to7rmgpd78j6o19/Green_Kernel 01 g63da72f.zip?dl=0

Revert

: "timer: Queue timers on least power cpu"; "sched: auto adjust the upmigrate and downmigrate thresholds".

https://www.dropbox.com/s/szkk61mkcqyq31t/Green_Kernel 02 g86fc55e.zip?dl=0

Test the kernels and let me know whether the sudden reboots have disappeared.

Thank you in advance.
[Xperia Z5 Compact Android Development] zacharias.maladroit:
Very nice ! :D

looks like you kept fuse in the kernel (at least you didn't uncomment it),

so what, besides compiling sdcardfs into the kernel,

did you do to get at that stage ?

Were you able to copy files to internal storage and microSD via MTP ?

Thanks

Great news !
[Xperia Z5 Compact Android Development] zacharias.maladroit:
Did any of you guys try to build the kernel with per-process reclaim enabled in addition to zram ?

was it stable ?

I remember that last time someone tried it, it supposedly was unstable or had at least issues ...

edit:

it, in fact, might even need zram since it relies on pages being swapped out for then being reclaimed.

Not really sure how this has to be set up properly ...
[Xperia Z5 Compact Android Development] GreyLeshy:
What kind of fuse are you talking about?

Added the activation module sdcardfs.

Unfortunately, cannot copy files to sdcard, but copying files to PC works.

There is no problem with the internal storage.
[Xperia Z5 Compact Android Development] zacharias.maladroit:
I mean the fuse driver which is used normally for these kind of things and which is replaced with sdcardfs.

Okay, so that ROM commit probably is only for switching back and forth between fuse & sdcardfs but once it works fully that won't be needed ( Give users and devices control over sdcardfs. )

Okay, gotta give it a try once I had a chance to backup my data & system & the kernel is stabilized a bit more.

Thanks !
[Xperia Z5 Compact Android Development] GreyLeshy:
Process reclaim requires debugging because when activate it (enable_process_reclaim), the device restarts after a while.

Activation and parameter setting via init.qcom.post_boot.sh.
[Xperia Z5 Compact Android Development] ModyCZ:
Thanks! Just tested the Green_Kernel_01 and no reboot during 8mins call. Should I try the Green_Kernel_02 too?
[Xperia Z5 Compact Android Development] trax7:
Tested 01 version as well, no reboots the whole day, performance stays high, no occasional hiccups, calls go through, everything seems good :):good:
[Xperia Z5 Compact Android Development] GreyLeshy:
There is no need to test v2.

That's good news, [MENTION=3037651]zacharias.maladroit[/MENTION] gave the right hint, a huge thanks to him for that.

Thanks for the test.
[Xperia Z5 Compact Android Development] GreyLeshy:
I added a nightly folder to dropbox.

In this folder will be laid out the latest version of the kernel.
[Xperia Z5 Compact Android Development] trax7:
A couple more logs - I can see the reboot comes from a Null Pointer dereference somewhere afaik. Heres logs :)
[Xperia Z5 Compact Android Development] GreyLeshy:
android_kernel_sony_msm8994_kitakami/commits/DEV/crypto

FIX for 3.10: "crypto: jitterentropy - return old rng the interface for 3.10"
[Xperia Z5 Compact Android Development] GreyLeshy:
Green_Kernel (04.05.2018) g98bebaa

* A bug is fixed when the screen is activated with a delay of three seconds during an incoming call. (cpufreq interactive)
[Xperia Z5 Compact Android Development] 6opo9a:
Hi, I'm experiencing random reboots with Green_Kernel (04.05.2018) g98bebaa.

Stock ROM 32.4.A.1.54, Green kernel, latest magisk.