yashsingh1/xdaforums huggingface dataset:
[Nexus S Android Development] :
Thats fine, thanks for that. If GPS is in use or bluetooth is active during screen off music playback, it will fall back to TOP ON mode, it only uses that as a fallback, so it's nothing to worry about.

I am looking at enabling it all the time, not just when music is playing back but that is for the next milestone though, and I'm working on squashing a bug which is making it inefficient at the moment, so you probably won't see any decent savings currently.
[Nexus S Android Development] :
IDLE2 v0.213 RC patch here.

This is not final, it depends if there are bugs or not and if my theory about residency time is right. My 2ms residency was far too low. It wasn't generating significant power savings.

I'm working on tuning the residency time to get maximum power savings, so there may be a few RC patches with various values for people to test.

Apply on top of v0.212.
[Nexus S Android Development] DerTeufel1980:
Hi,

Sorry for my late reply. Have been very busy lately.

I got no report of an issue with your idle2 for i9000. Sometimes it seems to stop working (reported not my experience), but maybe it just did not met the requirements in this case.

I got one report of a black screen which did not turn back on, with a captivate (similar to i9000), but these devices always have had issues with deep idle from Ezekeel, too. Would logcat/kmsg help you?

I am also testing your permanent idle2, and no issues to report till now. One time, right after boot, display turned off and not on again. Could not reproduce this.

Gesendet von meinem GT-I9000 mit Tapatalk 2
[Nexus S Android Development] :
So I managed to brick my Nexus S which basically means development here is paused until it comes back from repair. Read my kernel thread if you want to know how it happened, and let's not go offtopic on here please, this is simply an announcement, no comments necessary.

It'll probably be back within a couple of weeks.
[Nexus S Android Development] DerTeufel1980:
too bad :( just was reading what has happend to you.

little question/feedback

two users were reporting, that cpu is running at 800 mhz all the time, they are listening to music.

i have seen, that 800 mhz is defined as the freq to enter the idle mode. is it supposed to stay at this freq all the time? or is there something i don't understand correctly?
[Nexus S Android Development] :
Yeah, it should stay at 800MHz whilst the screen is off and music is active.

I've been modifying the assembly code, trying to make it stay in the idle state for longer, I could use a tester if possible because obviously I can't test it and it's completely untested at this time

PM me if you want the patch to test.
[Nexus S Android Development] :
Is that the same as the zImage you linked for me? I'll be running overnight drain test tonight. Or do you have a fresh one?
[Nexus S Android Development] :
Nah, this is something I did earlier after looking at how it is done in OMAP.
[Nexus S Android Development] :
It took me a while to get around to this, but I built a version without MTP as you suggested, and I can see it is working on the ammeter.

I'm using /sys/module/cpuidle... to enable and disable to compare the power usage.

The needle is bouncing around a little (muted music on a loop) making it hard to judge accurately, but still, disabled looks more power saving. It's bouncing around at a lower level. Only slightly though.

Do you think 800 MHz is necessary, or can I go to 400 MHz, which I believe is more power saving?

After that, have you any further thoughts on the residencies, or is trial and improvement still the way to go?
[Nexus S Android Development] :
I've been working on v0.300 yesterday and today.

So far, the diffstat is: 8 changed files with 256 additions and 608 deletions.

Within idle2.h i'm now using the platform suspend code to put the CPU into for C2 and C3 states and that enabled me to delete a load of code, such as idle2.S, the cpufreq frequency locking has been deleted, I removed several unneeded / unused checks in the code paths. I shifted the code around to make it cheaper if there is an interrupt pending and it has to bail before entering into an idle state. I modified the idle configuration selecting because we were messing about with an undefined bit which was somewhat pointless.

Within cpuidle.c the top on code is cheaper as I removed some redundant checks. I replaced a bunch of bools with an unsigned char and bitwise operations, I deleted the idle2_disabled & topoff_enabled module parameters and added a function to disable the hlt instruction for 30 seconds after booting starts. I have also conducted measuring of latencies for each state and set the exit latencies accordingly, although the residency is still a bit of of unknown at the moment, although having the exit latencies completely wrong wouldn't have helped.

It's also enabled all the time. It will either use C1 or C2 state when the screen is on and C1, C2 & C3 states when the screen is off.

C1 = IDLE

C2 = DEEP-IDLE TOP ON

C3 = DEEP-IDLE TOP OFF

With all the code changes, the ammeter test results on v0.2xx won't be relevant now, and if it's fluctuating around, it's not accurate anyway.

I'll release it when I'm happy with its stability and performance, whenever that may be. Stability is looking good, had a few imprecise external aborts yesterday whilst testing, but they appear to have been resolved after switching toolchains.
[Nexus S Android Development] :
By fluctuating I mean 'fluctuating a little bit around a point', not so much that it's hard to read, just difficult to assign a specific value to. With it enabled it was definitely higher. I can video it if you like, but it seems redundant now.
[Nexus S Android Development] :
Just a little teaser of something I've knocked up in the last hour or so:

adb shell cat /sys/kernel/debug/idle2/error_stats

------------------IDLE2 State Statistics------------------

C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
CPUidle requested state: 11799 5460
State entered: 11332 5435
State completed: 11196 5435
State not completed: 136 0

Requested & completed: 94% 99%
Entered & completed: 98% 100%
Failed: 1% 0%

-----------------IDLE2 Failure Statistics-----------------
VIC Interrupt MMC Active Clock Gating
----------------------------------------------------------
Bail Counts: 307 185 0

----------------IDLE2 Residency Statistics----------------
C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
Total time in state (ms): 122995 35508
Average time in state (us): 10985 6533
----------------------------------------------------------

I appear to have fixed the bug which was killing the power saving abilities of IDLE2, basically every time it was going into an idle state, it was being woken up by an unmasked interrupt, hence why the times in state weren't as good as they should have been, and the cpuidle governor was preferring C1 instead of C2 or C3, because it was waking as soon as it went it, but that is fixed now and I'm seeing some very healthy residency times. :)
[Nexus S Android Development] knzo:
Looks very promising. :)
[Nexus S Android Development] polobunny:
Very promising indeed! :D

I've found out that you can use Multi Mount SD Card widget to mount your SD card à la MTP but without using MTP in your kernel. Results? Not breaking IDLE2 but very fast transfers even from within Android. Who said you couldn't have it all?
[Nexus S Android Development] apatal:
What the? Where can I find this miraculous widget you speak of? ;)

Sent from my Nexus S
[Nexus S Android Development] sdhanjal15:
https://play.google.com/store/apps/details?id=com.rafoid.multimountsdcard.widget.free
[Nexus S Android Development] :
I'm just experimenting with different residency values at the moment.

Got the average time in state to 8ms with a target residency of 5ms.

I might release a pre-patch later for people to test. :)
[Nexus S Android Development] :
Some more stats, phone rebooted, adb wireless connected, wakelock taken:

adb shell "echo hack > /sys/power/wake_lock"

and screen turned off and phone left.

------------------IDLE2 State Statistics------------------

C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
CPUidle requested state: 211 14706
State entered: 8 14512
State completed: 8 14512
State not completed: 0 0

Requested & completed: 3% 98%
Entered & completed: 100% 100%
Failed: 0% 0%

-----------------IDLE2 Failure Statistics-----------------
VIC Interrupt MMC/NAND Clock Gating
----------------------------------------------------------
Bail Counts: 396 1 0

----------------IDLE2 Residency Statistics----------------
C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
Total time in state (ms): 6 501221
Average time in state (us): 817 34538
----------------------------------------------------------

Can't complain at those. :)
[Nexus S Android Development] :
I should probably explain the stats quickly...

CPUidle requested state means that the state was selected for entry by the CPUidle governor.

State entered means the IDLE2 code actually entered into the selected state.

State completed means the state was ran to completion, ie, the A8 was power gated and TOP/SUB blocks were disabled if selected.

State not completed means the state was entered and the prep was started, but it bailed before it could actually call the sleep code. The only reason for this is either the sleep state doesn't exist (which wouldn't happen anyway), or a VIC interrupt is still pending after they are disabled (this scenario should be rare).

Percentages are self explanatory, ideally, you want "Requested & completed" and "Entered & completed" to be as high as possible and "Failed" to be as low as possible.

I tend to get >90%, >99% <2%.

Bail Counts are times the state was skipped due to certain issues, VIC interrupts pending, MMC / NAND active, or certain clocks not gated. These stats aren't a bad thing, because it shows we haven't attempted to enter the state, we have skipped it and it is much cheaper to do this than attempt the state then immediately wake up.

Total time in state is the total amount of time spent in the idle state since boot, shown in milliseconds. This is only added when the state is actually completed and is not added if we bail.

Average time in state is the "total time in state" divided by "state completed", which is shown in microseconds for precision.
[Nexus S Android Development] apatal:
Thanks for the explanation. I finally see the beauty of the previous stats. As another poster mentioned, looks quite promising. .31 build with this is going to be great in terms of power saving.

Sent from my Nexus S
[Nexus S Android Development] :
Yet another follow up...

There appears to be a bug...

------------------IDLE2 State Statistics------------------

C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
CPUidle requested state: 43304 111338
State entered: 29791 957
State completed: 29652 957
State not completed: 139 0

Requested & completed: 68% 0%
Entered & completed: 99% 100%
Failed: 0% 0%

-----------------IDLE2 Failure Statistics-----------------
VIC Interrupt MMC/NAND Clock Gating
----------------------------------------------------------
Bail Counts: 123522 372 34

----------------IDLE2 Residency Statistics----------------
C2 (TOP ON) C3 (TOP OFF)
----------------------------------------------------------
Total time in state (ms): 550634 13167
Average time in state (us): 18569 13758
----------------------------------------------------------

It seems like the VIC permanently had a pending interrupt.

Toggling the wifi on then back off resolved it.

Going to rework the code slightly to disable interrupts earlier to see if we can avoid this.

adb shell cat /proc/interrupts
CPU0
16: 347 s3c-uart s5pv210-uart
18: 1516 s3c-uart s5pv210-uart
20: 1466 s3c-uart
22: 1026 s3c-uart
28: 14 s3c-uart
30: 3306 s3c-uart
33: 508174 s5p_vic_eint acc_int
34: 1 s5p_vic_eint proximity_int
38: 41 s5p_vic_eint sec_headset_detect
39: 3 s5p_vic_eint max8998-charger
43: 148831 s5p_vic_eint modemctl_mbox
44: 432 s5p_vic_eint pn544
47: 2 s5p_vic_eint modemctl_bp
50: 0 VIC s3c-pl330.0
51: 0 VIC s3c-pl330.1
52: 260 VIC s3c-pl330.2
58: 13 VIC System timer
61: 1345591 VIC rtc-tick
78: 15140 VIC s3c2440-i2c.0
83: 113686 VIC s3c2440-i2c.2
88: 8654 VIC s3c-udc
90: 175592 VIC mmc0
92: 24232 VIC mmc1
95: 5451 VIC onenand
97: 34662 VIC s3cfb, s3cfb
101: 400 VIC s3c-fimc0
102: 0 VIC s3c-fimc1
103: 0 VIC s3c-fimc2
104: 1 VIC s3c-jpg
106: 250936 VIC pvrsrvkm
109: 4106697 VIC s3c2440-i2c.1
110: 0 VIC s3c-mfc
112: 78309 VIC
130: 30592 VIC mmc2
157: 0 VIC debug-signal
164: 1457 s5p-eint bcmsdh_sdmmc
165: 17 s5p-eint bt_host_wake_irq_handler
166: 147 s5p-eint gpio_keys
167: 2 s5p-eint fsa9480 micro USB
169: 25 s5p-eint gpio_keys
170: 18 s5p-eint gpio_keys
171: 0 s5p-eint akm_int
172: 0 s5p-eint s3c-sdhci.2
173: 0 s5p-eint k3g
308: 8706 s5pv210-eint-group mxt224_ts
334: 197 s5pv210-eint-group cypress-touchkey
FIQ: fiq_glue
Err: 0
[Nexus S Android Development] brenix:
Thanks for the explanations. Looks promising! I'm excited to try some of the new patches and test it out on my kernel... I can provide stats if needed for debugging..
[Nexus S Android Development] FloHimself:
Any news about this bug and idle2? Did you resolve it?
[Nexus S Android Development] :
I did resolve that bug, that was actually due to the HSMMC3 channel being activated after suspend if the wireless wasn't enabled. I worked around that by not checking HSMMC3 status, which I didn't need to do anyway.

I do still have a stability issue though, in the guise of an 'Imprecise external abort'.

The issue which is being triggered by idle2 in v7_flush_kern_dcache_area(). It's not an idle2 bug, because the stacktrace goes nowhere near idle2 code, it's an existing platform bug which is being triggered when idle2 is enabled, which is being triggered by an Android futex() syscall, which is ultimately being triggered by the AudioOut_2 process.

What I think it is is a simple race, between the idle2 cache flush operation and the one triggered from the syscall, but then, my asm is n00bish at best. I suspect the range being requested has already been flushed by idle2 (we have to flush all the caches on every entry) by the time v7_flush_kern_dcache_area() is actually called, which means it is trying to flush an invalid area and borks.

I may *attempt* to add some error handling asm to it but currently I have reverted the platform code to the old method of flushing a page and it does appear to be working, at the moment.

If I'm happy that it is stable, I'll release the code. Normally it triggers within an hour, or about 10 minutes if I'm using TOP-ON and playing music. So far, so good though.
[Nexus S Android Development] :
I can barely understand C. What chance do I have of understanding ASM? :p

Can't the error handling be written in C and called from the ASM? It all ends up compiled anyway right? I know you're obsessed with cheap code paths, but isn't hand coding an error handling routing in assembly a bit masochistic? :p
[Nexus S Android Development] :
What does it matter if *you* understand it or not? Whose project is this lol?

I think what matters is that *I* understand enough to cludge round the bug. And no, not for cache flush operations, they are all done in assembly.

Anyway, I added an abort handler in the asm to flush the entire dcache if the virtual address held in r0 isn't mapped. Testing it now and it hasn't blown up yet, so I could be onto something.

It's as simple as:

9001:
b	v7_flush_dcache_all
mov	pc, lr
[Nexus S Android Development] :
It doesn't matter in the slightest! That was my attempt at self-deprecating humour. This is now my attempt at self-deprecating my weak self-deprecation skills.

Anyway, best wishes with the ASM, it looks very difficult.
[Nexus S Android Development] knzo:
Sucks to read that. Any more ideas to fix that race? I hope you don't give up, IDLE2 will be your crespo's magnus opus!
[Nexus S Android Development] :
Yeah...I have a few ideas. I guess I need to learn how to read a register dump firstly so I'm not stabbing in the dark.
[Nexus S Android Development] :
I could actually use some help from someone who knows low level ARM programming to be honest. If *I* manage to fix it, it will be because of a complete fluke, and not because I know what I'm doing, because this is wayyyyy over my head.

My understanding of low level ARM programming is none, none and none.

If such an individual exists on here, please come into my IRC channel which is linked in my sig.

All I understand so far is that it's an imprecise external data abort.

So far, from the documentation available, my understanding of this is that it is external because it comes from an error response on the AXI interface and it is imprecise because the abort may have occurred on an instruction subsequent to the one which causes the abort exception and it is a data abort because it is associated with data access.

Also, I doubt that it is to do with the cache, because an L2 cache abort would be an internal precise data abort. So I'm guessing (again) that it is related to another memory access, possibly a fetch from DRAM.

But hey, all of the above is guesswork...so meh. :p
[Nexus S Android Development] knzo:
I know only of one guy who may be of aid: dzo.

In my opinion he's the best developer in XDA, try sending him a PM.
[Nexus S Android Development] :
Oh yeah, he did some work on my bravo kernel. He is a professor at Massey University, I've got his email address, I'll give him a shout and see if he can help.
[Nexus S Android Development] knzo:
Exactly, he has a PhD in OS and he ported

Android

to the HTC Vogue; he's the only one I remember that should be able to help.
[Nexus S Android Development] :
Yeah, I read his presentation on it. Anyway, IDLE2 can wait for a few days, I've wasted too much time on it recently, although the core code is a lot more robust now, so it's not all wasted.

The thing that concerns me is this *could* be a hardware issue which can't be resolved. Funny thing is, it's much more reproducible with the TOP block on than when the TOP block is off.

But, just because I get it on my phone, it doesn't mean others will get it on theirs. Not every chip is the same.

But I'll ask Martin and see what he thinks.
[Nexus S Android Development] :
I've just pushed the latest IDLE2 code to https://github.com/thalamus/android-kernel/commits/IDLE2-v3.xx-wip

Diffstat:

arch/arm/mach-s5pv210/Makefile | 4 +-
arch/arm/mach-s5pv210/cpuidle.c | 608 +++++++++++++++--------
arch/arm/mach-s5pv210/herring-rfkill.c | 5 +-
arch/arm/mach-s5pv210/idle2.S | 236 ---------
arch/arm/mach-s5pv210/include/mach/cpuidle.h | 14 +-
arch/arm/mach-s5pv210/include/mach/idle2.h | 534 ++++++++------------
arch/arm/mach-s5pv210/include/mach/regs-clock.h | 4 +
arch/arm/mach-s5pv210/sleep.S | 21 +
arch/arm/plat-s5p/Kconfig | 3 +
arch/arm/plat-samsung/include/plat/pm.h | 2 +
drivers/tty/serial/serial_core.c | 4 +-
sound/soc/samsung/s5pc1xx-i2s.c | 14 -
12 files changed, 657 insertions(+), 792 deletions(-)

Caveats:

There are no commit messages as I like to do them right with a full explanation and I haven't had the time or the will to do them at the moment.

It's still work in progress, so there may be things you spot in the code and say, hmmm, whats that about.

There are still quite a few things to clean up.

There could well be an issue with imprecise external aborts, but your mileage may vary.

Tons of code churn since v2.xx, so there could well be bugs.

Benefits:

It does save power. Exactly how much is unclear, but it does.

Uses platform suspend code to save state and enter idle, this has made it much more stable.

More of the code is commented to say what it is doing.

Code is cheaper and faster than ever before.

Uses all 3 states whenever circumstances permit, has a logical fallback for when a state is impossible at the time of entry.

Extensive statistics:

adb shell cat /sys/kernel/debug/idle2/error_stats

Request:

**Strictly Developers Only** If you find any *new* bugs with it, add them to my issue tracker. Mark it with the Nexus S - S5PV210 and Project - IDLE2 labels please. No IDLE2 bugs from users though please.
[Nexus S Android Development] brenix:
Very nice! Thank you! Testing now.. will report anything strange I notice..
[Nexus S Android Development] knzo:
I built a version with BLN: link-removed

However, I don't know what happened in my brain but I transformed IDLE2 v0.3xx into IDLE3 so apologies for the mislabel.
[Nexus S Android Development] :
That really wasn't what I had in mind when I released the code. If I wanted it released to the masses as binaries, I would have done that myself. I didn't, so that kinda says that I didn't want that, yet. :(

My point of releasing the code was for some developer review / testing and possibly some *limited* user testing, not open the floodgates to all the morons.

It's a wip, which means it's not ready for widespread release.
[Nexus S Android Development] knzo:
Ups, I misread the intentions. Nuked the links.
[Nexus S Android Development] :
It's cool. It's just not ready for that yet, thats all. It hasn't had enough testing, as only I have been testing it. I only pushed it because I don't want it becoming vapourware. :p

But if you have a handful of people who can test it, who can provide good feedback, that would be helpful.

But, as I said in my post, I only want developers to open issues on my tracker, not users. I don't want to have to delete a load of crap off there. I want accurate, descriptive, reproducible issue reports.
[Nexus S Android Development] :
Are you satisfied with the residencies? I was unable to get /sys/devices/system/cpu/cpu0/cpuidle/state2/usage stats so i raised state2 residency to 10000 and state1 to 12000 (i assume that still needs to be higher?)

I can now get time accumulating. Also, what's the reason for raising state0 residency and exit latency from 1 to 20?
[Nexus S Android Development] :
Yes. Those residencies are not arbitrary values. I spent a lot of time profiling with the linux tracing infrastructure. You will see I added a tracer to output the residency time. 10000 and 12000 is not ideal, at all.

Strangely enough, everything works fine here the way it is, with no fiddling needed, and both states should have the same target residency.

I'm unsure as to why you think *raising* the residency would make you start accumulating time. You would accumulate more time at a lower residency...

I suspect there is something else at play here, as per usual.

Perhaps bother to check the stats which would tell you why which I spent considerable time developing for issues exactly like this. It tells you why it failed to enter the state.

C1 was raised due to the results I got from profiling latencies and from testing the exit latency of the state. I didn't choose arbitrary values.