yashsingh1/xdaforums huggingface dataset:
[Desire Android Development] melethron:
Update:

Recovery with new Kernel

(includes support for ReiserFS, btrfs and jfs - thx coutts99 for the kernel):

http://files.c0urier.net/melethron/index.php?dir=Avaritia_HD/recovery/

New Alignment zip files added

(btrfs and jfs - thx ownhere for utils):

http://files.c0urier.net/melethron/index.php?dir=Avaritia_HD/alignment/

(also thx to c0urier again for hosting )

Please note that:

- btrfs is still under developement. It is in rather stable state and many linux users use it already but its still marked as EXPERIMENTAL

- jfs is intended for proffessional server enviroment and it is rock solid there and proved to work fast, stable and with low cpu consumption - but this applies to battery backed (!) harddisks ... a powerloss / reboot on the phone can destroy the complete partition.
[Desire Android Development] xavierjohn22:
And i thought i could sleep early tonight, ha ha. I'm a flash addict so this comes in handy. thanks for this update melethron

i'll just need the Custom CWM + alignment zip files and im good for now.
[Desire Android Development] Muxeu:
melethron: tnx a lot for new recovery. Will try btrfs.
[Desire Android Development] melethron:
LOL:

ROM: 1.72 rebase of the ROM.

Kernel: Coutts CFS kernel (JFS version).

Governor : Performance (only for bench)

OC 1113.

Filesystem: JFS (direct mount = no loop)

+ libsqlite hacked (will be life-on-the-edge in the next version).

But thats not for daily use. I tried an higher OC and got a freeze (battery pull then). I hadn't filesystem check not yet included and got this:

I//system/xbin/busybox( 70): +++ Checking external Partition:
I//system/xbin/busybox( 70): mount: mounting /dev/block/mmcblk0p2 on /data failed: Invalid argument

xD

Still funny to get such a score without loop.

EDIT: Filesystem wasn't broken. The reason it couldnt be mounted is this:

JFS magic number: 'JFS1'
JFS version: 1
JFS state: [COLOR="Red"] mounted[/COLOR]
JFS flags: JFS_LINUX JFS_COMMIT JFS_GROUPCOMMIT JFS_INLINELOG
Aggregate block size: 4096 bytes
Aggregate size: 2975240 blocks
Physical block size: 512 bytes
Allocation group size: 8192 aggregate blocks
Log device number: 0xb302
Filesystem creation: Fri Jan 14 14:27:44 2011
Volume label: 'jfs'

Although it WASN't mounted (no clean unmount) and mount doesn't work if it is unmounted:

mount: mounting /dev/block/mmcblk0p2 on /data failed: Invalid argument

After i run fsck.jfs:

~ # /system/xbin/fsck.jfs /dev/block/mmcblk0p2
/system/xbin/fsck.jfs version 1.1.9, 04-Oct-2005
processing started: 1/14/2011 16.17.25
Using default parameter: -p
The current device is: /dev/block/mmcblk0p2
Block size in bytes: 4096
Filesystem size in blocks: 373504
**Phase 0 - Replay Journal Log
[COLOR="Red"]Filesystem is clean[/COLOR].

So there is no error and the system is clean. After that i get:

JFS state: clean

and mount works fine:

~ # mount /sd-ext
~ # cd /sd-ext
/sd-ext # ls
Avaritia app-private data local system
anr backup dontpanic misc tombstones
app dalvik-cache drm property zipalign.log
/sd-ext #
[Desire Android Development] coutts99:
http://www.linux.com/archive/feed/119025

Have you tried any recovery of crashed filesystem with JFS?
[Desire Android Development] Muxeu:
ReiserFS / BTRFS / JFS

Same kernel (coutts SVS-CFS-BTRFS-JFS Jan 14 2011 10:03:08 AM)

OC 1113

libsqlite patched

SD Transcend Class 6
[Desire Android Development] melethron:
Yep and it worked. Didn't know that jfs can't be mounted if its in an "unclean" state because no flag by unmount was set (works on all other fs).

I have some issue on dalvik-cache now but that can come from the boot wihout a proper mount.

BTW:

E/dalvikvm( 676): Bad cgroup data {8ƫJh��}
E/dalvikvm( 676): Bad cgroup data {8ƫJh��}
E/dalvikvm( 676): Bad cgroup data {}
E/dalvikvm( 676): Bad cgroup data {}

On your CFS kernel (with jfs).... can you make a a BFS one with JFS and btrfs.
[Desire Android Development] woti23:
the problem - i guess - is the same source for CFS and BFS compiles, because now the CFS source also includes allready BFS patches. there is no sense to mix up CFS and BFS. so we get errors on CFS because of the BFS patches and on BFS because of missing parts of cgroups.
[Desire Android Development] coutts99:
The CFS source had the BFS patch when I split them (and weren't getting CGROUP errors), what it didn't have was the ck patches. I;ve now removed the ck patches, so it is the same as it was when split, if that makes sense :) I've been tidying stuff today, may have got the CGROUP settings wrong?

CONFIG_GROUP_SCHED=y

CONFIG_FAIR_GROUP_SCHED=y

# CONFIG_RT_GROUP_SCHED is not set

CONFIG_CGROUP_SCHED=y

CONFIG_CGROUPS=y

# CONFIG_CGROUP_DEBUG is not set

# CONFIG_CGROUP_NS is not set

CONFIG_CGROUP_FREEZER=y

# CONFIG_CGROUP_DEVICE is not set

CONFIG_CGROUP_CPUACCT=y

# CONFIG_CGROUP_MEM_RES_CTLR is not set

# CONFIG_SCHED_AUTOGROUP is not set

# CONFIG_CGROUP_BFQIO is not set

# CONFIG_NET_CLS_CGROUP is not set

Thats what I have now.

Anyway http://mirror.couttstech.com/android/bravo/kernel/Test/SVS-BFS-BTRFS-JFS.zip

I'm updating of my servers and it's going very slow, so just keeping trying to download :)
[Desire Android Development] woti23:
i have your kernel source compiled CFS/CGS and running now for a few days just fine.

btw: SLQB didn't compile in this config, i took SLAB, i also left out BFQ

CONFIG_GROUP_SCHED=y

CONFIG_FAIR_GROUP_SCHED=y

# CONFIG_RT_GROUP_SCHED is not set

CONFIG_CGROUP_SCHED=y

CONFIG_CGROUPS=y

# CONFIG_CGROUP_DEBUG is not set

# CONFIG_CGROUP_NS is not set

CONFIG_CGROUP_FREEZER=y

CONFIG_CGROUP_DEVICE=y

CONFIG_CGROUP_CPUACCT=y

# CONFIG_CGROUP_MEM_RES_CTLR is not set

CONFIG_SCHED_AUTOGROUP=y

# CONFIG_NET_CLS_CGROUP is not set
[Desire Android Development] Fightspit:
Excellent !

It could be a holy grail ROM :D
[Desire Android Development] Terepin:
melethron, spare us of your talking about new ROM and RELEASE IT ALREADY FOR GOD'S SAKE!!!
[Desire Android Development] melethron:
Thanks im testing it the next minutes.

A Rom that doesn't boot wouldn't be something to release! I completely rewrote the init.d scripts and there were some bugs causing issues. Its not a problem for me to fix this while it boots because i fix the issue in the shell.

Also i wanted to include a kernel with BFS (without ck2 patches because the swap bug) with JFS and BTRFS support (because there are some people that want btrfs). And as you might have noticed coutts just compiled that some minutes ago....

Apart from that i needed the utils for btrfs and i also got them just today from ownhere.

In the last 2 hours i completly rewrote the script parts with fsck utils so that jfs and btrfs works without the need to stuff in adb shell ....

If you have a problem that i dont release half finished roms that wouldnt even boot on some conditions then use another rom.....

Edit: About the progress. Almost everything is fixed now. I wanted to update apks and check some apks. I'm pretty sure that it will be finished tomorrow.
[Desire Android Development] Alex-V:
mhhh i have the same bad croup problem with the bfs kernel that i have with the cfs kernel...(the btrfs kernel with my rom)

Did you have a idea..?

with kind regards...Alex
[Desire Android Development] Terepin:
Just because I didn't use smiley it doesn't mean I'm serious, you know... ;) I mean it as a dry joke.
[Desire Android Development] melethron:
Didn't test the kernel yet. I'm writing some userconfig stuff atm.

k :D
[Desire Android Development] martino2k6:
Recoveries currently don't support BTRFS, right? If the answer is no, then there currently isn't any way to back up other than mounting the card in Linux and copying the contents of the partition?
[Desire Android Development] Alex-V:
melethron has made one :)

http://xdaforums.com/showpost.php?p=10597539&postcount=455

with kind regards..Alex
[Desire Android Development] coutts99:
He's customised a recovery
[Desire Android Development] Alex-V:
ah ok..this explain´s it :p:p

with kind regards..Alex
[Desire Android Development] martino2k6:
Ah! Thank you, silly me. I thought he made just a ReiserFS version. Now there are so many filesystems we can use that it's going to be difficult to choose one lol
[Desire Android Development] Terepin:
Will you be switching to Edify?
[Desire Android Development] melethron:
I like to have the choice ;) . Especially if there seems to be so many difference for sd-cards
[Desire Android Development] melethron:
yep. i rewrote the script 2 weeks ago already because i wanted some ascii in the installer :D
[Desire Android Development] Muxeu:
Just tested JFS too and added its Quadrant score to post

Do you think BFS kernel will be better?
[Desire Android Development] martino2k6:
Nothing wrong with that :)

Can't wait for my 32GB SanDisk card to arrive and then I'll be finally free from dangerously low internal memory problems! Of course, only if you manage to release by then ;)

On a side note, data2sd is probably not great for the card's lifetime but it shouldn't be too bad, right?
[Desire Android Development] Alex-V:
Ok...i´m on BTRFS for a test now...with neo´s test kernel (hope coutts can fix the cgroup error :) )

and for what i can say at the moment very good feeling with this... fast..and very responsive....like it :)

so i say big thx for recovery edit and for the alignment scripts :) also coutts and neo for the kernel ;)

with kind regards...Alex
[Desire Android Development] melethron:
Be careful with jfs. If it isn't unmounted by a normal shutdown like useing "adb reboot" or the reboot button in the powermenu then the partition cant get mounted again.

On ext4 there is a "clean" and "unclean" flag set for that (so the fsck knows if it needs a force check). If the partition is "unclean" it can still get mounted.

On jfs there is a "clean" and "mounted" flag. If that flag is still set after reboot then the partition wont be mounted. Only after fsck.jfs is run it is "clean" an can be mounted. Since jfs is made for IBM server they made some almost paranoid data safety stuff there.

When the card does some good wearleveling it can be overwritten 100.000. The day the card dies we will have 16 TB cards or something like that. ;)

Note that cards can still die "randomly" like every hard disk.
[Desire Android Development] melethron:
E/dalvikvm( 695): Bad cgroup data {}
E/dalvikvm( 695): Bad cgroup data {}
E/dalvikvm( 695): Cannot get policy, owner thread id=708
E/dalvikvm( 695): Cannot get policy, waiter thread id=724
E/dalvikvm( 695): Bad cgroup data {}
E/dalvikvm( 695): Bad cgroup data {}
E/dalvikvm( 695): Cannot get policy, owner thread id=708
E/dalvikvm( 695): Cannot get policy, waiter thread id=724
E/dalvikvm( 695): Bad cgroup data {}
E/dalvikvm( 695): Bad cgroup data {}

Dont know where that comes from. Config seems all right:

zcat /proc/config.gz | grep GROUP
# CONFIG_CGROUPS is not set
# CONFIG_SCHED_AUTOGROUP is not set

:(
[Desire Android Development] Alex-V:
same on mine with both coutts test kernels...mmhh hope he can fix it.. :)

edit: but i know on the start of the cfs kernel..it was a similar issue..

with neo´s test kernel i Don´t have this

with kind regards...Alex
[Desire Android Development] Muxeu:
I return to btrfs :rolleyes: Seems the CW recovery doesn't make a sd-ext image on JFS.

/sdcard/clockworkmod/backup/2011-01-14.20.56.45 #
ls -la
drwxrwxrwx 2 root root 4096 Jan 14 21:00 .
drwxrwxrwx 5 root root 4096 Jan 14 20:56 ..
-rwxrwxrwx 1 root root 64914432 Jan 14 21:00 .android_secure.img
-rwxrwxrwx 1 root root 3145728 Jan 14 20:56 boot.img
-rwxrwxrwx 1 root root 12672 Jan 14 21:00 cache.img
-rwxrwxrwx 1 root root 130458240 Jan 14 21:00 data.img
-rwxrwxrwx 1 root root 222 Jan 14 21:00 nandroid.md5
-rwxrwxrwx 1 root root 4718592 Jan 14 20:56 recovery.img
-rwxrwxrwx 1 root root 250542336 Jan 14 20:59 system.img

I restored from btrfs image:

/sdcard/clockworkmod/backup/2011-01-14.19.38.36 #
ls -la
drwxrwxrwx 2 root root 4096 Jan 14 19:44 .
drwxrwxrwx 5 root root 4096 Jan 14 20:56 ..
-rwxrwxrwx 1 root root 64914432 Jan 14 19:42 .android_secure.img
-rwxrwxrwx 1 root root 3145728 Jan 14 19:38 boot.img
-rwxrwxrwx 1 root root 12672 Jan 14 19:42 cache.img
-rwxrwxrwx 1 root root 130437120 Jan 14 19:42 data.img
-rwxrwxrwx 1 root root 267 Jan 14 19:44 nandroid.md5
-rwxrwxrwx 1 root root 4718592 Jan 14 19:38 recovery.img
-rwxrwxrwx 1 root root 203212416 Jan 14 19:44 sd-ext.img
-rwxrwxrwx 1 root root 250542336 Jan 14 19:41 system.img

And what's a message about /dev/tty0 in recovery.log?

Starting recovery on Fri Jan 14 21:35:58 2011
can't open /dev/tty0: No such file or directory
[Desire Android Development] coutts99:
You have config cgroups not set? this the cfs or bfs kernel?

Ill check that tomorrow, are the normal non test cfs kernels OK?
[Desire Android Development] Alex-V:
i have it on both kernels (with my rom) if i use the non test kernels its ok on cfs (all versions) kernel and again with bfs

Sorry for report..I know its much work again

with kind regards..Alex
[Desire Android Development] coutts99:
I must have missed cgroups when building the test cfs kernel, the normal ones are built from a script so are always correct :)

Ill do the cfs one again tomorrow.

You'll always get cgroups error with bfs
[Desire Android Development] melethron:
That zcat was from svs bfs with jfs and btrfs. Shouldn't be set there. Well since the normal one work fine i think its not hard to find that issue:

also take a look at that patch: http://git.zen-kernel.org/zendroid/...s&id=3b03259b192f0fedd35e89cddb772d82e5ed77ae

+ The Simple I/O scheduler is an extremely simple scheduler,

+ based on noop and deadline, that relies on deadlines to

+ ensure fairness. The algorithm does not do any sorting but

+ basic merging, trying to keep a minimum overhead. It is aimed

+ mainly for aleatory access devices

(eg: flash devices).

Ill test your zenkernel now. Just need to nandroid before.... partition is jfs and i dont think you included that. EDIT: thought you compiled a complete zen kernel ... didnt see you just added v(r) sched.
[Desire Android Development] coutts99:
I've had a good drink tonight and not taking much in at the minute.

Ill re read this thread tomorrow :)
[Desire Android Development] Alex-V:
thanks :) ;) :p

with kind regards...Alex
[Desire Android Development] Alex-V:
Also very interesting :) News on news from day to day :p:p:p:)

with kind regards...Alex
[Desire Android Development] woti23:
you get it because config cgroups is not set ... but when you set it together with BFS patches it can cause other difficults when compiling BFS kernel

(as the two don't fit together well, but cgroup is needed to get no error messages, at 1.72 roms at least)

also note the two things:

cgroup = control group

and there is a group cpu scheduler
[Desire Android Development] Alex-V:
is this a true real problem or is it not really a problem..?

anyway..error´s are not good for a stable phone or.. i never use bfs on 1.72..because of this error

with kind regards...Alex
[Desire Android Development] melethron:
There must still be something that tries to attach something to a cgroup. ownhere said he searched the aosp source and didn't find anything. So this is either a HTC thing or kernel related. But having a log with more errors (errors and not only warnings) than normal debug or verbose messages isn't normal.

About autogroup scheduler: i know that "CONFIG_SCHED_AUTOGROUP" is for the "grouping by tty" patch (which is useless on android: no ttys and no one does "make -j 64" on his phone xD ). I just posted the complete output of the "grep".

EDIT: About the possibility that it's a kernel issue ... bfq for example uses cgroups.
[Desire Android Development] melethron:
Well the error isnt that much a problem about stability. Cgroups aren't nescassary needed.

I worry more about performance and battery drain because of this this.

For infos what about cgroups see this:

http://www.mjmwired.net/kernel/Documentation/cgroups.txt

The relevant part for understanding is this:

Edit: Well ill give a short explanation:

103	As an example of a scenario (originally proposed by vatsa[AT]in.ibm[DOT]com)
104	that can benefit from multiple hierarchies, consider a large
105	university server with various users - students, professors, system
106	tasks etc. The resource planning for this server could be along the
107	following lines:
108
109	CPU : Top cpuset
110	/ \
111	CPUSet1 CPUSet2
112	| |
113	(Profs) (Students)
114
115	In addition (system tasks) are attached to topcpuset (so
116	that they can run anywhere) with a limit of 20%
117
118	Memory : Professors (50%), students (30%), system (20%)
119
120	Disk : Prof (50%), students (30%), system (20%)
121
122	Network : WWW browsing (20%), Network File System (60%), others (20%)
123	/ \
124	Prof (15%) students (5%)

on android there are groups for "fore ground" tasks and "background" tasks. This is to assure that the app you use has more priority (like the profs in above example) like a background sync or something like that. Or if you run something in background that would use 100% cpu and open the browser you are still assured to get a minimum of 10% cpu power so the app you use doesnt lag.

This is a good idea in general. But it can be done different. The focus of BFS is to speed up interactive things more nativly directly from the scheduler:

source:

So the focus of BFS were slower systems (single core/dual core) and to have a native build in priority for interactivity. Its not intended for a big server but for "daily use" on a normal computer with "no monster hardware".
[Desire Android Development] woti23:
didn't mean autogroups.

there is an option (completly fair) group_sched with cfs, but not when bfs is enabled. (option disapears). my theory is that this option disabled when bfs has something to do with it. and the patches. and SLQB (when i enable this and cgroups and bfs, compiling breaks)

atm i try to compile bfs enabled (without havs, without bfq, without slqb, with vdd_levels, with control groups). will see if error messages still there.

edit: what i'm trying to say is, it seems like BFS is saying:

there is no room for other optimations if i'm enabled, i'm enough, get out of my way.

that would also be the reason that with bfs everything is patched again and again (look at the various linux distributions and the fora)
[Desire Android Development] melethron:
Not sure if you will be lucky on this:

It still has some features so give it a try ;)
[Desire Android Development] melethron:
Ok its definitive the kernel:

I asked ownhere and he also uses BFS on his own Kernel:

edit: @woti did you make the config completly new or did you setup menuconfig with an old config?
[Desire Android Development] woti23:
as i thought, neophytes bfs kernel (filesystems test kernel) also had cgroups support, and no error messages (only missed the vdd_levels). was .26 and i'm sure he had no ck patches too.

i use config from my cfs kernel (based on coutts) for almost everything than schedulers and groups and filesystems.

all filesystems included (ext + reiser in kernel, other modules)
[Desire Android Development] woti23:
well, kernel compiled, but now again the other variant of error messages

E/dalvikvm( 947): Cannot get policy, owner thread id=966

E/dalvikvm( 947): Cannot get policy, waiter thread id=1011

E/dalvikvm( 947): Failed to find cpu subsys
[Desire Android Development] woti23:
there is btw a hint to cgroup in

/data/anr/traces.txt

"java.lang.ProcessManager" daemon prio=5 tid=9 NATIVE
| group="main" sCount=1 dsCount=0 s=N obj=0x47a74bc0 self=0x2ad1f0
| sysTid=8240 nice=0 sched=0/0 cgrp=bg_non_interactive handle=2486424
at java.lang.ProcessManager.watchChildren(Native Method)
at java.lang.ProcessManager$1.run(ProcessManager.java:87)
[Desire Android Development] martino2k6:
@melethron

There is one request that I would have, if it is feasible. Since you're doing data2sd, wouldn't it make more sense to set setInstallLocation to default to the "internal" memory? Thanks.
[Desire Android Development] paulove:
[QUOTE]=melethron;10551088]Now that log is really strange:

I//system/xbin/busybox( 64): noop [deadline] cfq bfq

I//system/xbin/busybox( 64): +++ Changing scheduler for mtd:

I//system/xbin/busybox( 64): noop [deadline] cfq bfq

I//system/xbin/busybox( 64): +++ Checking external Partition:

D/AndroidRuntime( 123):

D/AndroidRuntime( 123): >>>>>>>>>>>>>> AndroidRuntime START <<<<<<<<<<<<<<

The script just "stopped" somehow. But i dont get it why. Even if there is an issue with the partitions there should be at least the " +++ Avaritia HD init script completed" message. Only explanation would be that busybox crashed but i think there should be an exit message from logwrapper then.... Also i wonder how busybox could crash on this. Do you use the kernel that comes with the rom? If so try that one:

http://mirror.couttstech.com/android/bravo/kernel/CFS/couttstech-2.6.32.28_OC1190_CFS_BFQ_SLQB.zip

Coutts made a really stable voltage table for that one and the next version will either include this or the BFS version (without the ck2 patches).[/QUOTE]

I had the same result with the new kernel.

paulove
[Desire Android Development] melethron:
Can you check if in neophytes test kernel: CONFIG_CGROUP_CPUACCT is set?

I git cloned coutts source and the option isn't even available (completly not available - not "not set")

Also about the test kernel. No way to set vdd_levels since its an HAVS kernel. That would need rewrite of the "avs.c". Also this kernel is an older version of coutts. I think there went something wrong patching bfs ....

By the way there were a complaint that the bfs kernel is not as smooth as it was before. Watch logcat and scroll the app drawer. Big error blog=>tiny lag.

Where is it located?
[Desire Android Development] coutts99:
This option is not available when BFS is enabled. It is available when BFS is disabled. I suspect it's the lack of this option that causes the error messages on the BFS kernels.
[Desire Android Development] coutts99:
Right the JFS/BTRFS test kernels now have simple IO support. The CFS one also has CGROUPS set correctly now.
[Desire Android Development] Terepin:
But SIO needs to be used by ROM, right? Or it is set as default?
[Desire Android Development] coutts99:
Do this to set sio as schedular

echo "sio" > /sys/block/mmcblk0/queue/scheduler

echo "sio" > /sys/block/mtdblock0/queue/scheduler

echo "sio" > /sys/block/mtdblock1/queue/scheduler

echo "sio" > /sys/block/mtdblock2/queue/scheduler

echo "sio" > /sys/block/mtdblock3/queue/scheduler

echo "sio" > /sys/block/mtdblock4/queue/scheduler

echo "sio" > /sys/block/mtdblock5/queue/scheduler
[Desire Android Development] Terepin:
ADB, right? Out of my league I'm afraid.
[Desire Android Development] coutts99:
Just do adb shell

then copy and paste this in

echo "sio" > /sys/block/mmcblk0/queue/scheduler
echo "sio" > /sys/block/mtdblock0/queue/scheduler
echo "sio" > /sys/block/mtdblock1/queue/scheduler
echo "sio" > /sys/block/mtdblock2/queue/scheduler
echo "sio" > /sys/block/mtdblock3/queue/scheduler
echo "sio" > /sys/block/mtdblock4/queue/scheduler
echo "sio" > /sys/block/mtdblock5/queue/scheduler
[Desire Android Development] melethron:
I wouldn't recommend to use it only for testing. Its a rather quick work and relative untested (according to the def of it). If i implement it in the rom i make it optional.

Thought its releated also, but as ownhere said he doesn't have this messages on his own BFS kernel.
[Desire Android Development] coutts99:
Ill download his rom and see, all bfs kernels should generate those errors on hd roms. I think that we should stick to cfs on hd roms.
[Desire Android Development] Alex-V:
Thanks for this info :) :)

With kind regards..Alex
[Desire Android Development] martino2k6:
Not sure to be honest, by default it is set to auto, but from the terminal it can be changed using the following command using su permissions (IIRC);

pm setInstallLocation value

where value can be 0 for auto, 1 for internal, and 2 for external.
[Desire Android Development] paulove:
@melethron

I have got this rom to work on my phone, the odd thing is that i had to use my original 4GB card not my 16GB card. I used the same size for the reserfs and swap.

:confused::confused::confused:

paulove
[Desire Android Development] coutts99:
I given up trying to download it, multiupload sucks