yashsingh1/xdaforums huggingface dataset:
[G1 Android Development] overpower:
Compcache .6.x

Supposedly the latest build of .6.x is working with some stats corruption from the built in module.

If someone can compile it against the cyanogen's kernel and post it then we can probably set the memlimit option.

Anyone have it? I haven't been able to figure out how to cross compile on a Mac, but if someone points me the right direction I can try to do it.
[G1 Android Development] Binary100100:
uwonsom pointed out something to me in your userinit.sh file you have this code:

Shouldn't that be uname_r?
[G1 Android Development] namsir:
i'm running cyanogen 3.9.2 and i did use the userinit.sh and it didn't work when i do the #free command......swap is 0 and the cat /proc/ramzswap didn't work either ....can anyone help?
[G1 Android Development] uwonsum:
Yes the script has a little issue. I am debugging it now. I will post the fix soon as am done testing...

edit: Fixed/update

-Resolved Linux-swap not turning on

-VM configuration works with Linux-swap

-Compcache works with any kernel version

-Ensure backing_swap is not always the default

Added:

-Minimum CPU scaling frequency

-CPU sampling rate (You should not touch this unless you know why it was added and what it means)

-Power bias

The fix/update is now posted. I take no credit for this script.

adb push userinit.sh /system/sd

adb push user.conf /system/sd

user.conf usage

To use only linux-swap

-set compcache=0

-set linux_swap_eanable=1

To use only compcache without backing_swap

-set compcache=1

-set linux_swap_eanable=0

-set cc_disksize=32 (32*1024 = 32768kb = 32MB)

-set cc_memlimit=0

To use compcache with backing_swap

-set compcache=1

-set linux_swap_eanable=0

-set cc_disksize=0 (doesn't really matter so long cc_memlimit != 0)

-set cc_memlimit=8 (doesn't matter the number used, it will default to 15% of total RAM--known issue)

These are default system settings:

page-cluster=3

laptop_mode=0

dirty_expire_centisecs=3000

dirty_writeback_centisecs=500

dirty_background_ratio=5

dirty_ratio=10

swappiness=60

sampling_rate=2000000 (200000 depending on kernel version)

scaling_min_freq=245760

powersave_bias=0

Update:

I re-uploaded the file due to clarification regarding Cyanogen's comments here that the sampling_rate should be configured to the power of 6 instead of 5 which some previous ROMs were using. There is no need to re-download. This has the same configurations as the previous.

Run these command to see your current settings and system info:

free
top -m 8; displays only 8 processes
cat /proc/swaps
cat /proc/ramzswap0
cat /proc/meminfo
sysctl -a | grep vm
cat /sys/devices/system/cpu/cpu0/cpufreq/stats/time_in_state
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
cat /sys/devices/system/cpu/cpu0/cpufreq/ondemand/sampling_rate
vmstat -d 5 -n 10
[G1 Android Development] uansari1:
What post is the updated script in? I'm having the same problem with the swap coming up as 0 when I run the #free in Terminal Emulator...

I had pushed the userinit.sh to /system/sd, and have created the three partitions on my SD card.
[G1 Android Development] uwonsum:
I just posted it, try and see if it works.
[G1 Android Development] uansari1:
Dumb question... both files go to /system/sd?

Edit: It seems to have worked... Thanks!
[G1 Android Development] Janis:
I did want some, got some!

I know this is bad form for ROMs, but I've noticed this thread is much shorter and no one's posted this, exactly, yet:

This worked for me on Cyanogen's 3.9.2

I was stumped, but I think this will allow me to run the apps I want to run with a near-normal number of fc/w clicks and some livable, actually very decent speed.

Just curious, what's wrong with clocking down to 128 when idle?

Thanks for the hard work!
[G1 Android Development] namsir:
this update gonna work on cm 3.9.2 ?
[G1 Android Development] namsir:
cool...got it working :D thanks alot mate
[G1 Android Development] uwonsum:
Question should be what is great about clocking down when idle?!

It allows the CPU to run at lower speed obviously which means better battery life. If you don't need the power, why use it...
[G1 Android Development] yukkio:
Hi mate, I've downloaded your script updated. I've pushed to /system/sd with 755 permissions. I have to change any value from user.conf to use with cm-3.9.2 or actual values are ok?

Great job man.

Thank you.

BR.
[G1 Android Development] uwonsum:
I cannot take credit for the script, I just added a couple things to it. Here is original version. You don't have to change anything for it to work, but you can play with the user.conf file to see what works best for you.
[G1 Android Development] yukkio:
Ok. I've seen the value of the variable laptop_mode is 0. Is it better 0 than 5?

Thanks.

/sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq shows 245000 instead 128000 when display is off. Is it normal?

BR.
[G1 Android Development] huanyu:
I have updated the script and start a new post http://xdaforums.com/showthread.php?p=4240913#post4240913

if there are anything you guys wanna to add.. just le me know. I am happy to do it.
[G1 Android Development] ThR1LL:
Sorry for the noobness, but since there's a new update to compcache and a new way to enable it, how do we edit how much space it takes and the swappiness?
[G1 Android Development] AustinKnight45:
c:\media /system/sd worked for me i have all my ringtones an do have look at me sound twice in notifications but that's it

o and dream them twice in ringtones
[G1 Android Development] scrappyabs2:
Repeat steps for after each flash?

Is it necessary to repeat these procedures every time you flash a new build?
[G1 Android Development] glock30:
If you want to skip on messing with your media files, you can use the instructions in the earlier compcache thread. After a little homework I found other threads that have info on starting the auto swap by editing the a2sd script.
[G1 Android Development] nurre:
You'd be wrong, it uses 25% of memory by default, you can modify it on insmod.

# insmod ramzswap.ko disksize_kb=90000 for example.

I'd also recomend toying with swappiness, compcache would want very different swapiness compared to what you'd use with a slow block device.

//edit, I also dont see why you put the media information in there, as it has nothing to do with the topic of getting compcache working and could be a completely seperate script.

//edit, also worth mentioning that busybox'es swapon doesn't include priority, so having swap + compcache will use swap at the same time as using compcache, more than likely slowing you down. You need a different insmod or swapon command to use the swap as backup, and I can't remember the syntax off hand.
[G1 Android Development] nicholasbithell:
Does anyone happen to know the terminal commands for this?:confused:
[G1 Android Development] nurre:
They are in the very simple script, this is what I use atm:

#!/system/bin/sh

insmod /system/modules/lib/modules/2.6.29-cm/compcache/xvmalloc.ko;
insmod /system/modules/lib/modules/2.6.29-cm/compcache/ramzswap.ko disksize_kb=32000;
mknod /dev/ramzswap0 b 253 0;
echo 100 > /proc/sys/vm/swappiness;
swapon /dev/ramzswap0;

exit;
[G1 Android Development] nicholasbithell:
I just dont understand how to go about getting this installed! ahhh I don't have ADB as I cannot get it working on my mac so I need console or terminal commands if that is at all possible! is any one working on a "ramzswap" app?
[G1 Android Development] nurre:
Look in the first post.

adb push = copy file to someplace on the filesystem.

adb shell = a shell command

All you need to do is put the userinit.sh on your /system/sd (that's a extX partition), although you'd need to modify it (for example use mine) if you wanna skip the media part.
[G1 Android Development] nurre:
Oh, and another thing people dont seem to understand.

24mb compcache != 24mb swap.

Very very far from it. 24mb compcache uses 24mb memory to give you 24mb of compressed memory. On my phone I'm seeing 3:1 compression ratio atleast.

meaning 24mb compcache = ( 24mb * 3) - 24mb (the 24 mb memory it uses to give you this) = 48mb "free" memory.

Please note that using a 90mb compcache size might seem like a good idea based on this it isn't. Linux considers compcache swap, it will pull data out of swap and into active memory before using it. If it could use it directly it might be pretty awsome for this usage.
[G1 Android Development] innerspace:
@nurre

I think you may be right about busybox, not educated enough to be sure but:

This is my compcache info:

cat /proc/ramzswap

DiskSize: 24468 kB

NumReads: 11379

NumWrites: 11753

FailedReads: 0

FailedWrites: 0

InvalidIO: 0

PagesDiscard: 0

ZeroPages: 270

GoodCompress: 71 %

NoCompress: 7 %

PagesStored: 5846

PagesUsed: 2226

OrigDataSize: 23384 kB

ComprDataSize: 8464 kB

MemUsedTotal: 8904 kB

and yes, approx 3/1 compression ratio

and at the same time:

free

total used free shared buffers

Mem: 97876 96540 1336 0 284

Swap: 24464 21232 3232

Total: 122340 117772 4568

so I'm unsure what this means, is swap running before or after compcache, or are they running side by side? I'd think if compcache is knocking 24MB of data down to about *MB that I'd still have another 16MB of available ramzswap available, and that my linux-swap wouldn't be active.

Is this correct?
[G1 Android Development] nurre:
free will report wrong data for compcache, if you wanna know how much is going to a swapfile/compcache you can check /proc/swaps
[G1 Android Development] innerspace:
ok... so then...

so:

cat /proc/swaps

Filename Type Size Used Priority

/dev/ramzswap0 partition 24464 24436 -1

is basically saying only compcache is active, and I'm assuming if the following is correct:

cat /proc/ramzswap

DiskSize: 24468 kB

NumReads: 15556

NumWrites: 14136

FailedReads: 0

FailedWrites: 0

InvalidIO: 0

PagesDiscard: 0

ZeroPages: 290

GoodCompress: 70 %

NoCompress: 7 %

PagesStored: 5826

PagesUsed: 2274

OrigDataSize: 23304 kB

ComprDataSize: 8714 kB

MemUsedTotal: 9096 kB

#

The system has acquired the entire 24MB of compcache, compressing 23304kB of data down to 8714kB and using a total of 9096 kB of the 24Mb ramzswap file. And android isn't ever activating the backup swapfile at all, or at least not under an idle state.
[G1 Android Development] nurre:
Correct, your swap partition is not enabled, and no it's not deactivated on idle.
[G1 Android Development] innerspace:
and then that leads me to think...

that if the compcache ever got to the point where

cat /proc/ramzswap

DiskSize: 24468 kB

OrigDataSize: ~70000 kB

ComprDataSize: ~24000 kB

MemUsedTotal: ~24000+ kB

then android would start writing to the linux-swap partition (which I have configured to 24MB) until compcache no longer needed it.

correct?
[G1 Android Development] Mikey:
After seeing like 5 scripts for this, which one is yours??

I keep my media on fat32 partition :)

EDIT: I'm guessing this script?

#!/system/bin/sh

insmod /system/modules/lib/modules/2.6.29-cm/compcache/xvmalloc.ko;
insmod /system/modules/lib/modules/2.6.29-cm/compcache/ramzswap.ko disksize_kb=32000;
mknod /dev/ramzswap0 b 253 0;
echo 100 > /proc/sys/vm/swappiness;
swapon /dev/ramzswap0;

exit;

with these commands:

Rename it to userinit.sh
create a directory c:\userinit
copy userinit.sh to c:\userinit
open CMD prompt
adb push c:\userinit\userinit.sh /system/sd
adb shell chmod 755 /system/sd/userinit.sh (not sure if needed being safe)
adb shell reboot
[G1 Android Development] jubeh:
what i wonder, though, is where do i put userinit.sh if i don't use apps2sd or a swap partition or any of the sort (i like being able to hot swap my sd card)

I'm guessing /etc, am i right?
[G1 Android Development] Saiboogu:
I've wondered this too, since I have another rooted G1 in the house (non A2SD). I believe this would work --

I haven't tested this, just a theory

su
mount -o remount,rw /dev/block/mtdblock3 /system
cd /system
mkdir sd
copy /sdcard/userinit.sh sd/userinit.sh
chmod 755 sd/userinit.sh

And since you don't have extra partitions on your card, want to get rid of all the other junk in the standard userinit script floating around.. so just try this:

#!/system/bin/sh

insmod /system/modules/lib/modules/2.6.29-cm/compcache/xvmalloc.ko;
insmod /system/modules/lib/modules/2.6.29-cm/compcache/ramzswap.ko disksize_kb=32000;
mknod /dev/ramzswap0 b 253 0;
echo 100 > /proc/sys/vm/swappiness;
swapon /dev/ramzswap0;

exit;
[G1 Android Development] bkmo:
I edited the post and replaced the script with the one you are quoting. The script will not do anything to audio files now, you will need to have them on your card.
[G1 Android Development] bkmo:
I was thinking of this myself while dozing of this AM. "What if you don't want aps2sd" because if you add the partition, the aps2sd script will move your apps there. This looks like it will work just fine for aps2sd. I also modified the post and userinit.sh script removing any of the move audio stuff.

Edit: the only con i can think of in doing it this way is that you will have to redo it after every update. System gets formatted during an update.
[G1 Android Development] MortICi:
Im not sure if you want to use 100 for the swapiness...

But ill test it out.

This works on jacHEROski as well 1.4C, just have to copy two of the modules over to the system.
[G1 Android Development] mgear356:
can i ask

swapper vs linuxswapfile vs compcache..

whats the difference? which is faster?

i am now running linuxswapfile.. 24mb on 3.6.5..

upgrading to 3.6.7.1 + compcache, is it gonna make my system fly?
[G1 Android Development] MortICi:
With swappiness at 100, the swap fills up really quick, either the disksize_kb needs to get increased (most likely to 96mb) or the swapiness needs to be reduced to 80 or maybe even 60.

But it is stupid fast.

Even when you open the keyboard on the home screen of hero, to reload it, it takes about 3 to 5 seconds if not maybe even less.

Hell even the music app works normal without lag!
[G1 Android Development] runderekrun:
Color me impressed.

I followed the instructions and man application switching is sooo much faster now.

Bravo to all involved.
[G1 Android Development] TheMan42:
Can I use this if my EXT3 is at 1GB? and also, what are the main benefits of this as I never used the linux swap partition before.
[G1 Android Development] richpadg:
Ok, just ran through the tutorial. Now how do I know I did this right and that it is working?
[G1 Android Development] Stretch3000:
can anybody direct me to a thread that has a tutorial on getting adb set up? It doesn't recognize my phone.
[G1 Android Development] ivanmmj:
Does it make JacHEROski any faster?

Does it solve the issue with the SMS program not receiving due to it being closed because of the lack of RAM?
[G1 Android Development] mdavidf:
You don't do ADB on the phone...it's from a computer...

Here's the link to setting it up on your computer:

http://xdaforums.com/showthread.php?t=502010
[G1 Android Development] mdavidf:
From a Terminal app on the phone -

cat /proc/ramzswap

If you're seeing GoodCompress in the resulting list, you're golden.
[G1 Android Development] mdavidf:
I got caught up last night in the mix when this tutorial was being developed and I ended up pushing my audio files to /system/sd. Now, since the audio portion of the userinit.sh file has been removed from the current file -

Do the audio files need to remain in /system/sd? If not, what's the proper (linux) way to remove them?

If the audio files should remain, how do we get rid of double listings in our ringtones, notifications, etc.? Do we delete the folders from our SD Card?
[G1 Android Development] richpadg:
It says 76%. Is this what I am looking for?
[G1 Android Development] bkmo:
Good to hear....I will set swappiness to 60, less aggresive should be fine for most people. I will re-up the script to the first post.
[G1 Android Development] mdavidf:
Yes, mine is 78%. If you want details of what it means, Innerspace asked questions and received responses on page 4 of this forum. :) Basically it means the original size of the data is compressed by 76%.

I noticed some lag after awhile when mine was set to 100; so, I changed mine to 60 and so far so good.
[G1 Android Development] bmk789:
from what ive read, performance is even better with compcache, can anyone moving from swapper to compcache say how different it was or if its worth the switch?
[G1 Android Development] Nedlinin:
Performance seems to be about the same (if not a tad faster) -- however, switching does allow you to not use your SDcard as swap, reclaiming a bit of filespace and not wearing your card down with extra read/writes.

So, worth the switch I think
[G1 Android Development] bkmo:
You can edit the script if you want the audio file section back in. You can also remove the files from the extx partition and put them back on the fat32. If you use the same dir structure on the fat32 supposedly you will not see them in the music app. To delete them just do adb shell rm -r /system/sd/media

In either case, make sure they are in only one of the locations, fat32 or ext partitions.
[G1 Android Development] phantasm714:
sexy time

I can confirm. This is sexy time.
[G1 Android Development] bkmo:
For those people that are unable to use adb, then what needs to be done is make sure the userinit.sh script gets copied to the root of the extX partition. This can be done in the terminal if you place the script in the root of your fat32 partition.

in terminal emulator:

su

mount -o remount,rw /dev/block/mtdblock3 /system

cp /sdcard/userinit.sh /system/sd

chmod 755 /system/sd/userinit.sh

not tested but should work.
[G1 Android Development] alapapa:
wowsers this is much faster than my swap partition setup (non class 6).

do like

EDIT: the speed difference is most obvious when listening to music, etc when reading files stored on the SD card (b/c it's not doing IO over the same channel for swap)
[G1 Android Development] nurre:
Yeah I use 32mb disksize and skip all the media stuff.
[G1 Android Development] nurre:
It filling up isn't a bad thing, it's just prioritize using it highly. Chances are it's gonna be used regardless of your swapiness simply due to the limited amount of memory (and android not killing off apps until it actually runs out of memory).

swapper = swapfile, compcache is compressing it in memory, writing to memory (including when using the cpu to compress) is stupidly much faster than writing to a block device.
[G1 Android Development] nurre:
You can put it wherever, including your sdcard, or just run the commands in the script.

The only difference is it wont stick over reboots.
[G1 Android Development] rigamrts:
can someone tell me the two modules i have to transfer to get this to work on jac
[G1 Android Development] pascanu:
I did some testing and 60 looks like the upper limit. Anything higher and I start feeling some lag.

FYI: once you have the userinit.sh on /sd you can play with swappiness in terminal by typing:

echo 60 > /proc/sys/vm/swappiness

(replace 60 with whatever value you want tot test).

Play with your phone for a while to see how it runs and once you found the right value for you edit the userinit.sh file to make settings permanent (that is they will be retained even after a reboot).
[G1 Android Development] maxisma:
sorry for the idiotical question, but do you have to enable swap (via swapper) to activate compcache or is compache a standalone?
[G1 Android Development] phantasm714:
the userinit.sh enables swap with the 'swapon' command. in short, this script and compcache are all u need.
[G1 Android Development] rigamrts:
/system/modules/lib/modules/2.6.29-cm/compcache/xvmalloc.ko

system/modules/lib/modules/2.6.29-cm/compcache/ramzswap.ko

/system/bin/shinsmod

if this is what is needed to get this to work none of it exists in 3.6.7.1
[G1 Android Development] scrappyabs2:
linux-swap partition?

I'm wondering what to do with my linux-swap partition. It's no longer needed, right? In that case, should I leave it alone or remove it? Are their performance benefits of removing it entirely vs. leaving it idle?
[G1 Android Development] dwang:
jachero's kernel is not based off cyanogen's so these modules won't work. You need to ask him compile the compcache modules for his ROM.

I think jacheroski's rom is based on cyanogen's so it should work.
[G1 Android Development] bkmo:
Now do you think people would be reporting success if they did not exist? They are on my phone. Have you installed the rom?
[G1 Android Development] bkmo:
It is just an empty partition at this point. Merge it with the partition to the left if you want the disk space back. Or leave it there for a rainy day when you are bored.
[G1 Android Development] sim64one:
Very good!!!

no comment! ;)
[G1 Android Development] warburto:
Thanks for posting the script. I got it working and played around with it. A thread in the main Cyano thread refers to an increase in battery drain, which is important to me. I'd like to revert to "stock" settings for cyano i.e. no swap in order to test this out. What's the best way to do that? Right now the only thing I can think of is flashing 3.6.7.1 again.

I would be interested to know if I could push a blank userinit.sh (just guessing wildly, have no idea) and avoid flashing.
[G1 Android Development] rigamrts:
i went to the dir in the zip itself and they are not there
[G1 Android Development] here.david:
I see this was added:

disksize_kb=32000

to the:

insmod /system/modules/lib/modules/2.6.29-cm/compcache/ramzswap.ko;

two things:

1. is it needed

2. if ones swap partition is 24mb should it be changed to disksize_kb=24000

Here is mine without it

#cat /proc/ramzswap

DiskSize:	24468 kB

NumReads:	41691

NumWrites:	23361

FailedReads:	0

FailedWrites:	0

InvalidIO:	0

PagesDiscard:	0

ZeroPages:	266

GoodCompress:	72 %

NoCompress:	4 %

PagesStored:	5850

PagesUsed:	2298

OrigDataSize:	23400 kB

ComprDataSize:	8409 kB

MemUsedTotal:	9192 kB

# free

total used free shared buffers

Mem: 97876 93828 4048 0 360

Swap: 24464 14504 9960

Total: 122340 108332 14008

Tx's...:)
[G1 Android Development] spinjason:
compcache

I used the terminal and everything went smooth. Thank you for the T.E. commands, I do everything from my phone. Everything is gravy!
[G1 Android Development] glock30:
You may want to look through this thread as well

Quoted from there:
[G1 Android Development] Jak Crow:
I have a question that no one appears to have asked. If doing apps2sd, what about using the unused space on the phone's memory for the swap file? I would imagine it would be faster than using the sd card. I'm using the extended space SPL, so I have 89MB of internal mem that goes unused since I'm doing apps2sd.
[G1 Android Development] spinjason:
userinit.sh

Do I have to keep the userinit.sh file on my sd card or can I delete after the prosses is done.
[G1 Android Development] mdavidf:
You can remove it from your SD Card (Fat32). If you want to keep CompCache, it needs to remain in the /system/sd.

EDIT: Original response was poorly worded, sorry.
[G1 Android Development] Nedlinin:
Basically, you don't want to write to that memory constantly like a swap would.. otherwise, you will kill your memory lifespan and, in essence, your phones lifespan

Yes you need to keep it. This file is run each boot and without it you will lose compcache after next boot.

EDIT: As someone else pointed out, as long as you've moved this file from your Fat32 partition (/sdcard) to your ext3 one (/system/sd) then you can remove it from the Fat32 one.. - if you haven't done the copy/paste or move command to get it into /system/sd, compcache isnt actually setup yet
[G1 Android Development] jroid:
anyone workin to get this working on Jac hero??
[G1 Android Development] Jak Crow:
Is 300 - 350MB okay for the ext3 partition for use with compcache on the sdcard instead of 500MB? I've often found performance is better when the app2sd part is smaller. Does the addition of compcache really need 500MB?
[G1 Android Development] glock30:
from what I understand all compcache is doing is using the userinit script to start up... so I don't think it needs to be 500MB... that is just a suggested starting point.