yashsingh1/xdaforums huggingface dataset:
[G1 Android Development] student.driver:
My understanding is as follows:

disklimit (32MB in my case) establishes what the peak of data will be stored in RAM that is uncompressed. So, since ours is 32MB, you can store only 32MB of uncompressed data in RAM, even if it compresses to 1MB in reality.

memlimit (18MB in my case) establishes the peak of data that will be stored even when compressed or otherwise. So, if you have 30MB of data (less than disklimit) that compressed to 16MB, you're within both limits. However, if you get 2MB of data that isn't compressed, you will hit the memlimit setting (assuming no backing swap) and you will get less than 100% good compress.

backing swap (32MB partition in my case) is where all non-compressed stuff goes, and eventually may be used in case the memlimit peak is hit. So, if you have BS enabled and not peaked, you will have all the non-compressible data paged there and you will have 100% good compress up to memlimit. BS will be cleared if the commands are given (either the app is flushed, or individual pages, via the "discard" callback from the kernel).

So I thought, like Mike, that as long as BS is enabled you will get 100% good compress since all those uncompressible pages are thrown there. So, you have the 32MB uncompressed limit, plus the backing swap. I am still trying to work out all the RAM space that's available to the kernel.
[G1 Android Development] dwang:
this is straight from the man page..

Specify backing swap device. This can either be a swap block device or a swap file. Writes are forwarded to this device when memory limit is reached or when a page is incompressible. You must swapoff backing swap device before init is issued, otherwise init will fail with -EBUSY error. Also, backing swap must be a valid swap device, otherwise 'swapon /dev/ramzswapX' will fail as if swapon was issued directly on the backing swap device.

http://compcache.googlecode.com/hg/sub-projects/rzscontrol/man/rzscontrol.html

So basically if you use backing swap and have a 32 meg partition, the kernel will think you have a 32 meg swap. Compcache will internally control what pages gets put in the compressed ram and what pages gets put on the disk swap, but the limit for uncompressed data is 32 megs.
[G1 Android Development] miketaylor00:
Ok, I guess that makes sense. But you would still get more out of it than you would of 32MB of Linux Swap because some of it is compressed. That's why I wanted to know the compression rate so I could figure out how much Linux Swap it would be.

When I was doing my testing I didn't try increasing the disk size above 32MB. When I get some time I will see if bumping it up will help at all. Thanks for the info.

It seems weird that it would work that way. Since the compressed is in RAM and the uncompressed is linux swap. I'm getting some ideas about how to get a better config. Now I have a new challenge :)
[G1 Android Development] dwang:
I'm not sure if compcache compresses the data before it puts its on the swap actually, if the memlimit is reached.

Thinking logically, it wouldn't make sense for compcache to compress the overflow data as that just adds extra overhead on top of the I/O overhead.
[G1 Android Development] student.driver:
Yeah, I mean it sounds like ultimately (in my case) I am sacrificing up to 18MB of RAM and 32MB of SD card space to house 32MB of uncompressed data. The real difference in performance is CC determines what gets to stay in physical RAM and what goes to physical disk, even though the kernel thinks it has 32MB of disk space to treat as swap.

So, the kernel makes the call to swap something out of RAM to its "disk", our RAM drive controlled by CC. CC tries to compress it, and if it can it stays in RAM, and if it can't it goes to swap space. If it hits memlimit but is beneath disk limit, it will continue to swap pages (even if they could have been compressed) until disk limit is reached. Correct?
[G1 Android Development] miketaylor00:
Nope, it says on that page that it only compresses up to the memlimit. Then it moves to the swap file. It would not perform well on a hdd to compress it and write it to the disk.
[G1 Android Development] dwang:
I think it all depends on how compressible the data is. I tried using backing_swap on a 64meg swap_partition with the default memlimit of 15% of RAM which comes out to be 14 megs.

This means that the kernel sees the total swap size as 64 megs (of which 14 megs is RAM).

compcache was able to take 45 megs of uncompressed data and compress it down to about 12 megs, which means it never had to go to swap except for data that couldn't be compressed.

I could never get my uncompressed data to go over the 45 megs, no matter what I ran.
[G1 Android Development] student.driver:
Now that I think about it, it must be virtually impossible to come up with a single combination of this to suit even most people. Until we know what apps the kernel would want to page out, and if they have uncompressible pages, this would be all over the place. Currently, the config I'm using is fantastic, but I would like to know what apps I use that I want to keep "available" (not have to be reopened or refreshed in a slow, and painful way) and see if being paged out to the SD is helping or hurting its performance (ie, would have just been better to reopen/refresh the app).
[G1 Android Development] miketaylor00:
There's no way to really pin it down exactly, but like I said in the other thread it doesn't really need to be exact. Most of the apps on these phones don't really use a lot of data. And you don't really need to keep everything that's in the browser. The 32MB swap plus or minus 10% will probably work for 99% of the users out there. Since memory works better in multiples of 8 it's best to stick with that. People who use these huge swap files are just causing themselves more problems down the road. They don't get the delay from having to clean up the swap until it gets really out of control and then they see a huge slow down. I've done a lot of tuning with Java on application servers. It's the same concept with garbage collection and heap size. Bigger isn't always better. It's about finding the right balance for the hardware and what you are using it for.

I spent a day coming up with the config that I have been using. It worked good so I gave it to other people on xROM. I thought I would take another day and make it better but so far I haven't been able to. I've spent a lot of hours tweaking and I can't really get any more significant improvements. I might be able to now that I understand the compcache/backing swap ratio better. But it will mostly be about having less reloading and less slow down after time. With something like this were usage varies by user and even by time of day you can't be too precise.
[G1 Android Development] miketaylor00:
Has anyone done any comparison between linux swap partition and linux swap file? I've read that linux swap file on hard drives is less efficient than partition because the blocks are contiguous on a swap partition and swap file isn't. So you get fragmentation that will slow things down. I'm not sure that will be as much of a problem with the phones, considering the fact that almost everyone is using less than 100MB of swap instead of being the GB+ range for swap. The OS should be able to handle the smaller swap file without much trouble. So far I haven't been able to get backing swap setup with a file to test it out. Has anyone else done any testing with swap file on the phone?
[G1 Android Development] Vlad83:
So I am curious, what setting should I run? You guys seem like you know what you are talking about :)
[G1 Android Development] miketaylor00:
Read student.driver's signature ;)
[G1 Android Development] student.driver:
Yeah, what Mike wrote. :)
[G1 Android Development] Vlad83:
Thanks guys, just updated the userinit. :)

Edit: After running sh /system/sd/userinit.sh -s I am getting the default vaules of:

Compcache enabled

CompCache: DiskSize 32768(system) 32768(user)

CompCache: cc_swappiness - 30(system) 30(user)

with everything else disabled. Is it supposed to be like that or did I screw something up?
[G1 Android Development] student.driver:
Did you put his user.conf in there as well? Did you chmod it to 755?

chmod 755 /system/sd/user.conf

chmod 755 /system/sd/userinit.sh

It appears you are using the swapiness setting from the userinit.sh (or the stock user.conf that it came with) and not using Mike's user.conf, which has a swapiness of 28.
[G1 Android Development] miketaylor00:
I agree. That should fix it. Do the following to make sure you have the right user.conf

cat /system/bin/user.conf

If it isn't the right one, copy it over agian.

You know, if you asked me if it would make a difference to have swapiness of 28 instead of 30 I would have told you no way. Most of the changes I made in my conf seem minor but they all made an improvement. Sorry for going off on a tangent. :)
[G1 Android Development] RAQemUP:
For us Cynogen Mod users, the command is:

cat /system/sd/user.conf

I think Mike keeps mentioning the /system/bin folder because that's the folder for a different mod than Cynogen's?
[G1 Android Development] miketaylor00:
Oops, my bad. You are right. xROM uses /system/bin so I do that out of habit. Plus it just seems to make more sense to me to have these files in the bin folder. I'll try to post both paths from now on.
[G1 Android Development] dwang:
I wanted to mention that with the recent tweaks to compcache by cyanogen, it really makes the case for only using compcache without the need for backing swap or a swap partition.

The fix the cyanogen made to compcache was that unused or stale memory pages are now freed dynamically in real time from the compcache swap in RAM. Whereas before those stale pages stayed in RAM for quite a long time before being freed.

This means that compcache only uses the amount of RAM it needs, thus allowing more RAM to be used by the kernel.

From the compcache wiki:

"Support for swap free notify callback. This will give ramzswap a callback as soon as a swap page becomes free -- stale data, no more! Apply patch found in compcache/patches/ directory and just compile the kernel as usual -- currently, patch is against 2.6.31-rc6 but it should apply to slightly older kernels too. This will enable 'swap free notify' feature. This allows kernel to send callback to ramzswap as soon as a swap slot becomes free. So, we can immediately free memory allocated for this page, eliminating any stale data in (compressed) memory. "

Prior to 4.0.3, swap free notify wasn't working correctly.
[G1 Android Development] gunigugu:
I'm a bit behind the 8ball in the cache area... but is there any advantage to using customized userinit files? Because I thought Cyanogen has compact turned on now by default on cleans roms?
[G1 Android Development] nyijedi:
FWIW, I was running a backup swap on Cyanogen for a while and decided to get rid of it to compare performance. One thing I noticed right away is that w/o a backing swap enabled (or I guess any swap), when I'm multitasking, I frequently will have to wait 30 seconds or so on a blank homescreen while the icons are loading. I don't think this happened a single time when I had my swap enabled. In fact, I think I'll set it back up now.
[G1 Android Development] dwang:
I never see this myself. Are you using the stock launcher?
[G1 Android Development] nyijedi:
Yes, I'm using the stock launcher. This type of behavior was very common when running the stock roms, and on cyanogen before I enabled a swap file. As soon as I disabled my swap file, it started up again. It doesn't happen every time I go to the home screen, but it does happen when I'm doing heavy multitasking, and it especially happens when I'm using the browser a lot. I just re-enabled my swap and I'll see if it happens anymore (I doubt it will).
[G1 Android Development] Vlad83:
Thanks miketaylor and studentdriver. Phones is more responsive now. I was getting low 15s on the Benchmark Pi program and now getting low 12s...
[G1 Android Development] miketaylor00:
That's about what you should expect. I'm gald it helped.
[G1 Android Development] miketaylor00:
This is really common if you don't have backing swap on. That is the reason I always recommened that people use backing swap.
[G1 Android Development] nyijedi:
I agree with you 100%. I enabled backing swap again this morning and I was multitasking like crazy today w/o a single hiccup. Thanks for your user.conf, btw - It's absolutely great.
[G1 Android Development] AsaSpades:
I Have a 64mb Linux Swap Partition, and I had Mikes script running, and I decided to disable the Compcache, and just run Linux Swap with 30 swappiness, I was surprised it was alot more responsive. But Im gonna play around with the compcache and backing swap w/o the linux swap more and find what works best for me.
[G1 Android Development] miketaylor00:
You're welcome. I'm working on getting it working with swap file. Hopefully I'll have that soon. Also, I'm working on one for Hero. I don't know if I can make the same improvements with Hero that I did with Cupcake but we'll see.
[G1 Android Development] miketaylor00:
My guess is that it will seem more responsive at first but will slow down after a little usage. If not, let me know. If someone can come up with a better config than what I'm using I am all for that.
[G1 Android Development] AsaSpades:
I had it running for about 3 days, stayed pretty consistent, ill keep testing tho
[G1 Android Development] dwang:
I have a feeling its because of the swappiness value. I use a swappiness of 60 and I never use backing_swap and I never have problems with the launcher getting pushed out of memory.

I played around with using a lower value, but that always cause my browser to refresh more often than I'd like and that's really the main reason I use compcache or swap is to keep the browser in memory.

Whenever I use backing_swap, I always get more lag especially when playing music in the background or using imeem than without backing_swap.

Seems like lots of people are having success with using Mike's config, which is great. It seems to boil down to what type of applications people run.

The settings I use are optimized to keep the browser in memory while having other apps loaded. I tend not to run too many apps in the background. I only have a weather widget and a twitter app running every hour.
[G1 Android Development] dwang:
another thing I wanted to mention is that using a backing_swap gives you different default parameters for the amount of RAM compcache uses.

If you enable compcache by default, it only allocates a 24 meg compcache swap, which translates to about 10 megs of actual RAM used.

If you enable compcache with backing_swap, it sets the limit of RAM used to be 14 megs, which translates to about a 32-40 meg swap size. And if you use Mike's config its set to 18megs which translates to an even higher swap size.

This could be another reason why people are seeing more reloads without backing_swap than with.

If you want to really test compcache without backing_swap, I would configure the disksize parameter to be higher than the default of 24 megs.
[G1 Android Development] miketaylor00:
Good point. Although having 18 MB instead of 14MB is going to make a difference. I've noticed the reloads even with the 18MB of CC though. Again, it's all about finding that sweet spot. I found if I go over 18MB I start to get slow downs. When swap_free_notify gets implemented it is going to improve things a lot. I can't wait to test that. It may even drop the memlimit for CC a little bit which could help too.

Oh and on the compression, I have found that compcache usually compresses data to around 20%-25% of it's normal size. I need to do some more testing with different types of data to make sure but that is what I have seen so far. That translates to 72MB or more. I'm currently running JACxHEROski trying to come up with a better config for that. I think when swap_free_notify gets implemented we will see a big improvement for using CC with Hero.
[G1 Android Development] yukkio:
Hi. In summary, what is the best setting? CC+BS. CC only or linux swap?

Thanks.

BR.
[G1 Android Development] cal3thousand:
Summary is: every one has different needs based on app usage. You should read the thread. There's lots of good knowledge to be gained.

There's no certified champion yet. Why don't you come up with a good configuration and share it? miketaylor00 has a pretty solid one for the average user. Try that one and go from there.
[G1 Android Development] dwang:
Are you talking about the hero roms, because swap_free_notify is most definitely implemented in cyanogen's ROM with version 4.03 or higher. In fact, I was the one that notified cyanogen that it wasn't working in his previous versions and he fixed it.
[G1 Android Development] miketaylor00:
Yes, I was talking about Hero ROMs. I think Cyanogen is the only one who has implemented it in a ROM. It has to be manually compiled into the kernel code from why Cyrowski told me.
[G1 Android Development] ERamseth:
Saw this in the changelog for the custom userinit.sh thread:

Does that mean using a file for swap is closer?

(I'd be happy to help test, by the way. I don't feel like re-re-re-partitioning my SD card when I have the whim to use swap or no swap or compcache or no compcache or backingstore or no backingstore...)
[G1 Android Development] miketaylor00:
It was supposed to work in that build. However, there was a bug with Compcache that would not allow it to work on our phones. I opened an issue with the Compcache project and they provided me with a fix yesterday. JAC is going to help me get it compiled and I will work on testing it.
[G1 Android Development] miketaylor00:
I've done testing on both linux swap and compcache. Using Benchmark tools I have never been able to get as good of performance using linux swap as I have with compcache with backing swap. I've tried, I have no personal bias either way.

There have been dozens of people who have emailed me or posted in threads that my configuration made their phone noticabley faster. 1 person said it made no difference, although it turned out he didn't have a userinit.sh file so it wasn't doing anything. 3 people said it was only slightly better but they also made their own changes to it so they weren't really using my config. If you use my config, don't change anything. I have tweaked every setting in there. If I change any of the settings the numbers get worse on the benchmarking tool. And 2 people said that Linux swap was faster, including the guy on this thread. The 2 that said their config was faster were basing it on their personal perception of how it "feels", as far as I know. No one has used any benchmarking to show a faster config. If you just want to base it off of peoples perceptions, well over 90% of the people saw a noticable improvement in performance.

BTW, I would LOVE for someone to come up with a better configuration than I have. If you think you have one, benchmark it. If you don't want to do that send it to me and I will test it. I try tweaking my config all the time and I can't get a significant improvement. When I get swap free notify and swap file working on a build I will work on it again because that stuff might change the config.
[G1 Android Development] prince.siraj:
Oh ok well I've used my 256mb swap partition with the miketaylor userinit file at 30 swappiness and I'm not sure but it might be faster than compcache... Care to give it a shot?I have a class 2 SanDisk SD... Better than nothing but I'm sure class 6 would eliminate all lag...

I had cyanogen's original compcache before that and my swap was faster than that for me... Why 256MB? Idk really
[G1 Android Development] miketaylor00:
256MB? Holy cow. I guess if it is big enough it will never fill up? I have a list of things to test but I will put that one on the list.
[G1 Android Development] dreamhunterr:
loved your user config @ CM 4.04. I really noticed a speed improvement.

Cheers,
[G1 Android Development] miketaylor00:
Thanks for the feedback.

You have swap free notify enabled if you are on that version so it should work even better than what I am getting. Swap free notify allows Compcache to clean up memory that isn't being used anymore. So you don't have stale memory sitting there taking up space. It should be a lot more efficient that way.
[G1 Android Development] worldestroyer:
Which benchmark app are you using?
[G1 Android Development] miketaylor00:
Benchmarking Tools

I use 2, Benchmark Pi and Benchmark. Benchmark Pi will screw up your auto-rotate if you aren't careful. Don't open your keyboard or rotate to landscape while you are using it. If it does break I have a script that makes it a little easier to fix it. I got tired of messing with it so much.
[G1 Android Development] dumfuq:
Just to through another loop in...

I've been running cm4.04 with compcache w/backing swap of 32. I noticed that the memory used for compcache is fairly minimal using usually no more than 5mB even with lots of web pages and multiple apps. I decided to try lowering the memlimit to 8192kB(8mB) for comcache and am currently testing it. So far compcache is still behaving as expected and I have still yet to see it have a need to use the backing swap.
[G1 Android Development] miketaylor00:
CM 4.0.4 has a new feature in compcache that gets rid of stale data instead of having it just sit in memory. Its called swap free notify. It should increase the efficiency of memory use quite a bit. I found a really cool article that talks about Compcache and how it works. I don't have it now but I will post it tomorrow. Most of my testing has been on xROM so iwhen JAC gets the new compcache modules implemented in xROM I will be tuning that some more. I think compcache still has a lot of maturing to do but it has a lot of potential.
[G1 Android Development] MagicDude4Eva:
do you mind posting your user.conf / userinit.sh you use on CM 4.0.4?
[G1 Android Development] meistertom:
dude, have a look on miketaylor00's signature
[G1 Android Development] miketaylor00:
It's ok, I changed my signature after he said that. I email the script to 3 or 4 people every day so I thought I would just add it to my signature along with a couple other minor fixes I have helped people with.
[G1 Android Development] MagicDude4Eva:
thanks for adding links to your sig - appreciated. Wouldn't it be better to have a Cyanogen / xROM specific thread where people can post their experiences with the configs?
[G1 Android Development] miketaylor00:
I don't know. I never felt that me putting up my config was important enough to deserve it's own thread. I was just trying to help out. It might be now, there are getting to be a lot of people who are using my config.
[G1 Android Development] MagicDude4Eva:
I think your work deserves a new (sticky) thread to get feedback from people. I applied your user.conf on Cyanogen 4.0.4 and free reports now 28MB free (from previously 8). Also applications seem to start faster.
[G1 Android Development] miketaylor00:
I just started a thread for it. You and a few other people convinced me. Feel free to post your feedback there too.

I had another guy message me tonight who was fed up with his phone and was going to drop apps2sd and everything else. He said he was using my settings and things got worse when he went CM to 4.0.4. I helped him out. He re-setup his phone from scratch, we partitioned it differently to give him more room since he only has a 1GB Class 2 card. Helped him out with a few apps. I told him to try it out with the stock settings. Then try mine. If it doesn't get better move it back. I also told him to install Benchmark Pi to measure it. So he tried it both ways. He said my settings were faster. His Benchmark Pi score went from I think 14.5 seconds to 12.5 seconds. Which is about what I saw on CM 4.0.4 yesterday when I was on it.