yashsingh1/xdaforums huggingface dataset:
[Galaxy S I9000 Android Development] RyanZA:
sleep 10 actually seems to work perfectly every time.

Anyway, I've been playing around with bind mounts, and I think that will work much better.

Basic idea:

unmount /sdcard, /data

hdparm -z /dev/block/mmcblk0 to cause the kernel to reload

sleep 10, or a script to check if p3 is available

and now we make /data/data2 and /data/data3

we mount the EXT2 onto /data/data2

and we re-mount RFS onto /data/data3 (can re-mount /data as much as we want)

And then we can check if /data/data2 is empty. If its empty, we copy everything from /data/data3 to /data/data2, and then sync and reboot

If it isn't empty, we bind-mount /data/data2 onto /data/data

/data/data is now completely replaced by EXT2! :) If any system processes needed anything from /data/data while we were doing the initial boot (such as GPS), they will be able to grab their socket files from RFS, which should be just fine.

Actual apps load up later, and they will be pulling data off of EXT2 since it will be up by then.

In the event that anything goes wrong, RFS /data is still happily sitting there, and the device will simply use the old RFS if anything goes wrong. A script check could also be added to the boot up to check if we've tried to boot before and fail, and disable the EXT2 lag fix.

Done right, this method should be 100% safe in every circumstance, and be as fast as possible too, all without any custom kernel, and will work on every I9000 variant with an internal SD card. It also allows the user as much app storage as he wants, like a crazy 5GB /data. :)

The space on RFS /data can be used as a perfect and complete backup mechanism too, which can be toggled to at any time with a bit of scripting.

I'll try put this together when I have time. Any help with the scripting or anything else is welcome. :)
[Galaxy S I9000 Android Development] RyanZA:
After the reboot your changes will be lost. Maybe you're selecting the wrong values? Does the resize fail to run at all? If so, you probably haven't unmounted /sdcard yet! :)
[Galaxy S I9000 Android Development] staticduo:
Thanks a lot,...

Running on JPC
[Galaxy S I9000 Android Development] anilkuj:
i did unmount sdcard

it ran fine , i just get a message saying t Warning kernel ..... i won't see changes until i reboot

i ran resize 1 32.8kB 13GB

that went fine except for that msg

next when i run mkpart and try

i get option fro primary/extended

i can't enter logical , there i entered primary

next i entered 13GB for start and 14GB for end

thats when i get the overlapping error
[Galaxy S I9000 Android Development] hardcore:
You've gotta put in 13.4GB start and 14.4GB end.

This is my modified MBR. You can see it with the "print" command.

print

Model: MMC MAG8DD (sd/mmc)

Disk /dev/block/mmcblk0: 16.4GB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Number Start End Size Type File system Flags

1 32.8kB 13.4GB 13.4GB primary fat32 lba

3 13.4GB 14.4GB 1028MB primary

2 14.4GB 16.4GB 2013MB primary lba
[Galaxy S I9000 Android Development] hardcore:
WARNING! You might lose all the data in Internal SD if you do this! BACKUP FIRST!

I think I lost all the data in my Internal SD. Luckily I did back up the important stuff.
[Galaxy S I9000 Android Development] hardcore:
@RyanZA: It doesn't work for me on JPC on a 16G SGS.

I got the new MBR copied over but then the kernel won't re-read it, and it fails after that. I can't see the 3rd partition in the /dev/block/.

I also lost the data from the Internal SD when the partition was resized. I noticed during the partition resize that it had to move some data - that could be risky.

Right now I've somehow messed it up enough that I've lost the su function. Going to reflash JPC first.
[Galaxy S I9000 Android Development] RyanZA:
If you get the "can't unmount" problem, you need to rename your fsck_msdos file to .bak like the first post says. Alternatively, I think the fsck process could be killed and then restarted later. I'll play around with it when I get time. :D

As far as losing data from the internal SD... you really shouldn't be if you do the resize command. parted FAT32 resize is pretty stable, and I've never lost files using it before. Maybe you didn't have enough free space, or you did something to cause that. I'm not sure. If you have any other ideas about why parted fat32 resize would fail...? :)
[Galaxy S I9000 Android Development] RyanZA:
You got it to run? Awesome. :)

So at least it does work for someone besides me!

I'm going to be playing around more with trying to kill and restart the fat32 fsck instead of just renaming it, plus I'm going to be doing more testing with bind mounts and lazy loading for EXT2 on top of RFS.

Once I've got it working nicely on my phone, I'll start work on an app.

Maybe I'll try steal supercurios robot voice idea too...? ;)

EDIT: Oh, for a status report: I ran this fix as in the first post for the whole day today, and didn't notice any issues. Everything seemed to be working fine, so the lag fix in the first post does actually seem to work moderately well already. :)
[Galaxy S I9000 Android Development] hardcore:
I did rename the fsck_msdos file, but still couldn't unmount it. Maybe its a Froyo problem?

I think the Internal SD got corrupted because it was using the default MBR (can't unmount and kernel didn't re-read the new MBR) with the resized partition and somehow it must have gotten corrupted.
[Galaxy S I9000 Android Development] RyanZA:
Not a Froyo problem, unless you're not using JPC? I'm gonna give it a try on the new Froyo soonish. :)
[Galaxy S I9000 Android Development] hardcore:
Yeah holy cow, JPH is out! I think I'll try that first.

Another thought was - the complicated part of this is the whole partition-resizing and tricking the kernel into accepting a new MBR. Maybe another safer approach would be to just use an EXT2 partition on the External SD (like mimocan's fix, but without the custom kernel).

But I'm sure with enough experimenting, you'll be able to make it pretty much fullproof for the Internal SD. ;)
[Galaxy S I9000 Android Development] moonstruxx:
Hi ryan! I will try it with jph and report back asap.i could also help with some shell scripting.

Cheers
[Galaxy S I9000 Android Development] RyanZA:
External SD is just plain a lot slower, and then you can't pull the thing out. I much prefer the internal SD myself. As far as the complicated part, that will all be handled by an app in the future, so no need to stress. ;)

The complicated part is getting the kernel to accept the MBR though. I have it working fine on my phone, but if it will work on other peoples is up in the air right now. I believe it should be possible with a bit of kill -9 and whatever else to get it to accept it always though. :) Although, then the complicated part becomes starting everything back up after! But since we have root, it's far from impossible.
[Galaxy S I9000 Android Development] mcbooth:
RyanZA - Will this work on other Galaxy variants, such as the Captivate??

If not, what changes would be required to your directions?

Great work BTW
[Galaxy S I9000 Android Development] etking:
Sounds promising, but I would like to see an ext4 version. With Onclicklagfix my SGS currently always takes more than 10 minutes to boot (just showing the S-logo), I guess because of the filesystem check, which is fixing errors. I use quick reboot widgets quite often.
[Galaxy S I9000 Android Development] Val3r10:
Awesome, RyanZA

As far I understand, the 2gb rfs partition will stay completely unused, then

So, a question rises soon: as we're dealing with repartitioning and rewriting MBR from scratch, what's the problem in reducing the rfs (for instance, shrinking on its right... without any data move) and then appending the EXT2 at the end of internal memory, instead than the middle ?

Let's say, for example

6gb: unchanged Fat32 /sdcard

1gb or less: RFS unused

1gb or more: EXT2 lagfix

I do understand the need of space when undoing the lagfix for movin out the installed apps, so this example is more or less similar to your loopback OCLF, but this choice could be left to everyone, leaving for example a 500mb RFS and 1.5 EXT2... with the prescription to remove most of the apps before uninstalling anything.

For the upcoming froyo, let's say.

Does it make any sense ?

Thank you
[Galaxy S I9000 Android Development] RyanZA:
The kernel writes to /data a bit before we can catch it, so it would probably end up in corruption. :(

I'm having trouble getting this to go in the new JPH unfortunately. The kernel init script has changed a bit, and it seems to be breaking it. Maybe I can get it going with some work... but fuser -km /data is killing the processes, but they start back up too quickly for umount to grab /data away from em! Must be a way.. :)
[Galaxy S I9000 Android Development] RyanZA:
So hopefully someone who knows the workings of the Linux kernel will pass by and see this:

fuser -m /data

returns nothing, showing that nothing is using /data

umount /data

fails, because it is already in use. I can't find out what is using it though! This is specific to JPH kernel. :(

Does anybody know of any tools or similar which can ask the kernel what's up, and why it doesn't want to let go if nothing is using it?
[Galaxy S I9000 Android Development] Benson17:
maybe you could try df -f for infos about mountpoints or dmesg to get kernel messages - but don't know if they're all working under android...
[Galaxy S I9000 Android Development] RyanZA:
df -f doesn't work

dmesg works, but doesn't return anything relevant...
[Galaxy S I9000 Android Development] imagoX86:
hmmm so thats why I cannot get it to work on jph...keep getting device/resource busy(been trying for a last 2h)....unfortunately my linux knowledge is very limited so I cannot help...anyway I do appreciate your work...
[Galaxy S I9000 Android Development] hardcore:
I had the same problem with JPC too. Something using it. Maybe the vold service?
[Galaxy S I9000 Android Development] Bierfreund:
I'm on JPH since yesterday and wanted to give this a try.

At step 5 I get the following error:

# busybox mkfs.ext2 -b 4096 -m 0 /dev/block/mmcblk0p3
busybox mkfs.ext2 -b 4096 -m 0 /dev/block/mmcblk0p3
mkfs.ext2: applet not found

Any ideas what's wrong?

Can I start again from the beginning?
[Galaxy S I9000 Android Development] imagoX86:
version of busybox which U R using does not have mkfs.ext2(you need busybox 1.17) but this lag fix is not working on JPH at the moment.
[Galaxy S I9000 Android Development] Bierfreund:
Sure it's not working with JPH? RyanZA wrote that it should work with JPC Froyo, so I thought it should work with JPH, too.

I tried with busybox that came with the root update and afterwards with busybox that you can download with Titanium Backup.

Edit: copied required busybox to the system and tried again. But still does not work:

# busybox mkfs.ext2 -b 4096 -m 0 /dev/block/mmcblk0p3
busybox mkfs.ext2 -b 4096 -m 0 /dev/block/mmcblk0p3
mkfs.ext2: can't open '/dev/block/mmcblk0p3': No such file or directory

I attached /dbdata/log as log.txt to this thread, maybe it helps. Or is this because of JHP.
[Galaxy S I9000 Android Development] RyanZA:
I haven't been able to get this working on JPH yet. :(

Something changed in the init which stops /data from being unmounted. I don't really know why - there doesn't seem to be any process holding onto it. I probably just need the correct tool which can find out what is holding onto /data, and then it should be a simple matter of killing it and restart it after.

This is on hold for a bit though, since I'm a bit stuck in how to do that. I'm gonna upgrade the OCLF app with a V2 lagfix so long though, which will use bind mounts instead of symlinks, so that the correct space will be reported by Android.
[Galaxy S I9000 Android Development] hardcore:
@RyanZA: I've managed to get a modified version of this working with an EXT2 partition on the External SD card, thanks to your original OCLF and these new tools.

The parted-static makes it easy to repartition the External SD card from the phone itself.

Will post up a procedure later when I've tested it for a while.

Unfortunately I couldn't solve the Internal MBR reloading stuff as well, so decided this was the safer route. Got 1682 in Quadrant using JPH firmware, with an 8GB Sandisk Ultra II Class 6 MicroSD.
[Galaxy S I9000 Android Development] hardcore:
Also trying several tweaks in the userinit.sh startup script, like changing the lowmemorykiller settings in both the minfree file and the props.

I tried the deadline io scheduler and it was faster than the default cfq. I tried noop but that didn't seem as 'smooth'.

Then I noticed that Samsung disabled the low_latency mode in the cfq scheduler. So now I'm trying cfq with low_lantecy enabled. It seems fast too.

P.S. RyanZA, based on the playlogos1 hack, could the vold service be interfering with your script? How about a similar 'hack' to run the userinit.sh script from vold instead of playlogos1 - would that work?
[Galaxy S I9000 Android Development] Bierfreund:
For someone with Unix/Linux knowledge it should not be a huge problem to find out which process is locking the /data mount?
[Galaxy S I9000 Android Development] ed10000:
Tried the new APK version and didn't find it to be that much better than the old OCLF fixes. Voodoo was quite smooth but I had battery issues. Guess I have no choice but to try this out now.

How do I uninstall the fix, though? That is, partition it back to the way it was before, replace the MBR with the stock one, etc.? On Voodoo, flashing new firmwares without disabling the lagfix first proved to be problematic, so I'm assuming it's going to be the same with this? Hence I just need to know how I can revert everything to stock before I go ahead and experiment with this.

Thanks :)
[Galaxy S I9000 Android Development] RyanZA:
I played around with the low_latency myself, and didn't notice much difference. Samsung probably thought it would help out RFS somehow, and it does seem to help a bit. For EXT2 though, the difference seems tiny.

Vold instead of playlogos sounds like an awesome idea! :D I'll give it a try when I have time! Thanks! That is, if it is the vold system that is causing the lock.. I don't think it is, as vold doesn't touch /data as far as I know.

It is probably GPS or something that is grabbing onto it!

You'd think so... but I can't find it! Any help here is appreciated. :)

The real pro's (like cyanogen, etc) at this kind of modding are keeping very quiet for some reason. I'm hoping they're putting together something good though! :) So that leaves people who are mostly just learning about Linux kernels as they go along, like me. I've had very little help with these lagfixes in general though. :(

The OCLF2 is very similar to OCLF1, it has fairly small performance increases, and is a bit safer. Mostly came out of the work on this with bind mounts, which work a lot nicer than loopback or symlinks.

This fix works on 2.1 firmwares, but is pretty messy if you use the script from the first post. Probably not recommended for daily use yet, although it does work alright. I used it for a day on JM8 without many issues.

I'll clean it up into something nice once I've worked out why it isn't working on Froyo, though. Or I might just clean it up for 2.1 only, and release it like that. :)
[Galaxy S I9000 Android Development] RyanZA:
I'm actually really liking this idea - but I think maybe taking it a step further is even better. At bootup, the init.rc starts up a number of services in /system/bin/ - I think we could replace every one of these binaries with scripts that would check for the existance of a file, and not run until it appears.

This means we could actually block the whole system from starting until a special file is created. Then we'd be able to do whatever we want in peace, and start up the system whenever we want! :)

Maybe even make a custom bootup animation to show that the lagfix is busy working, and then play the normal SGS one when we'r ready to boot up the system. :D

Can anybody think of any reason why replacing all the binaries in /system/bin/ with scripts would be bad?
[Galaxy S I9000 Android Development] hardcore:
One danger is that these binaries are considered 'services' I think, and they may be run again some time *after* startup. So it might be bad if we assume they only run once at startup.
[Galaxy S I9000 Android Development] RyanZA:
If we make a script like this:

1. Does /file exist?

2. YES: launch real binary in /system/bin/backup

3. NO: Sleep for 5 seconds, then go to 1

So even if it gets called again later, it should still run the backup binary right away. :)

Basically a small wrapper around the binaries so we can control the start time of the services. And if we put /file on a ramdisk, it will be removed with each boot. :)
[Galaxy S I9000 Android Development] hardcore:
@RyanZA: I've been running a 'custom' lagfix the last 2 days and its really nice and fast. Basically the same idea of using a real ext2 partition, but re-partitioning the external SD card instead. This way we don't have the problems of trying to trick the kernel into re-loading the MBR on the internal SD.

I've basically used a similar procedure as your OCLF 2-3, except for all the loopbacks.

It's fast, works well, and I have no problems using both Internal and External SD in USB Storage mode with the PC. Quadrant about 1700. Using it on JPH.

The most difficult part of the procedure is really to repartition the External SD card - not sure if that can be made scriptable for a "one click" operation. Gparted worked well resizing the original FAT32 partition though - no data lost.
[Galaxy S I9000 Android Development] RyanZA:
It relies a lot on the speed of your external SD card. It has to be high quality. Plus, if you pull out the external SD card, the phone will freeze. It also may use more battery, since the external sd card is always 'on' and can't go to 'sleep'. To be honest though, I have no idea if external SD cards can even sleep, so this might not be an issue. :D

Parted should be fully scriptable, so a one click solution for the external SD card is possible. Would make a nice option, anyway. :D
[Galaxy S I9000 Android Development] moonstruxx:
Are there any mounts on the partition? That could be a reason for the business of the device..i'm still on jph atm but I'll flash back to jm8 later..

Edit: did you try 'find /data -type d -exec fuser -m {} \;' and the same with -type f
[Galaxy S I9000 Android Development] miki4242:
Just throwing this out here, it would be nice if someday something like Upstart could be ported to Android (Fedora and Ubuntu people will know what I mean, others: please google).

The init service management on Android seems to bear more than a passing resemblance to Upstart already.
[Galaxy S I9000 Android Development] Finguz:
Excellent work guys, this looks promising :)
[Galaxy S I9000 Android Development] hardcore:
Great work! :)
[Galaxy S I9000 Android Development] woeds:
Great stuff Ryan! Any chance on loading up the ext4 modules on boot?
[Galaxy S I9000 Android Development] RyanZA:
I need someone to hook me up with a EXT4 module for the 2.2 samsung kernel. :D I don't believe the EXT4 module for the 2.1 samsung kernel will work.
[Galaxy S I9000 Android Development] Phlogiston:
Did you try using lsof? Maybe that tool does show who is using it/blocking from umount...
[Galaxy S I9000 Android Development] Phlogiston:
No way to use the old module.. The kernel version did change. Do we have the samsung kernel sources of JPH already? Where can I get them?
[Galaxy S I9000 Android Development] RyanZA:
I don't have an lsof binary. Do you have a statically compiled one? It isn't an issue any more at least, since just replacing every startup service gets around the problem much nicer. :)

Is there no way to compile a module without the rest of the source? If so, then I guess EXT4 won't be possible for awhile. Not that I mind, I've had no problems running EXT2 for a few months now. *shrug*
[Galaxy S I9000 Android Development] gshklover:
Don't think you need the sources from samsung - very little chance that they changed smth significant enough. My guess you could compile the ext4 modules from stock sources of the same kernel version.
[Galaxy S I9000 Android Development] RyanZA:
Nice. :D If someone can build the module, I can have EXT4 support added in the future. :)
[Galaxy S I9000 Android Development] woeds:
That'd be awesome, especially if this entire procedure could be run from an Android-app :) would make it the bestest lagfix evar :D
[Galaxy S I9000 Android Development] Phlogiston:
No I don't have it, but I think it should be easy to compile it for busybox. I won't call that way nice :o It's a very dirty hack, but yes I guess it works.

The benefits from using EXT4 are minor anyway I guess. But no, afaik its not possible you need the sources or at least the relevant parts of it, e.g. config.

Did you ever think of hacking on a "full" rom? Or do you have had any contact with the cm guys working on it/ making progress?

I like your work/hacks but I start thinking that we might better put the energy into a full rom running on extX anyway ;)
[Galaxy S I9000 Android Development] Phlogiston:
Actually true... stock android kernel 2.6.32.9 ext4 module should work.

PS: I'm cloning ;)
[Galaxy S I9000 Android Development] Bierfreund:
Huh, sounds nice! Could you write an short instruction like in post 1 including the new method if the the whole process is finished, sothat i can try it on my JPH?
[Galaxy S I9000 Android Development] ed10000:
This seems to be going very well. I can't wait! :) Good luck guys!
[Galaxy S I9000 Android Development] Phlogiston:
The compiled ext4 module does not load. I might not have used a sane .config for the kernel... anyone knows how to get a default config that works? A config that is as similar as possible to the kernel running on JPH.

PS: I can't find 2.6.32.9 on android.kernel.org. I just tried with .32.
[Galaxy S I9000 Android Development] RyanZA:
I don't think the .9 should matter much, since it should just be bug fixes. Unless those bug fixes were for EXT4 compatibility or something! :D

There will be an app. ;)

I need to work out how I'm going to do the repartitioning though. I don't think resizing /sdcard will work too well while the system is running, but it might.

Backporting all of the samsung drivers to work in the stock Android sources would probably be a fairly monumental achievement! I don't think I'm capable of it. :D If anybody else wants to try, I will be happy to help em in any way possible. But I don't think I have the knowledge to do it myself from scratch.

It would be awesome though, and the device would be a lot lot better. In particular, the ram situation would be way better. ;)
[Galaxy S I9000 Android Development] FadeFx:
drakaz did it on the i7500, they are already running froyo... ;-) however, they he with 2.1 because samsung refused to give a working 1.6 rom and now they have a complete aosp build. took them a while, but it works...

ah, and i have to say we many more expirienced devs here than the i7500 comunity ever had. there never was that lot of rom flavours skins and kernels arround for i7500.
[Galaxy S I9000 Android Development] RyanZA:
It just takes one person who knows what they're doing to get it started. :D If drakaz could give us pointers on what exactly we'd need to do to port aosp across, then I'd give it a try. But I don't even know where/how to start. ;)

EDIT: Plus, I think we'd need a custom bootloader then? I don't think we have that working properly yet.
[Galaxy S I9000 Android Development] OrionTC:
you could maybe ask the vibrant guy(eugene373) who ported ASOP, altho he didnt get it working 100% yet. should be the same principles for the i9000 i would think?

i have no idea on the scale of porting something like this.
[Galaxy S I9000 Android Development] RyanZA:
Not just the same principles.. almost identical code. Once he has his done, it should be simple to change the drivers slightly for the I9000.
[Galaxy S I9000 Android Development] Phlogiston:
They will release the sources one day when official froyo is out, so... But i think cm and its crew is working on it anyway.