yashsingh1/xdaforums huggingface dataset:
[Desire Android Development] melethron:
Sibere why don't you tell people that the quadrant boost is FAKE and is simple abuse of the write cache instead of telling them risky mount options.

This "boost" is only a "boost" to have file corruption.....
[Desire Android Development] sibere:
Why should I say that that they'll have file corruption when I never had any since I started to use DATA2SD?

I feel that I have warned them enough on the OP.

They're free to use my script or yours.

If a user want the boost, they assume the risks.
[Desire Android Development] melethron:
Didn't say you should tell this .... i said you should tell people that the "boost" is not a "boost" .....

Edit: Lol just wanted to show you a "real" boost with tmpfs:

com.aurorasoftworks.quadrant.ui.standard -> /dev/quadrantcheat

LOL on bench start: "Tmpfs filesystem detected. The benchmark can not be executed".
[Desire Android Development] sibere:
Ha ok. I thought I read something about file corruption in your post ;) But yes the boost is just a Quadrant tweak using data cache.
[Desire Android Development] hajen78:
I have been running data2sd for some time now, and i can say that tit works perfektly, and my phone has never run this smooth ;)

Big thanks to iinga and sibere.
[Desire Android Development] bolek76:
I've trying in terminal and i've get ($ export PATH=/data/local/bin:$PATH

$su

# system/xbin/busybox mount -o loop,noatime,nodiratime,nosuid,nodev /mnt/asec/ext/ext4 /data

mount: mounting /dev/loop0 on /data failed: Device or resource busy

#
[Desire Android Development] melethron:
You call it tweak ... i call it cheat ....

Do this:

mkdir /dev/quadrantcheat
cp -a /data/data/com.aurorasoftworks.quadrant.ui.standard/*
busybox mount /dev/quadrantcheat /data/data/com.aurorasoftworks.quadrant.ui.standard/
busybox chown 10086:10086 com.aurorasoftworks.quadrant.ui.standard/

and youll get the same score as on loop device ...

A loop device is just one kind of method to let quadrant write to RAM instead of the sd-card/nand.

Quadrant checks if /data/data/com.aurorasoftworks.quadrant.ui.standard/

is symlinked to the tmpfs to prevent cheating. But it doesnt (or maybe cant bcs of permissions) check if you mount its directory in RAM or if there is writecache abuse due to a loop device....

EDIT: i have a better idea ... photoshop .... it tweaks quadrant to 1000000 points.
[Desire Android Development] bluelavender:
I personally dont care whether it is a cheat or a tweak, or whether it's 2-year old system or 30-year old system. For me, the phone is usable, no noticable lag, smooth, then it's enough.

I did a test last night. I ran a leedroid with data2sd and i yanked the battery out without turning off. When i put it back, the phone turned on normally without problem.. so data2sd pass my "oops" test ;)
[Desire Android Development] melethron:
Leedroid is smooth on normal ext4 partition without the use of loop device. I used it to make most test with data2sd.

Did you open all apps afterwards? Did you check lost+found ...

Change this line in siberes script:

/system/bin/e2fsck_recvy -y -v -f /mnt/asec/ext/ext2

to that:

EDIT (sorry doesn't work that way ... you need to copy to sdcard)

/system/bin/e2fsck_recvy -y -v -f /mnt/asec/ext/ext2 > /data/filecheck.txt;
mount /sdcard;
mv /data/filecheck.txt /sdcard/filecheck.txt;
umount /sdcard;

This way you get a logfile of the filesystem repair that is done every reboot. Take a look at it after every reboot. I did ... and i dont use loop anymore ...

If you want to use loop than do it with sync on ... if fsync() is skipped it is not a problem. I really dont mind what you do, but i can't stand it when people who know a lot like sibere fool people by telling them there is a "boost" without sync.
[Desire Android Development] sibere:
I'm not fooling them. They do get a boost in quadrant even if we know that it's a cheat. Please keep in mind again that the main purpose for me with data2sd was and still is storage space. Quadrant score is a side effect. I didn't want to loose speed when I made my little script, but was not intented at all to get the higher score at quadrant!

Now , if some people ask me how to get rid of sync at boot even though I don't recommend it, because they do it manually on every boot, it makes no difference if I give them the way to do so or not. They will do it anyway.
[Desire Android Development] sibere:
Honestly, I do a log of e2fsck on every boot ( and I use 2>&1 to get stdout and stderr) and it returns with no errors. I don't understand why you get so many errors like you say.
[Desire Android Development] Droidzone:
Alpharev S-OFF Process fails after DATA2SD is installed

@Sibere..

I dont know if anyone already documented this. If most of them were already S-OFF before adding DATA2SD, probably not.

Trying to install Alpharev after adding DATA2SD fails to S-OFF the Desire.

After Steps 1, 2 and 3 of Alpharev are done, it fails with an error message and gives the Linux shell. The message displayed is that Alpharev failed to push the necessary files to data. The device fails to boot into Android, and just displays Hboot after that. It is still not bricked, and Recovery is intact.

I'm assuming that's because DATA2SD mods the /data location.

To solve it, I Nand restored an old Stock Rooted backup from Clockwork after changing my sdcard, then redid the process from Alpharev. It then works fine. Going back to DATA2SD was as simple as switching cards and restoring a Nand which was done after the DATA2SD process.

I know it's not a bug in DATA2SD, but b/c of the way Alpharev handles the file pushing. Just thought you should know, so that any Alpharev devs you know can be made aware. And the solution to anyonw wanting to do S-OFF after DATA2SD can be as simple as doing what I did, or by doing the DATA2SD process after S-OFF.

And by the way..

It's the same reason I took Data2SD over Data2Ext. Storage is my main issue, rather than performance. And if a little bit of performance increases, wow..great. But it's a good sideeffect, rather than the objective. And I'd prefer this one over Data2Ext any day. It's really stable, no crashes or reboots, at all. If installation went ok, it will go butter smooth. Sometimes the initial process doesnt work out, b/c of some issues with the Rom or Sd-card. That's all.

Thanks again for the awesome code.. :)
[Desire Android Development] ownhere:
I found some interesting things:

App2SD by Android 2.2 Froyo use loop, too.

After install a app2sd app, enter 'losetup', we can see a new loop device been mounted.

So, Google use loop device to get POSIX attribute on VFAT, and may be speed up app which installed in slow SD card. Seems that 'loop' is not unsafe.
[Desire Android Development] sibere:
You must do the change in the 01data script in recovery mode. You obviously cannot mount your ext4 when android is running b/c it has already been mounted on boot :)
[Desire Android Development] sibere:
What I do have here is that froyo is creating a dm-* device in /dev/block/ with a major of 254 and a minor equal to the dm-* number (ie: 254,0 for dm-0, 254,1 for dm-1, etc)

This device is then used to mount the SD moved apk via /mnt/asec/

I dunno the nature of this dm-* block device but loop has a maj,min of 7,0 to 7,7.

Is dm-* a kind of loop device too ?

Strangely If I do a losetup I only have my loop0 used for mounting the ext2 file. Maybe it's a difference between HTC and AOSP rom.

edit: what happens if you install more than 256 apps to SD with froyo app2sd?! Appart from getting a long boot and a even longer shutdown, froyo may be out of dm-* available and won't let you install more apps on SD.
[Desire Android Development] sibere:
I guess it's because alpharev is installing something in /data (sd) when android is running but cannot find it when rebooting in recovery mode because it's the NAND /data which is mounted.

A workaround should be to disable the 01data script and reboot the phone. do the alpharev hack and when S-OFF is gained, enable the script again before rebooting.
[Desire Android Development] melethron:
Rofl. You cant compare that. The file is installed to sdcard/.android_secure and THEN mounted. And on update its unmounted -> installed -> mounted. This means THERE ARE NO WRITES AT ALL TO THE LOOP DEVICE....

Loop on /data/data almost get constant writes from apps.

ROFL i never said that READING from a loop device is dangerous. ^^

EDIT: Apart from that i wouldn't mind if stuff like apps is on loop device. The only way a corruption could occur is when the app is installed or updated and then the battery is pulled. This would be really rare cases compared to the writes of /data/data that occur far more often.
[Desire Android Development] sibere:
I must say that it's even more rare for me to pull my battery off my phone before the cache gets commited to the SD ;)
[Desire Android Development] ownhere:
Attachment is my screenshot.(DesireHD on Desire, move 2 app to SD):
[Desire Android Development] sibere:
OK I see. No loop for me. That's surely a Desire HD ROM thing compared to Desire.

How many loop devices do you have in /dev/block ? I have loop0 to loop7 so I wonder what happens when you have more than 7 apps on SD :)
[Desire Android Development] melethron:
What do you think where the "speed" of loopfile comes from? An app that is coded proberly will fsync the data. But where is it synced to .... the loopfile and not the fs. But when is the ext4 file + the meta-datacommited to the fs? Oh ... i forgot ... u use fat... no journal at all......then .... when is the ext4 file commited to the fat partition ... ?

Btw... i didnt get an "unclean" flag after shutdown on an ext4 partition mounted to /data.......
[Desire Android Development] sibere:
Stop beeing aggressive please.

We all understand that you are committed to get the highest level of data safety.

I know that loop device provides data cache.

I don't care about the journal because I use EXT2.

Why are you talking about committing the journal to FAT? Are you running mad?

Take a deep breath dude. If one uses ext4 over FAT using loop, the journal will be written to the EXT4 file. There is nothing about the FAT except that it's holding the ext2/3/4 file. Nothing is written to the FAT partition (it's full anyway)

My partition is ALWAYS unmounted nicely with the clean flag on. My e2fsck on boot reports ALWAYS a clean file system after each reboot.

If you want super safe data, by yourself a stone and a hammer.

You can have the best ever superhigh-supersafe file system, if your support dies, you will end up anyway with data lost. SD or SSD or HDD, nothing is 100% safe.

Even on my linux server , if I have a HDD crash or if the cache is not committed to the HDD and the power is lost, I will have a good chance of loosing data. But I don't use sync anyway on my server because it's for personal use and I still wanna have a bit of performance.

Phones are for personal use too and I'm sure that everybody understands that there is a little risk of data loss with DATA2SD.
[Desire Android Development] danarama:
Not really sure what the aim of all these posts are, but they are cluttering the thread a little.

Would you mind possibly restricting this "debate" to PM please as there are people here who follow this thread who may struggle to keep up.

Obviously if either of you come to an agreement / conclusion, please feel free to post it.
[Desire Android Development] sibere:
No problem with me dude :)

Users are free to choose whatever solution they like :)

I'm not pushing my solution trying to convince everybody that it's the best ever solution on earth because it's not.

It's just the one I personally choose for my own phone and I choose to share it with the xda community.
[Desire Android Development] ownhere:
After restored 200 apps, it is 15 or more app in app2sd mode. But I forgot check the loop devices at that time.:)

Yes, In my kernel, the loop device is limited to 7. but my phone have more app2sd app been installed. maybe the system only speed up front seven. hehe

EDIT:As you see, I losetup /dev/block/mmcblk0p2 to a loop device ,then mount the loop device to a directory. I found 'loop on device' provides same performance as 'loop on file', and maybe more safe(google search tell me the loop device have no async commit issue :)). Also, 'loop on device' can reduce 'dd' time of 'loop on file'. You should try it.
[Desire Android Development] sibere:
Ho I see! :)

When I install apps I always check that it has been installed to the phone because some of them are forcing the installation to the SD and I don't want that.
[Desire Android Development] ownhere:
As you see, I losetup /dev/block/mmcblk0p2 to a loop device ,then mount the loop device to a directory. I found 'loop on device' provides same performance as 'loop on file', and maybe more safe(google search tell me the loop device have no async commit issue ). Also, "loop on device" needn't "dd" time of 'loop on file'. You should try it.
[Desire Android Development] danarama:
Sorry I'm aiming it at the other dude really. I like your method and I am very happy with it, but would like to see progress made with the data2sd process without wading through posts of someone basically "Spamming" your thread.
[Desire Android Development] Droidzone:
As far as I know, DATA2SD remains the most stable way to improve my Internal Memory!

My only wish is that a method be evolved to restrict Nandroid backups to data that is really present on the 2nd FAT32 partition. Backups are now huge! :) If that's difficult by the method DATA2SD works, then no probs.. Everything comes with a cost, anyway! :)
[Desire Android Development] sibere:
I have updated here and there the OP to reflect issues and questions posted within this thread.

I haven't planned any "revolutionary" update though ;)
[Desire Android Development] sibere:
If you want to save some space, you can zip the sd-ext.img. You'll save a lot of space and if you want to restore, just unzip it before rebooting into recovery mode.

Even if your phone is unable to boot into android, you can always unzip the sd-ext.img using a card reader and a space PC :)
[Desire Android Development] sibere:
Hum yes never though about mount loop a device.

Good advice! thanks.

I'll try it on spare time.

did you try to see (or feel) the difference btwn the 2 methods?

What is best for you? loop + sync device or simply mount + noloop + nosync ?

b/c mount + sync kills performances :(
[Desire Android Development] Droidzone:
Cool! Now why didnt I think of that? :D

The sd-ext backup is just a lot of empty space anyway, rt?! :)

Thanks a lot..
[Desire Android Development] sibere:
Yep, a hell lot of 0 plus a few hundreds of Mb of real data :)

I just wish that recovery would provide on-the-fly zip/unzip. This would be heaven! :D

But I'm gonna try ownhere idea later just to see how the backup is done (full partition or only the actual data)
[Desire Android Development] Droidzone:
+1

I wondered the same. :)

Anyway zipping it up after reboot isnt tough. Will do that for the time being..
[Desire Android Development] danarama:
I just keep a couple of backups and archive the rest

If you use clockwork, you could delete system and some other backups. System doesn't change often. Sd ext us the important one if using this method imho
[Desire Android Development] Droidzone:
Good idea.
[Desire Android Development] dragonithe:
just a quick question, would this work whit MIUI?
[Desire Android Development] sibere:
Well, if you just want to save /data, you can always use tar to do backup. tar can even compress on the fly :)
[Desire Android Development] melethron:
Hey, first of all this wasn't intended to be in any way aggressive. I should have set some "irony" tags. If you felt offended im sorry for that. ;)

Now once more.... this time with irony tags ....

[ IRONY ]:

Yeah you are right. If the fat partition is full anyway and there is only the ext4 file there are no physical writes at all. I forgot that a loop device doesn't need any kind of physical hardware underneath and is written directly into the "matrix" we all live in xD.

[ /Irony]

Sorry i .... simply ... couldn't ... answer to THAT seriously ^^. Btw i forgot that you use a ext2 file on the fat partition. I thought of ext4 with journal as loopfile on a partition. I this case a commit to the journal isn't an actual write of data to the fat partition. But this also applies to any data writen on a loop-file:

data to loop file + fsync -> loop file .... But the loopfile isn't necessary written to sd because this would require a fsync of that file then. Thats what i meant. So you practically skip the "real" fsync.

Also about the "unclean" flag set on mount. I didnt get in on mount with ext4. Since i do some sqlite benches atm i'll take a look at this. Might be an issue of ext4 but im not sure....
[Desire Android Development] sibere:
Yes could be an ext4 thing because it's journalised.

I can't tell you I haven't tried yet

Looks like there is some confusion. I don't have a journal mounted on a FAT or something partition via a loop device. Looks like your making too much assumptions sometimes :)

I Just have the entire ext file hosted on the FAT32 2nd partition mounted via the loop.

Whatever writes goes thru loop cache -> ext file and is commited to the SD when the cache is flushed. ext2 file could be hosted on any other FS but I choose FAT32 because it's standard and compatible and doesn't cause any safety issue in this particular case IMHO. And yes nothing is written to the FAT partition but to the ext file which is hosted on the fat partition, and that's quite a considerable difference.
[Desire Android Development] melethron:
As i said i thought about ownhere's script when i wrote about journal. But not using a journal at doesn't make this in any way better so the essence of the arguement stays the same.

And also i DO know that it is a difference if the file is hosted on a partition. THAT is actually the main point of all my critic on a loop file. If you don't understand what i meant read this:

http://xdaforums.com/showpost.php?p=9962533&postcount=86
[Desire Android Development] sibere:
How can I make it clear... I DO understand your point and I already explained my choices of having loop + sync. The data is commited way quicker than 19 secs and I do prefer the extra speed provided by the cache of the loop device to a very little risk of data loss based on a sudden and hypothetical power loss (not a shutdown).

When I put both in balance, cache wins. that's all.

Once again I respect your technical choices orientated towards data safety, please respect mine orientated on a more balanced option.

I never experienced data loss or corruption since I started to use my data2sd nor sudden power loss on my phone since I bought it. I just try to avoid being paranoid about all that stuff.
[Desire Android Development] melethron:
No you DON'T understand as you have just proven with this post. I never said anything against cache and if you look at my script you'll see that i DONT mount with "sync". I never said that on should disable write cache. This would be completely nonsense......

What you do is disabling the write cache with sync mount for the loop file. And then use the cache of the loop file:

This is 1. NO SPEED IMPROVEMENT as ownhere has clearly shown with the benchmarks and 2. it skips "sync". That is my point ....

and not cache ......

The reason there are higher quadrant scores is this: Quadrant is intended to meassure the REAL write speed and thus calls sync after every file. A sync after every file is not needed in most cases others then:

1. Make sure data is save so a developer will call fsync() after something is saved (preference, savegames etc).

2. If some app like quadrant wants to check the PROPER write speed.

What you do is disable the write cache with sync and then use the write cache of loop instead. -cache+cache = same as before. The difference happen then when developer WANTS data to be written properly like in above sample. So all this does is fake quadrant score + risk data corruption. There is NO speed increase at all compared to a normal mount. (well you may still speed up bad coded apps were an uneeded sync is called)

Either you don't understand it ... or you fool yourself ....
[Desire Android Development] sibere:
Wrong.

The two caches are different, so the result cannot be the same so please stop bending the truth to you point of view. If the result is "same as before" then explain me why with loop I have almost no lags whereas I have lags without loop but with the filesystem cache on a ext partition (not a file). You see? Different caches = different results. Maybe you could say that the "write" speed are the same, but the user experience is different.

What you don't understand is that ATM the loop solution which forces cache even if, as you say, the dev wants so sync it's data, is the only one that's suits me regarding lags.

I've experienced bad lags with and ext2/3/4 partition mounted directly even without sync.

Maybe the dev wants something but I know, as a user, what I want too.

Let me force cache with a loop device if I want to.

Let me use "sync" to avoid a double caching if I want to.

Let users decide if they want loop or not.

**** Quadrant. I don' give sh*it about quadrant. Is it clearer that way ?

Keep working on your solution in you own thread and if you come finaly with a stable and more efficient solution, I will surely try it and adopt it if it suits me better.

For the moment my solution suits my needs even if both of us know that it's not perfect. got it? Stop bugging me with data corruption, I already told you many times that I had none.

Please don't become a nuisance as you're doing a great job at finding a better solution in your thread. This thread is data2SD and helping people who have questions or issues with it.
[Desire Android Development] danarama:
Would it restore it to the ext file in the same way an sd-ext restore via clockwork would?
[Desire Android Development] melethron:
Hey i don't to convience you that YOU use a more safe solution. I think i made my point on this clear enough.

If you want loop im fine with that. But there is still a better solution than a loop file on fat.

EXT4

without

journal directly mounted with loop as mount option. This has following advantages.

a.) While the loopfile gets unmounted the fat partition doesn't. A direct loop mount does.

b.) EXT4

without

journal is faster then EXT2 (the speed decrease from ext4 is due to journal)

c.) If you create a ext2 partition out of a file it will use the first 63 sectors for the superblock. This makes your ext2 file then unaligned to the fat partition beneath it.

This would be what i call a balance between Speed and Safety. This would make your script safer because of a)

and

faster because of b) and c).
[Desire Android Development] op8867555:
Just mount the ext file to /data before you restore it.
[Desire Android Development] melethron:
@sibere:

That's what i also don't get with you and ownhere. Ownhere showed that direct loop mount is faster. But you ignore this.

You showed that /data gets unmounted proberly and ownhere ignores this and makes 2 partitions instead of one .....

On top of what i said above please also note:

You cant compare HD Roms with data2ext with normal sense roms. I used leedroid with ext4 (journaled) and used baadnwz and robociks rom with ext4 (also journaled). Leedroid was FAR more smooth then both HD roms.

Also note that the scheduler makes a HUGE difference. Noop makes a rom much more smooth. The first part of my bench test explaines this:

http://xdaforums.com/showpost.php?p=9959265&postcount=257