yashsingh1/xdaforums huggingface dataset:
[G1 Android Development] Just_s:
HTC EVO?

Will this work on the HTC EVO? I migrated over from G1 and I'm having switchrom withdrawals!!

I was going to try it, but I'm still pretty green, so I didn't want to risk bricking without seeking some further, more experienced input.
[G1 Android Development] derfolo:
OK, I don't see the answer here or in the new recovery 1.4 thread. Does this script work with recovery 1.4? (If it is included, it is not listed as included in the changelog.)
[G1 Android Development] devsk:
One of the ideas that I am thinking of implementing is some sort of compression support. For example, data.img in nandroid backup can gzipped to 4MB compared to 12MB. This can reduce the space requirements

considerably

.

Compression does make the whole process of storing and restoring much slower though. So, I need to work out some compression levels to keep it quick but still gain some ground. Another design aspect to make sure that enough juice is present for store/restore to complete.
[G1 Android Development] devsk:
I haven't moved to 1.4 yet. I don't see any reason for it not work. Unless nandroid backup location or directory structure changed, it should work.

cyanogen is sleeping (literally I mean) at this time I think.
[G1 Android Development] Deuces:
works fine in 1.4
[G1 Android Development] derfolo:
Yeah, it does work in 1.4 as well. You might want to add that to your first post.
[G1 Android Development] devsk:
Thanks edited.
[G1 Android Development] My_Name_Is_Neo:
On that line of thought it is a feature no doubt. But it will still overwrite all the user data while extracting the tar ( old version of userinit.sh over new one). as the tar was made as a snapshot of whole /system/sd and not just app, app-private & cache. If you can conditionally tar just these 3 folders ( ask the user to full backup or apps backup ). OR by not overwriting the existing files when extracting the tar, that to conditionally by asking user.

I hope I am not asking for too much.
[G1 Android Development] devsk:
I did some timings:

data.img size = 11.94MB

gzip -1 data.img => 11.8 seconds, size = 4.28MB

gzip data.img => 29.6seconds, size=3.92MB

gunzip data.img.gz => 3.5 to 3.6 seconds for both.

The difference in default level and level 1 is not that much in size but times are almost one 3rd. This is very good news. It seems to me that gzip -1 can be used without hesitation because the tar itself can be done in parallel, so its not going to add much time to store/restore. I am going to cook up changes to nandroid script and switchrom to use gzip -1.
[G1 Android Development] devsk:
Very good point! I think restore should be restricted to the folders which affect the ROM behavior only. I will make that change in next version.
[G1 Android Development] overground:
Don't forget app_s for hero builds. If switching from a hero build back to a cupcake, for instance, app_s should be removed. Just restored a backup after clearing my EXT partition, and it worked perfectly!

Thank you again!
[G1 Android Development] devsk:
It does a removal of app*, so app_s is covered.

You didn't need to do that with this script.
[G1 Android Development] kizer:
I thought I read up some where removing an old copy of the nandroid backup after the script creates a switchROM. My only worry is if the script should ever hicup it would delete the original nandroid backup that is created in the recovery. I personally think it should be up the user to delete that file if its a matter of wanting to save space. I normally backup to my PC myself.

Lets face it most of us are running 8gig cards or something that is above and beyond the 1gig so space isn't a super huge problem.

Gui would be slick, but then again its not hard to run a few command lines. If you can root your phone you definately got the talent to type a few command lines or you shouldn't be switching roms in the first place. ;)

Is there anyway for this script to insure its a true and complete backup meaning is there any error checking tested?

As well has anybody done and restoring as of yet?
[G1 Android Development] devsk:
This was my original thought as well. But I think this should be a tunable option may be.

I am going to reduce the overall size of backups, both nandroid and switchrom's.

if you look at the script, you will see a couple of TODOs. Although I wrote checking for error conditions very well, I don't have a way to verify if the backup or restore is valid. I am still thinking about this one.

I have....:D Every time I get back to my JACMyTouchSki ROM after testing a new ROM (drizzy is one freak of a guy and soulife needs to get a life, may be me too....:D), I feel elated that I took the pain to write this script.
[G1 Android Development] My_Name_Is_Neo:
Well in the script there is a place holder i see where there will be md5 check done in future by devsk. :) . md5sum command can generate them and nandroid already creates a file called nandroid.md5 which has the hashes for all img files. so its a matter of comparing them. i think devsk will be on it for the next release
[G1 Android Development] devsk:
Yeah. But the problem is that /sdcard is fat32 and anybody can write anything anywhere. This means that even the md5sum's can be written. So, essentially if someone was a rogue application (and not necessarily a root application) and it wanted to install something as part of the nandroid restore, all it has to do is to write code to modify the nandroid backup images and modify the md5sum file also. This is a BIG security hole at this time. And just verifying md5sum hashes in switchrom.sh doesn't solve this issue.

I am thinking I will create a folder on the ext partition with root-only permissions during the recovery session to store hashes which can only be changed by root (we already know that if you give access to root application, you have just handed the phone to application writer). And then verify the hashes during restore being sure of the fact that some random joe script kiddie hasn't tempered with the nandroid backup and their hashes.

Does anybody oppose this idea?
[G1 Android Development] My_Name_Is_Neo:
I like the idea as just storing the hashes will not take much space on ext partition so its not a problem.

But in most of the tutorials I've seen people suggesting oh you wanna format EXT partition...just rm -r on whole /system/sd ... in that case you will loose

the hashes.
[G1 Android Development] devsk:
One way to avoid the 'rm -rf *' kind of step would be to create a .switchROM folder, so it won't match '*'. And once user sees that the only thing left is .switchROM folder, he knows that its switchrom.sh that needs it. Hopefully, he will leave it alone. If he deletes it, he can kiss his ROM bookmarks goodbye....:D

Also, the nice thing about ext filesystem is that I can make the folder read-only but still create writable files in it. So, removing the folder will require users to use '-f' argument to rm. If they say 'rm -r', it will crib about folder being read-only and ask for confirmation.
[G1 Android Development] Chahk:
Do Android's gzip/tar allow for password-protecting the archives? That would solve the security hole. Of course if you happen to forget the p/w, your backup is useless. At that point it becomes a user problem though :)
[G1 Android Development] devsk:
neither tar nor gzip allow for password protection.
[G1 Android Development] mookie3three:
But if your using the same ROM there won't be any issues? Cool. Thanks will try this when I upgrade to the next CM rom.
[G1 Android Development] devsk:
yeah, if you are using close enough version, may be no problems. e.g. between cyanogen 3.6.8 and 3.6.8.1, I wouldn't expect a change, but between 3.6.8 and 3.9, there will most likely be a change. But why take a risk when this script can restore it for you....:D
[G1 Android Development] DirectMatrix:
Not sure.... [SOLVED]

I don't know if I'm doing something wrong on my end but I can't get past this point (see image of cmd window)

I can't press anything on the keyboard to accept or deny the action.

Am I missing something?

EDIT: Solved the problem thanks to devsk's suggestion of upgrading to the latest SDK.....great script!!!
[G1 Android Development] devsk:
Its something to do with how keyboard commands are passed to the sub-shell which is created by adb shell. Is there a reason for you to run the subshell with adb? why not just say 'adb shell', get in there and then run 'sh /sdcard/switchrom.sh ...' from within that shell? Can you try doing that and post if you hit the same issue?

Also, what version of adb is that? I hope its from 1.5r3 sdk.
[G1 Android Development] Chahk:
Try typing in "yes" (without quotes) and pressing the Enter key.
[G1 Android Development] Ellises:
this is an excellent script, I've been playing around with it and it works marvelous. I like to see something like this built into cyanogen's recovery, like if you could make backups, and then select one to restore, that would be great.
[G1 Android Development] My_Name_Is_Neo:
I used it yesterday and it works perfectly.

But... after i Restore from script it promps to wipe and the do nandroid restore. which is a little odd. If I am doing nandroid backup why is there a need to Wipe. Well i did not wipe and it works just fine.

Devsk, my question is. why is there a need to Wipe?
[G1 Android Development] devsk:
There is NO need for wipe. Its like one of those things: you do it for no apparent reason, just trying to "make sure", accounting for the unknown, unseen, unobserved.....:D
[G1 Android Development] My_Name_Is_Neo:
Aahh... alrite.. well in the next version you could change the text to "You can wipe just in case" :P ... neways kudos to you man.. thx..
[G1 Android Development] overground:
Haha...I'm aware of that. I was just reporting it worked perfectly after an EXT wipe. Thank you.:)
[G1 Android Development] Diceman4:
This is the best most useful script since Nandroid was released! THanks!!!!!!
[G1 Android Development] devsk:
I am glad it helped you!

Next goal is to find some time from my work and reduce the size of backups by a factor of 3...:D So, you have 2 more ROMs to switch to given the same amount of space...:p;):D
[G1 Android Development] kizer:
Im sure its a dumb question, but I'll ask anyways.

1. Is it possible to expand on your script just a little. say

"sh switchrom.sh inv" which would give you a list of say your switchROM folder? Of course all it is "ls /sdcard/switchROM" Reason being when I built a backup I did the following:

"sh switchrom.sh store cm2.6.5phonefix" when it should of been "cm3.6.5phonefix" and I about freaked when I tried to restore and found out that well I named it wrong. Did the directory diving to figure it out. No biggie either way, but just trying to cause you more work. :D

2. If you rename a folder does it make a difference in the switchROM folder being that I created the wrong folder "2.6.5" opposed to "3.6.5". I'm guessing it doesn't care since its just refering to a folder and the name isn't stored anywhere.
[G1 Android Development] jasherai:
gpg to compress and secure?

this is absolutely true, just wondering if gpg has been compiled for android yet?

gpg'd files and dirs tend to have similar compression to a gzip'd ones... with the crypto... or did do in the use cases I had at work.

just an idea for you to ponder ;)

jasherai :cool:

p.s. Great work btw!
[G1 Android Development] overground:
It just stores them as you name them...it doesn't care what build it actually is.

I second the possibility of a "list" feature, so we don't have to remember exactly what we named them. I ran into the same scare yesterday and realized I forgot an "s" in one of my names. Careless, I know. If this could be done, it would save the hassle of having to keep a build name list.

I cannot say thanks enough for this script, however.
[G1 Android Development] kizer:
As well I've not offically said Thanks, but I've given my share of questions and ideas.

Thanks a bunch.
[G1 Android Development] devsk:
1. list and delete commands are coming.

2. Yeah, the script doesn't care. If you rename it, a.) now, you will need to provide the renamed name, b.) in future, the list shown will have the new name.

I am still testing compression. I will see if I can make it in next version.

Interesting! I might have to investigate this.
[G1 Android Development] anddep:
switchrom makes a folder in the sdcard with the backup right? can i delete whats in the nandroid folder? because i want to backup my new updates and its taking up space
[G1 Android Development] devsk:
That is correct. Why not archive your entire nandroid backup on PC?
[G1 Android Development] anddep:
good idea! i will but when the nandroid folder is moved to the PC.. is not needed to restore a rom? being that theres a switchrom folder with the back up
[G1 Android Development] devsk:
Yeah. You can do whatever you want with nadroid backup folder. I create a tagged copy in the switchROM folder. In fact, you can manually copy the folder from switchROM to nandroid and restore it if there are issues with the script.
[G1 Android Development] anddep:
thank you very much!:)
[G1 Android Development] jaapschaap:
Wow cool hoop it will be build in the next recovery img, so it will be a bit easier for me ;)
[G1 Android Development] nickL0V3:
Ok, read through the thread and didnt see this problem anywhere:

I type

# mount /sdcard
# sh /sdcard/switchrom.sh store cyan-v3.9

and I get the error:

sh: can't open ' /sdcard/switchrom.sh'
[G1 Android Development] bittermormon:
i needed to launch the script from the SD card so

# mount /sdcard
# cd /sdcard
# sh /sdcard/switchrom.sh store cyan-v3.9
[G1 Android Development] nickL0V3:
Same result. I feel like Im doing something wrong here.
[G1 Android Development] Deuces:
your switchrom.sh must not be on the sdcard, or you didnt change it from .txt to .sh
[G1 Android Development] nickL0V3:
I promise it is/I did :P
[G1 Android Development] BombaMyte:
This script sounds promising. Testing it now; I've got a few hours to kill...

EDIT: Wow... All I can say is wow... Doesn't take long to make the tagged nandroid backup, doesn't take long to restore from it. This DEFINITELY needs to be a part of Nandroid. If anything it'd save us ~100mb/backup (since this makes a copy of the nandroid backup files, if I read correctly), perfect for folks with low-space sd cards (ordered my C6 8gb yesterday, *joy*).

Great work. +1 beer
[G1 Android Development] devsk:
That file has a name switchrom.sh.txt...:D Because Windows, like a smarty pants, shows switchrom.sh but says "Text Document", which really means its .txt file.

You need to enable "Show extensions" and then rename it to switchrom.sh.

PS: BTW, your image only showed up once I did a "Quote" on your message.
[G1 Android Development] nickL0V3:
ah! Thanks :] I knew SOMETHING wasn't right. as for the image...I dont know why it didnt show. It didnt for me either, I had to click Open In New Window
[G1 Android Development] nickL0V3:
Nevermind!
[G1 Android Development] BombaMyte:
Not trying to be the rotten apple here, but I decided to try something and I think there may be a bug in the way the script handles creating the backup. Note: I'm using CM 3.9.1.

About an hour ago I took a short walk to the store. On the way there I decided to check & see if I actually needed the nandroid backup made before storing the tagged backup. I deleted *all* nandroid backups in my nandroid folder, rebooted into recovery, created a new one as a starting point. Used the script to make the backup, then deleted the nandroid used to make it.

Next, I used the script to restore my tagged backup. I followed the directions (Wipe, restore latest nandroid backup), but here's my question: I've been back from the store for ~20 mins now, so why is nandroid *STILL* restoring? I also have to mention that I took a look at the file size of the tagged backup folder, and it's nearly 5x that of any other nandroid backup I have (~110mb nandroid, ~471mb switchROM :eek:)

Another ? while I'm thinking about it: I know the script makes a backup of your ext data. I'm wondering tho, is it making a backup of the entire partition? If it is, then would explain the ginormous switchROM backup size; my ext3 is 384mb...
[G1 Android Development] devsk:
Actually, its making the backup of the entire ext partition. This point was already raised. I am going to change this behavior and just backup/restore app* dalv* folders.
[G1 Android Development] My_Name_Is_Neo:
Can you go to the terminal and run these commands

su
cd /system/sd
du -sh

this will tell u the USED size of your EXT3 "SD". please post it back

Then anyone here can make a calculated guess that problem is with the amount of data in the partition or script
[G1 Android Development] BombaMyte:
105.8M, but this could be a false reading. I flashed JACHero 2.7.4r2 recent.
[G1 Android Development] devsk:
Version 1.0 is out and has support for direct backup i.e. no need to do nandroid backup first during store, and no need to do nandroid restore during restore operation afterwards, everything is done by the script. It does impose the necessity of CM recovery V1.4.

List and delete operations are supported. And so is compression but it takes a long time to do due to the way it is done in nandroid script. You can definitely try but it will take like 17 minutes. I will post patches to improve that later on.

Couple of points to note:

1. Do not try to restore older backups with the newer script. Migrate older backups by restoring them and then backing up with newer script or just leave the older script around in /sdcard and use that if you need to restore older backups.

2. Newer script needs to be in /system/sd because nandroid script fails if /sdcard is already mounted.

Read the OP.
[G1 Android Development] kizer:
Never mind I re-read the first topic and appearently I can't read very well.
[G1 Android Development] johnnylicious:
great job on V1.0 gonna try it right now =D
[G1 Android Development] rondey-:
I liked the script before this update, now its even better. I have plenty of room on my sdcard so compression isn't a big issue for me but the more features the better!

We need cyanogen to integrate this script into the recovery image and do away with the current nandroid backup and restore latest.

Great work!
[G1 Android Development] anddep:
Agree would be much easier
[G1 Android Development] Chahk:
After pushing it to /system/sd/ do we need to chmod the script in order to make it executable?
[G1 Android Development] rondey-:
I didn't try it before but I went ahead and gave it 755 permissions and it worked.

On a note for the compression, I am running what is currently in my signature without too many apps. Uncompressed my backup was 142mb. With compression it came down to 85.

I didn't time it but the compression took significantly longer.
[G1 Android Development] devsk:
Yeah, the compression is significantly slower. One of the problems is that cyanogen replaced the gzip executable with busybox link, which killed the compression level option. We know that there is the difference in size between level 1 and level 6 (the default) is about 8% (level 1 being bigger than level 6) while level 6 takes 3 times longer than level 1. If we can somehow get the level 1 working, we can reduce the compression times significantly.

About the permissions, it doesn't matter if you call the script with 'sh' like "sh /system/sd/switchrom.sh ...".
[G1 Android Development] rondey-:
So the 85mb compressed backup I have would be roughly just 8% bigger and take a third of the time?
[G1 Android Development] devsk:
yeah, pretty much. Give or take some. Different files are compressible to different degrees.
[G1 Android Development] Chahk:
I'm sorry, I can't hear you over how awesome this script is! :)

Seriously, I was able to tag CM 3.9.2, re-format the SD Card with parted, install and configure JACHero 2.7.4, tag it, and restore the CM 3.9.2 ending up right where I started with a fully functioning system. All in under an hour! Here's hoping that recovery 1.4.1 gives it a nice GUI :)
[G1 Android Development] overground:
Don't know if this is normal, maybe too many "-" in my naming structure:

/ # sh /sdcard/switchrom.sh -c -s dudes-sick-a2sd
sh /sdcard/switchrom.sh -c -s dudes-sick-a2sd

Storing ROM dudes-sick-a2sd, proceed? (y/n) y
y
Storing compressed app data...
Storing compressed nandroid backup...
umount: can't umount /sdcard: Device or resource busy

Error occurred during nandroid backup...
Have a look at "/system/sd/nandroid.log" to find the cause.

Here is my nandroid.log...looks like it's just an unmount error:

nandroid-mobile v2.1

Compress
Using dudes-sick-a2sd- prefix to create a compressed backup folder
Found gzip, will compress the backup
mounting system and data read-only, sdcard read-write
mount: mounting /dev/block/mmcblk0p1 on /sdcard failed: Device or resource busy
mount: mounting /dev/block/mmcblk0 on /sdcard failed: Device or resource busy
Error mounting sdcard read-write
umount: can't umount /sdcard: Device or resource busy

EDIT: @ devsk got it, thanks for the help...moving to /system/sd
[G1 Android Development] My_Name_Is_Neo:
Hey devsk . Now You changed the TARing method in the new script to only tar aap* and Dal* so its all good. But you know it will break when users flashing a new rom (Not restoring) but flashing a new rom which requires you to wipe off /system/sd. people will loose what they had other than app* & dal*.

they should not be in an impression that they are backing up everything. because they are not.

Now here is something which you can add as additional option. to remove only the stuff which was actually backed-up in the script.

you can call it "Remove backed up data from ext partition"
[G1 Android Development] devsk:
This happened because you put the script in /sdcard. Nandroid script is buggy, it doesn't like it if /sdcard is mounted when you call it. So, we need to place this script in /system/sd and run it from there. Please do that and let me know if you encounter any errors.
[G1 Android Development] devsk:
Neo, the script s storing app* and dalv* and its just restoring that only. Its not storing the whole partition. The issue with storing the whole partition was that it can be pretty large if people have other stuff in there, which my script doesn't backup and doesn't remove. The goal was to restore the behavior of the ROM, so this is acceptable. If people want to backup whole ext partition, they will need to do it separately. Or may be I can add an option to do this in next version....:D
[G1 Android Development] My_Name_Is_Neo:
No that's not what i wanted to imply here. The scripts does exactly what it needs to. and does if perfectly.

Here is an example -

Someone backed up the current system install with the script. now went to a NEW ROM thread say Drizzy's Hero ROm thread. the instruction says run

rm -rf /system/sd/*

if he does this he will loose the data which was not backed up. So instead of executing above command he should be executing

rm -rf /system/sd/app*
rm -rf /system/sd/dalv*

to protect the data which was not backed-up.

Now if you provide another option in the script to just remove the data which was backed up it would be nicefor lazy fingers like mine and ppl who do not look into the script before executing.
[G1 Android Development] devsk:
Neo, I understand what you are saying. I have a note regarding this in the first post. I will add an option in the next version to make sure its explicit.
[G1 Android Development] My_Name_Is_Neo:
Cool.. i just read the Note in RED :)