yashsingh1/xdaforums huggingface dataset:
[Samsung Galaxy Nexus] shaaXo:
Exactly, but you can also skip the second <name-of-Rom> and just directly use "adb push some/path/to/foo.zip /sdcard/".

Just beware that on my Nexus S, on the first boot after every flash, android complains the sdcard isn't readable and has to be 'initialized' (iirc). After confirming this dialog and rebooting, everything is back to normal (no data loss), but you may want to make an /sdcard backup to be sure.

You can also backup the encrypted device using adb pull (pull the block devices /dev/mmcblk0 and /dev/mtd/*), but I haven't tried to actually restore or decrypt the encrypted backups yet.
[Samsung Galaxy Nexus] B1ny:
Thanks to both of you, will test it asap after work. Will report back.
[Samsung Galaxy Nexus] TheShoura:
Glad to see some development on this, hoping eagerly to see something! I'd love to use encryption
[Samsung Galaxy Nexus] B1ny:
Everything goes fine until I shall flash, CWM then gives the same "E can't mount sdcard" as usual :-(
[Samsung Galaxy Nexus] B1ny:
Is it possible to compare the Nexus S and the GN, when the Nexus S don't have encrypted SD card?
[Samsung Galaxy Nexus] netmasta10bt:
removed... Need more coffee.
[Samsung Galaxy Nexus] shaaXo:
Any errors while mounting /sdcard as tmpfs? What does "adb shell cat /proc/mounts" show?

The storage inside the NS and GN should be exactly the same (single internal emmc), but partitioning should be different (single big partition on GN, splitted into /data and /sdcard on NS).

What I read about the GN partition layout, maybe CWM looks directly at /data/media. In that case: "adb shell mount -t tmpfs none /data" and "adb shell mkdir /data/media" and push into /data/media.
[Samsung Galaxy Nexus] netmasta10bt:
It looks like it uses /sdcard

clockworkmod gets the fstab from line 81:

https://github.com/CyanogenMod/android_bootable_recovery/blob/ics/roots.c

function opens this file to get location of /sdcard:

https://github.com/CyanogenMod/android_device_samsung_tuna/blob/ics/recovery.fstab

recovery.c mounts /sdcard line 134:

https://github.com/CyanogenMod/android_bootable_recovery/blob/ics/recovery.c

I'm not sure that your "adb shell mount -t tmpfs none /sdcard" was successful.
[Samsung Galaxy Nexus] B1ny:
Hey man!! You really made my day!!

This works like a wet dream!

With this post you have made things a lot easier for a lot of guys!

I will not even bother trying to say how much I want to thank you!
[Samsung Galaxy Nexus] efrant:
[HOW-TO] Flash a new ROM when you had previously encrypted your data

So, to sum up:

If you have encrypted your data, and now want to flash a new ROM but can't access /sdcard due to the encryption, shaaXo has figured out a way to do it.

Follow these steps:

1) Boot into CWM.

2) Open a command prompt in the directory where you have your adb.exe file

3) Mount the RAM as /data by typing: adb shell mount -t tmpfs none /data

4) Create the /sdcard folder: adb shell mkdir /data/media

4) Push your ROM to that location: adb push /path/to/ROM.zip /data/media/

5) Navigate the CWM menus to flash the ROM.

Note: On the first boot after every flash, the OS may complains the /sdcard is not readable and has to be initialized (or something similar). After confirming this dialog and rebooting, everything should be back to normal (no data loss), but you may want to backup /sdcard to be safe.

Thanks shaaXo!!
[Samsung Galaxy Nexus] bunklung:
Great news. So what is the size of this partition? I know the Gnex has 1G ram, but I am sure it doesn't use all of this. Is there a way to do a full nandroid backup and restore too? I don't think Nandroids are larger than 1G.

orcruin also devised a solution to flashing custom roms by formatting /cache but it seems near identical to shaaXo. orcruin makes no mention of the error message on boot:

http://xdaforums.com/showpost.php?p=24923240&postcount=16

Hello Everyone

To flash a new ROM you can do as follows:

boot in Recovery
format cache partition
from your PC use adb to mount /sdcard as tmpfs:

adb shell mount -t tmpfs none /sdcard

put your ROM on the (fake) sdcard

adb push <your rom> /sdcard

use the recovery to flesh the ROM as usual

This worked form me.

Sorry to be late...

Are we talking about the same solutions here? Expert chime in :)
[Samsung Galaxy Nexus] efrant:
Exactly the same...
[Samsung Galaxy Nexus] bunklung:
Do you think there would be a way to shrink the encrypted sdcard partition (32G) used by the vz gnex and leave it as some sort of temp unencrypted file system so that you can push/pull a nandroid backup file from them?

I just checked my nandroid backups and they are all over 1G. That rules out the device memory.
[Samsung Galaxy Nexus] efrant:
Not sure what you are trying to do? Why not just flash a ROM and, then, restore your backup. It's obviously very easy.

As for changing partitions, I'm not sure that is easily done. If Samsung devices work the same way as HTC devices, then in order to change the partition sizes, we would have to have a bootloader that will allow it. Feel like disassembling the bootloader? ;)
[Samsung Galaxy Nexus] shaaXo:
Making a backup isn't that hard, you could create the tar-files while android is running or get the encrypted images via recovery.

But I don't know a comfortable way to actually restore the backups, especially when they are larger. Splitting up the backups into RAM-sized pieces and puzzling everything back together on the phone is doable, but tedious and error prone. I'll look into getting CWM to expose *all* block devices via USB Mass Storage, so you could just backup/restore using standard tools on the connected PC.

---------- Post added at 10:07 PM ---------- Previous post was at 09:20 PM ----------

I think I got it already. When booted into CWM, use the following command to get access to the entire internal memory (mmc card).

adb shell "echo /dev/block/mmcblk0 > /sys/devices/platform/usb_mass_storage/lun0/file"

The storage should be detected immediately on the host and can be used to make/restore direct disk images with appropriate tools (I just use linux and dd).

On the Nexus S, this is pretty nice, as it has a separate 1GB /data partition and makes independant backup/restore possible. The partition layout of the Galaxy Nexus looks like there would be just a single huge 'userdata' partition, that has to be read/written as a whole. Obviously this makes the whole process extremely slow and potentially reduces the lifetime of the flash storage (when restoring a lot). But it allows a complete (as in every-single-bit) backup and restore and is relatively safe.
[Samsung Galaxy Nexus] bunklung:
I am trying to do what most people use nandroid restore and backup for, but my hands are tied because of encryption. It's great that we have a sorta kinda work around with flashing roms. When you flash a rom and restore your backup, what do you mean? I can't make a nandroid backup.

---------- Post added at 05:48 PM ---------- Previous post was at 05:38 PM ----------

I recall making a manual backup within Adb/windows when I was trying to figure out how to make a Odin backup. My thinking was that I could restore within Odin. I never got that far, but was able to use fast boot to restore to a stock kernel.

Could I use adb while in recovery and restore all those partitions effectively doing a manual nandroid via a windows host?
[Samsung Galaxy Nexus] rocketnz:
Hi All, my SD card was encrypted and i have just performed a full wipe on internal to reset to defaults and now the phone doesnt know the SD card is encrypted and wants to format it, how do you access your SD card after a full phone wipe? (internal)

dont tell me all my data is lost?
[Samsung Galaxy Nexus] bunklung:
99% certain it's lost, but maybe someone else can chime in.
[Samsung Galaxy Nexus] shaaXo:
I would have expected the factory reset process to wipe the internal storage.

Maybe the encryption master key is still around somewhere. Google says (source.android.com/tech/encryption/android_crypto_implementation.html) that they keep it in the last 16KiB of the partition, but on my Nexus S, it's saved in the /efs/userdata_footer file. You could look in the efs folder (adb shell ls -l /efs) and see if it's still there. If not, the crypto footer may be there, but checking for that is more complicated.

Also a quick status update: I am now able to manually setup dmcrypt and can access the encrypted /data and /sdcard from my linux box. I'll probably write a small script for that.
[Samsung Galaxy Nexus] bunklung:
Sweet. Keep us posted.

Do you think FreeOTFE would do the trick on Windows?
[Samsung Galaxy Nexus] shaaXo:
Technically, yes, but it's probaby easier to get CWM Recovery to mount the encrypted partitions.
[Samsung Galaxy Nexus] bunklung:
Slightly confused here. In your below comment, you are still referring to using a Linux host machine connected via a USB cable? And when you say mount the encrypted partitions, you are refering to using ADB when in CWM Recovery and using this command, "adb shell "echo /dev/block/mmcblk0 > /sys/devices/platform_mass_storage/lun0/file""?

It's too bad CWR didn't have native dmcrypt support.
[Samsung Galaxy Nexus] rocketnz:
Ok WEIRD. I just did a full wipe on my phone again. then re rooted it, went to CWM and it could see my SD card, so i load up file explorer and boom, there is my SD card and all my files. even though its encrypted, ok now i'm confused.

thank christ. full backup now before anything else goes wrong. praise the lord! hahaha
[Samsung Galaxy Nexus] shaaXo:
I'm writing it in python right now, so the script itself should run fine on Windows/Mac OS.

Actually, it does. dm-crypt is just the kernel component and the CyanogenMod CWM-based recovery does have the necessary modules built in (it uses the same kernel for recovery and normal booting). But besides kernel support, there is just no userspace tool to configure it (on a typical Linux host, there would be dmsetup and cryptsetup). So I compiled a small C program, based on the code that android/vold itself uses to set up dm-crypt and copied it over into the recovery system. Now I can access my encrypted partitions inside CWM and already succeeded in making a backup.

Restoration is a bit tricky, since CWM probably tries to format the wrong device before restoring, but I'm happy with my progress for today.
[Samsung Galaxy Nexus] shaaXo:
I've finished a beta version of androidcrypt.py, which is able to get the recovery system to decrypt and mount the encrypted partitions. It should detect the connected android device, do some checks, look for the crypto footer, ask for a password and finally decrypt and mount /userdata and /sdcard, so you can backup to or update from /sdcard. It includes a slightly modified binary of dmsetup, I put the source in a git repository.

I tested it on my Nexus S on Linux and on Windows, on both you need to have Python installed (Windows Installer) and have adb on your PATH environment variable (probably python, too, for convenience). It will probably fail with the current version of CWM recovery, as it seems only the CyanogenMod CWM-based recovery has all necessary flags set correctly.

If you have made backups and want to try it nevertheless, here's quick step-by-step guide.

Extract attached androidcrypt.py.zip file, open a terminal window and navigate into folder where you extracted it.

Make sure you can execute 'adb' and 'python' without error (you can exit python by entering 'exit()'). If it doesn't work, check you Android SDK install, especially the PATH, CyanogenMod Wiki has a detailed howto.

Run 'python androidcrypt.py', see what happens and report your output.

Do not try to restore backups or format partitions inside CWM, I have no idea what will happen.
[Samsung Galaxy Nexus] ghost shell:
Amazing work shaaXo!

But I have some questions since I'm with SG S2 with WanamLite:

- is dmcrypt available on all CWM versions or just the CM9 version?

- adb doesn't see any connected device when I'm in CWM-recovery mode. Any idea why? (maybe a usb or mtp issue?)

It would be great if your work could start to be integrated in the ROMs out there.
[Samsung Galaxy Nexus] shaaXo:
Thanks. :)

What I can tell is that the upstream version from koush does not support the required encryption modes, fixing this would be a very minor change, but I haven't contacted him for the moment.

There are some different, CWM-based recovery images floating around, at least the CyanogenMod version for the Nexus S does have the required kernel settings. (It should also be possible to replace the kernel of a recovery image with a different kernel, using the kernel from the system image would guarantee compatibility.)

On Windows, it's probably a driver issue, took me almost an hour setting it up correctly on a VM. Once I had the driver correctly installed and no more question marks in the device manager, it worked fine.

I'm trying to get my hands on a few different phones to try it out. It may also be possible to get the dmcrypt/cryptsetup source into the CyanogenMod source tree, to have it preinstalled on every system or recovery image.
[Samsung Galaxy Nexus] ghost shell:
I'm using linux.

It's strange because the adb interface works fine when the phone is in "normal user mode".

It's only when in CWM-recovery mode that it doesn't work. Makes me think that it could be something broken with my CWM version or kernel.
[Samsung Galaxy Nexus] shaaXo:
Are you using a udev rule to avoid restarting running the adb server as root? At least on my Nexus S, the USB product ID changes from 4e21 (normal boot) to d001 (recovery), which is problematic if the udev rule only allows access to the "normal" product ID.
[Samsung Galaxy Nexus] bunklung:
Do you have any insight into replacing the kernel in a recovery image?
[Samsung Galaxy Nexus] shaaXo:
I successfully replaced the kernel of the current CWM-touch-recovery image by using split_bootimg.pl and put the image back together with mkbootimg. Unfortunately, the whole process is rather complex because it requires compiling the mkbootimg binary (which I already had from building CM from source) and the correct base kernel offset for the device.
[Samsung Galaxy Nexus] makelegs:
That's awesome!!
[Samsung Galaxy Nexus] ghost shell:
I do have an udev rule:

cat /etc/udev/rules.d/70-android.rules
SUBSYSTEM=="usb", SYSFS{idVendor}=="04e8", MODE="0666", GROUP="plugdev"

But I have no clue if this is the cause of the problem or not.

When plugged in CWM-recovery mode, lspci gives me nothing and from dmesg I get:

From google I found this http://xdaforums.com/showpost.php?p=23051153&postcount=10017

Not much info, but seems it may be a ROM issue (note FWIW: I'm *not* using the ROM in the link above, I'm using WanamLite 10.7).

Any suggestions?
[Samsung Galaxy Nexus] shaaXo:
Could try a few different recoveries (with different kernels) like CWM Recovery, CWM Touch Recovery, Stock Recovery or TWRP. Or maybe re-plugging the phone while running helps.

IIRC, I had a similar issue before, when switching the kernel of the non-touch CWM Recovery to the CyanogenMod kernel. Using the same kernel with the newer-versioned CWM Touch Recovery surprisingly worked, though (but killed touch support).
[Samsung Galaxy Nexus] makelegs:
Sooooo....are you attempting to invoke decryption from recovery to allow partition mount/access? That would be SO full of win!!!
[Samsung Galaxy Nexus] shaaXo:
That's exactly what androidcrypt.py does. I just uploaded a new version to my original post, that improves finding encrypted partitions.

It's nice that you already compiling CM for many devices, as CM's recovery.img seems to be the only recovery that just works™ right now, without any modifications.
[Samsung Galaxy Nexus] makelegs:
Ya, TWRP didn't seem to work with adb while encrypted, for me, but CWM does. I'm hoping there's a way to execute from recovery without adb or any need to be tethered to a computer.
[Samsung Galaxy Nexus] coolfx35:
I tried these, but after i try to flash the ROM, I get this message

Can't open /sdcard/gapps-ics-20120514-Eclipse.zip

(bad)

Installation aborted.

---------- Post added at 12:26 AM ---------- Previous post was at 12:20 AM ----------

did anyone try this yet? I am little bit scared what it will do to my phone :)
[Samsung Galaxy Nexus] makelegs:
I've been following the suggestion in post #39. It works like a charm. Only problem is you have to be near a computer with adb every time you wanna flash something, and whatever you push to your device gets deleted every time.

Beats not flashing anything at all, though!!
[Samsung Galaxy Nexus] coolfx35:
makelegs,

Does your Galaxy Nexus automatically shut down at times? While in sleep mode?

Mine does it 3 times yesterday, I am not sure if it's caused by the custom rom.
[Samsung Galaxy Nexus] makelegs:
No. Mine does not. I run my own builds, though. No clue what you're running on urs.
[Samsung Galaxy Nexus] nrgaway:
I am currently running the latest CM9 nightly with TWRP recovery.

I am interested in encrypting the phone and I see that it is best to use the CM recovery over the TWRP.

How would I go about re-installing the CM recovery without doing a full wipe of my current installation (and were would I get CM recovery from)?

Thank you!
[Samsung Galaxy Nexus] netmasta10bt:
Supposedly the latest TWRP should fix this (v2.1.6) but the goo manager only sees 2.1.2 as the latest to flash at this time.

I went back to clockwork (CWM) using 'rom mananger' last night to upgrade my cm9 nightly on my encrypted gnex. just download rom manager (from the market) and it will download cwm 5.5.0.2 and flash it while your phone is running.
[Samsung Galaxy Nexus] Dees_Troy:
I have some test builds of TWRP with some initial decryption support. It requires you to place your password in an open text file in /system until I can get a keyboard added. If you would like to test this functionality, PM me. Please know that this hasn't been tested thoroughly yet, so consider yourself warned.
[Samsung Galaxy Nexus] makelegs:
If any of you are interested in testing this, I can confirm that it works remarkably well! I personally flashed a ROM from recovery on my toro (on the fly at a bar). I haven't tested backup/restore, but I believe Dees_Troy tested that on his device, successfully. Once he's able to nail down the keyboard for TWRP, it should be completely seamless and exactly what I think all of us have wanted.
[Samsung Galaxy Nexus] nrgaway:
Awesome, I can't wait for the keyboard!!!
[Samsung Galaxy Nexus] makelegs:
Oh believe me....it's already awesome!!!

Dees_Troy has been a true humanitarian to even consider this issue that, arguably, doesn't impact him at all.
[Samsung Galaxy Nexus] la7lou7a:
O Hai..

Let me understand what you guys are saying..

1.You have your phones encrypted

2. Running custom roms

3. Able to flash custom roms through CWM

a) By loading /sd partition into Ram

b) By decrypting before flashing

Am I right or am I still a noob? :P

also (I have never tried encrypting) currently does your phone asks for a password everytime you boot it? If so, does it ask the pass even before the bootloader? I'm trying to figure out if having the phone encrypted while begin root and with unlocked bootloader still means the phone data cannot be accessed even with a usb and a pc without the pass.

Thanks..
[Samsung Galaxy Nexus] makelegs:
By using this test version of TWRP, you can place your encryption password in an open txt file that TWRP will use to decrypt in recovery, thereby allowing data to mount and flashing to occur. It's beautiful!

FTR: if you don't need to encrypt your phone, then I would recommend you not do so.....even though this makes the day to day issues vastly more manageable from a crackflashing standpoint.
[Samsung Galaxy Nexus] theking_13:
The amount of people that encrypt their device in the first place (so a thief doesn't see the pictures of their dogs I'm assuming) and then flash a rom is ridiculously big. Almost depressing.
[Samsung Galaxy Nexus] la7lou7a:
cool
[Samsung Galaxy Nexus] la7lou7a:
I have a question

Would the phone be usable if I turn on encryption after flashing a custom rom with a custom kernel?
[Samsung Galaxy Nexus] scandiun:
Cyanogenmod supports it since ICS.
[Samsung Galaxy Nexus] makelegs:
The encryption is all-consuming. There is no turning it off once it's been encrypted, without reformatting. This allows an encrypted phone to mount the encrypted partition in recovery which allows backup/restore and ROM/kernel flash. In other words...it lets the device function as a crackflasher needs it to, but still requires the encryption password upon reboot. When Dees_Troy has completed the keyboard implementation on TWRP, recovery will allow password entry to mount. The current test TWRP merely accesses the password from a text file in order to mount. The final version should be completely secure, where this version is not truly secure as of now.

Hopefully that's clear. I've been using it since last Friday, and I flash my own builds plus a kernel. I have not actually tested the backup/restore, but he verified that his testing was successful, so I presume it works.
[Samsung Galaxy Nexus] la7lou7a:
I understand that, but I was thinking of doing something like this, but I'm wondering if it will cause any problems:

unlock/custom recovery/root/flash and customize as I wish/unroot/stock recovery/relock and then encrypt. So I don't really need to access the custom recovery. Would this work out?

Only CM? Does AOKP or other roms support encryption too? Or is it something related to the kernel?

Thanks guys
[Samsung Galaxy Nexus] makelegs:
If that's your plan..... then none of this applies or relates to you.
[Samsung Galaxy Nexus] efrant:
What makelegs is trying to say is: yes that will work. This thread is about flashing a rom after encryption...
[Samsung Galaxy Nexus] rssaddict:
Very exciting that you were able to get TWRP working with decryption, look forward to seeing this in a public release!
[Samsung Galaxy Nexus] efrant:
[slightly off-topic]

Not sure if anyone in this thread can help, but would it be somehow possible to create a nandroid backup directly to your PC (Windows that is), or to a OTG mounted flash drive, instead of in /sdcard/

Discussion started in the thread here.

Thanks.

[/slightly off-topic]
[Samsung Galaxy Nexus] Dees_Troy:
TWRP 2.2.0b1 (beta1) for GNex with keyboard support and decryption:

GSM aka Maguro

Verizon aka Toro

Sprint aka Toroplus
[Samsung Galaxy Nexus] mimox:
Very nice! I just fastbooted the img, the keyboard seems to work fine and the decryption worked as well, I was able to browse the file sytem :laugh:

Looking forward to play a bit more with this during the weekend.
[Samsung Galaxy Nexus] segv11:
Can't wait to try it out, but the link for toro version gives "Cannot connect to the MySQL Database Server".
[Samsung Galaxy Nexus] kobik77:
Doesn't use encryption, but this is top stuff guys.

This recovery is amazing!
[Samsung Galaxy Nexus] ridobe:
In the goo app it lists the other two but not toro.

edit: nevermind. For some reason they weren't listed in the same folder. The link on page 9 works now.
[Samsung Galaxy Nexus] nrgaway:
Awesome! Could someone maybe quickly tell me what the fastboot command would be to load/install it from windows; its been awhile since I needed to use fastboot.

Thanks!

---------- Post added at 06:25 PM ---------- Previous post was at 06:21 PM ----------

I understand when you boot your device that has encryption, you are of course asked for the password you used to secure the device.

When the device in then fully booted, and when you encounter a lock screen, are you required to enter the same password again, or can you choose to have a simpler lock screen password / pattern?

Since I can now update ROM easy, I am going to encrypt my device and don't want to be entering a 40 char password if it needs to be entered every time lock screen is activated.
[Samsung Galaxy Nexus] efrant:
fastboot flash recovery twrp-2.2.0b1-maguro.img