yashsingh1/xdaforums huggingface dataset:
[G2 and Desire Z Android Development] pardsbane:
I think this is what xile6 is getting at, but its common for an embedded system to be divided into two partitions. You start out running on partition 1. When receiving an OTA update, you burn the whole update to partition 2. Then you reboot to partition 2. If it fails, to can fail back to booting from partition 1 again. This guarantees your device can always be upgraded, in a way that is safer and easier to manage than just having a recovery partition.

I haven't done much Tivo hacking in a while, but I remember the Series 2 TiVo had this type of setup.

Also, as another alternative, I wonder if the extra space on the chip is reserved for the fastboot/hibernate system. I know that isn't enabled on the vanilla G2, but if the hardware is the same.... Doesn't that need to write some memory state to persistent memory so it can resume quickly? I'm guessing that its not like a laptop's hibernate where it saves the current state, more likely it just ships with an image of the "fresh" post-boot state and uses that repeatedly (and doesn't update it) but I don't really know. If it does update the state, I could see it needed up to 512 mb (or more) free just to save the ram.
[G2 and Desire Z Android Development] xile6:
basically.

It seems to me like it might have a system of around 512mb. then the backup/ota is about 1gb.

Makes sense to download the ota to an ota partion then.

Makes a checksum for the ota on the backup partition.

Flash the ota to the system partion. On boot if the checksum fails revert to backup partion.

On boot of backup partion clear system and mirror back?

Maybe that is were are rooting will come in. we probably need an ota to see how things work. Because i know it has to be restoring the backup part when the system doesnt pass the checksum .

Can some one verife this. I dont got a counter.

What i need to you to do is power off the phone then back on and time how long it takes to boot. then do a soft root. power off and back on an time how long it takes.

I removed amazonmp3.apk and power off seems to take a while to boot. Then i powered off and back on seemed alil fast but as i said i dont have a counter. If it does take longer after soft rooting then something is 100% rewriting the system
[G2 and Desire Z Android Development] tep065:
what if u redirect the boot up command? maybe have the image of the desire z on the sdcard? any way to specify boot drive? make it so the phone boots to the disire z rom on the sd and not the nand. Would anything useful come out of that? full root on sd card OS? or to tell the Nand OS or Temp Boot OS to look at the SDCARD instead. Maybe you'd be able to attack the nand that way, with that kind of root. Not a permanent root solution idea. re-reading it doesn't sound like a good idea anyway....
[G2 and Desire Z Android Development] Maisum:
But then wouldn't this be like the HD2? Just booting off the SDCARD meaning worse battery life and quicker time of death?
[G2 and Desire Z Android Development] LaNsLyDe:
00:01:08:923 - boot up after temp root

00:01:07:634 - boot up after that
[G2 and Desire Z Android Development] biosehnsucht:
This means that if power loss occurs while writing to NAND that the data isn't corrupted (at least at the 512-byte 'sector' level). This has nothing to do with the G2 throwing away changes to the filesystem on boot. Once the write to the new sector is complete, the mapping in the NAND of logical to physical sectors (as used to support wear leveling, etc) is updated. Presumably, this is usually done first ... so if the write doesn't complete, the data for that logical block is corrupt.
[G2 and Desire Z Android Development] tep065:
I did say it wasn't a permanent solution :) just a way to maybe get around some restrictions and get into the nand.
[G2 and Desire Z Android Development] xile6:
@ biosehnsucht

sorry im all sick and stuff. this damn weather, so my reading hasn't be all that good :P i must of read it wrong. But there is still something that is rewriting the system if the boot times are changing.

I also dealt with this before on something else. but that was long long time ago i cant remember what it was. (video game system, pc,laptop it was something).

I just remember that the backup partition was lock out unless you upgrade which then was a command sent to unlock and flash that backup partition, then from there it would flash the working part of the system. But that never restored on reboot only when you held some keys. (damn i really cant remember for nothing what it was,I wanna say laptop restore/bios but im unsure)

but on the problem for the g2 it seems like once we get an ota update we will learn more about how the nand reflashes the partitions, but with an update there also comes patchs to stop root :(

Ill rack my brain some more and see what i can think but im not a program i just been around alot of them and ive done some stuff before.
[G2 and Desire Z Android Development] cmstlist:
Can anyone find evidence of being able to load, mount, view etc. the backup partition?

Also, is there any evidence of a script anywhere that is responsible for the reversion?

Is there any way to get debugging logs over USB that would explain more about the reversion process?

Just some food for thought. Good luck to all.
[G2 and Desire Z Android Development] xile6:
Seems like it would be in the boot.img or something because its cleaning up on boot not shutdown. Also seems to have a checksum of the system to tell if its been modded. Just throwing that out there

Anybody not how to dump the nand?
[G2 and Desire Z Android Development] nolsen311:
Have you guys taken a look at the layout of the emmc?

From the cat of the various partitions, it wouldn't appear (to me) that there's a straight backup partition anywhere, unless it's counted as part of one of the larger partitions (of which there are only a couple).

http://xdaforums.com/showpost.php?p=8426148&postcount=1

y'know, like as if it were maybe stored on mmcblk0 which appears to be about 2GB in size...and is not one of the "standard" partitions mounted for daily use.
[G2 and Desire Z Android Development] xile6:
Packed into an img or compressed file and not a full partition maybe?
[G2 and Desire Z Android Development] sino8r:
That's what i was thinking. i'm gonna look into this more since we are sure it's there now.
[G2 and Desire Z Android Development] xile6:
If you point me to some nfo on commands I will try them. I did linux long time ago forgot some and some I didn't know. But I'm always willing to learn new stuff
[G2 and Desire Z Android Development] M9x3mos:
Dumb question maybe I missed it so not sure was already discused but has anyone tried to flash back to a older version of software.

The two roms over here are older then what shipped seems like for vision so should be for ours.

http://shipped-roms.com/index.php?category=android&model=Vision
[G2 and Desire Z Android Development] xile6:
Cant flash older ver. System will check an fail due to being older then installed ver.

We did this on the g1 with a recovery .nhb file which doesnt check ver. #
[G2 and Desire Z Android Development] biosehnsucht:
Yeah, that's being done at the software (bootloader code) level not hardware level. Tivo's operate similarly. In addition to having dual partition layouts (primary/secondary) for update purposes (for an update, it overwrites the currently unused partition set, if it fails to boot it goes back to the previous version that worked) but it also on boot checks all system files to make sure they match their shipped versions and if there are any discrepancies replaces them.

Most likely the bootloader is just taking a compressed image of some kind or perhaps just a raw copy of the filesystem and replacing the working filesystem set with it when necessary when something fails the sanity checks.
[G2 and Desire Z Android Development] sino8r:
That makes the most sense. I think. We need an a moddified bootloader and recovery to throw off the check. That would also allow us to flash an update into clockwork to load a custom rom.

I don't really know how to get one made though. We already tried flashing while rooted but they still fail signature checks. Older eng build versions won't appear to work. I'm. Not sure what to do since this is something no one has had to deal with while rooting before. We need an expert to revaluate this now since we know have this image being restored by the bootloader. This is new territory pretty much...
[G2 and Desire Z Android Development] xile6:
Yea modded bootloader to stop the system restore then modded recovery to flash roms.

And yea the g always have something new. The first andriod phone now the first if this type of nand.

And the big chefs arent going to chime n till a few wks when they get there phones.
[G2 and Desire Z Android Development] smjfosho:
Ok, I've just been watching and reading this thread, going to dual boot win7 and linux tomorrow morning and hope to contribute to finding root in some way, but I have a feeling once we crack this *****, this will be the best device out there. Thank you guys that are working on this as well.
[G2 and Desire Z Android Development] sino8r:
Good job, man! That was my first step when I decided to help out with android. Its really handy to have also.
[G2 and Desire Z Android Development] Denkai:
I know this is a stupid approach that is so obvious that I'm sure it's been tried already, but I figured I'd ask, and searching through 31 pages is a pain in the butt.

So what is stopping people from getting temp root, changing /system/ to rw and then pulling recovery.img, editing it to get root through the recovery file, upload, and then loading a rooted stock rom build? is it something to do with the bootloader?
[G2 and Desire Z Android Development] sino8r:
Because it resets all changes after a reboot. We think its a new copy protection feature added by htc. Until we can turn off this protection somehow, no rewrites to the system will work.
[G2 and Desire Z Android Development] HamNCheese:
In a nutshell:

* Root has been acquired.

* Changes to /system do not take due to hardware write protection. You can remount /system rw, but changes to not persist through a reboot.

* Attempts loading other valid rom images have failed at the bootloader (it's version aware, so no dev roms)
[G2 and Desire Z Android Development] HamNCheese:
It's not HTC. It's the NAND that is write-protecting everything.

http://xdaforums.com/showpost.php?p=8458225&postcount=280

So right now it's a catch-22:

Can get root (temporarily) via exploit

Can't make permanant changes to /system (or anywhere else that matters)

Can't modify bootloader (h/w write protection)

Cant' load unsigned roms (bootloader)

Once HTC provides kernel source, there may be a way to interact with the NAND write protect commands to unlock various partitions. But until then we're kind of screwed.
[G2 and Desire Z Android Development] biosehnsucht:
I've looked through the init.rc and init.vision.rc in the two sets of g2 related system images I've got (posted as g2images.zip and PC10IMG_Vision_TMOUS_1.19.531.1_Radio_12.21.60.09b_26.02.01.15_M2_release_149459_signed) and neither does anything like nuking / reloading the OS, so it's definately happening in the bootloader before the kernel is booted.

Also, there are files / commands referenced in both of those that do not exist in their boot (normal or recovery) ramdisk images or system images, for example in rc.init :

service flash_recovery /system/etc/install-recovery.sh
oneshot

So I suspect that neither of those dumps are really "complete", or whatever would need flash_recovery (some kind of update?) is expected to bring it's own copy. If they're not complete then the "real" system images must contain these, and these dumps just have the basic filesystem present and the bootloader then copies things on top of them when missing/modified (and nukes things not on the approved list)? Replacing / nuking files is similar to how Tivo protects its systems from tampering, except that this is done in the init process of the kernel startup not during "BIOS" (pre kernel) boot. Eventually it was discovered that Tivo's bootloader ROM was only checking the signature of the kernel itself, and not the ramdisk... so the ramdisk got zero'd and the proper root= and init= commands were set in the kernel command line instead to boot from the HDD without doing the initrd. Sadly, that does not look viable here.

So we do need either:

A: compromised bootloader (pre-kernel)

B: compromised signature-check (find a way to change things while letting it still pass the signature check)

C: access the "sane" files partition that the bootloader uses to restore and replace them (and their accompanying signatures, if they're not automatically generated)

I don't have a G2 myself yet (just a G1) so I'm kinda hands-tied here as to what I can muck around with.

The PC10IMG... set has an hboot file and I was wondering what was in it, but I can't find much about .nb0. Not sure if its some kind of boot code + filesystem/etc or just flat code or whatever. I found some references for nb0 on windows devices but I tried one of those and it failed due to not finding whatever magic it was looking for.

The hboot is small enough that I doubt it has the signature checks in it for anything beyond kernel code, filesystem stuff is probably separate in the "sane" filesystem it uses to overwrite modifications.

This of course is all assuming that the above assumptions of how things work are true...
[G2 and Desire Z Android Development] biosehnsucht:
The NAND isn't write protected, or not entirely - init.rc mounts /system first as read-write so that a filesystem checkpoint can be created then remounts it read only.

# mount mtd partitions
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount yaffs2 mtd@system /system
mount yaffs2 mtd@system /system ro remount

# We chown/chmod /data again so because mount is run as root + defaults
mount yaffs2 mtd@userdata /data nosuid nodev
chown system system /data
chmod 0771 /data

The problem appears to be that the bootloader is overwriting any changes made to /system and so forth at boot time, with what it wants to see.
[G2 and Desire Z Android Development] HamNCheese:
I think it's been shown pretty conclusively that iNAND is at least capable of this. It looks as if we have to ask for permission to write and commit to non-volatile storage. There's no evidence that it was ever written in the first place.

I'm not sure I buy this. The time it would take to scan and potentially re-write every file the system would take way longer than it does to boot. And again, we have no proof it was ever written to begin with.

This whole thing reeks of a live cd scenario. Sure, you can "change" files, but they won't persist. No writing or scanning necessary. If it's a persistant image, linux keeps a delta of what has changed. The same tactic is used in WRT type devices, you don't actually change the file, you change a reference to it.

All of this is just wild speculation, but there are a few things that are clear:

* There is a LOT of space "missing" from our precious 4gb (likely recovery imgaes and whatnot)

* The iNAND write protection is perfectly capable of preventing us from writing to /system, the boot loader, or any other sensitive partitions

* We either have to break in and overwrite the bootloader, or go through the front-door (iNAND or whatever) to write to /system.

Once the kernel source becomes available, we will see how block devices that are presented to the kernel as being r/w are reverting to their previous state upon reboot.
[G2 and Desire Z Android Development] sino8r:
I think we need to try and spoof an ota much like MTS root which is based the unrevoked method. The only difference is we'll have to root it first. And then push the update. I understand those leaked eng builds aren't complete but can we modify them somehow to fool the NANDs checklist? I just don't see having to what on a stupid kernel source that will take for ever to be released. I mean the MTS bluetooth drivers weren't ever released and its over 4 months old now. We have to think of something else. Well a real ota will work, I imagine. We should ask around for another eng build and not these system dumps that aren't very helpful. Well they might not be dumps but they are definately incomplete and/or don't match the current build well enough to fool the NAND.
[G2 and Desire Z Android Development] philosophics:
Just askin a simple question that kinda sorta sounds like an eta so please don't flame me cause I miss cm settings like the rest of you.

How long does it usually take for htc to release the kernel source?

Sent from my HTC Vision
[G2 and Desire Z Android Development] sino8r:
Generally, a couple months. A ota is our best bet... or a complete engineering build leak.

I'd say it'll be a month at the latest before we get permanent root.
[G2 and Desire Z Android Development] biosehnsucht:
I've been googling this eMMC thing a bit, and it can be nefarious. Apparently the boot loader CAN set up an area to be denied access and deny it.

Came across this thread :

"Patch: eMMC boot partition needs to be deactivated for linux to find user partitions"

http://www.mail-archive.com/linux-mmc@vger.kernel.org/msg01423.html

This may not apply since we are probably booting from eMMC (but depending on what all is included in the 'boot partition' and how inaccessible it is after boot may be an issue):

Evidently, boot partition is switched off (or unused) since obviously Linux boots on this thing already and finds partitions. However we don't know how big the "boot partition" is set - what if the entire "missing" space is in the "boot partition" (possibly containing both the actual hboot, and all the restore data)?

I'm reading through the eMMC stuff now... (free registration required to DL PDF):

http://www.jedec.org/standards-documents/docs/jesd84-a441
[G2 and Desire Z Android Development] kholk:
So...if it's Linux that is locking the eMMC partitions, we can overwrite the Linux kernel with a custom loader that redirects the kernel address to a partition on external SDCard or make the phone to boot via USB (on Qualcomm this is possible, but I don't know if our BL allows that) and we won't have a locked eMMC.
[G2 and Desire Z Android Development] lbcoder:
I believe that it would be prudent for you to cease speculating that information is speculative. The information is confirmed, that is why I posted it as if it had been confirmed.
[G2 and Desire Z Android Development] lbcoder:
No, it can't be a matter of being write protected since the new data becomes readable from the disk. If it was write protected, that would have the same effect as being read-only. In addition, a permanent write-protect would prevent them from ever updating the OS.
[G2 and Desire Z Android Development] lbcoder:
How come the data is readable then when you write it to the /system partition?
[G2 and Desire Z Android Development] lbcoder:
Did you WRITE anything to the /system partition after the temp root?
[G2 and Desire Z Android Development] lbcoder:
You have a bunch of code there that isn't applicable. mtd/yaffs2. The filesystem on VISION is mcc/ext3.
[G2 and Desire Z Android Development] theclueless:
maybe the r/w value on the csd register can be flipped on bootloader during official update. making any parts of the nand perm read-only doesnt make sense (maybe bootloader and recovery can be make perm read-only, to ensure no one can write and screw it up and always able to recover)

since we know any change to /system will be flushed, so maybe theres a checksum with the backup rom (read-only on boot?) and a checksum with the actual rom being generated during boot, any mismatch will force the backup to overwrite the rom
[G2 and Desire Z Android Development] :
Anything that gets written to system is reverted right?

What happens when things are removed from system?

edit: has anyone monitored one of the updates to a gapp(new car home for example) to see how it's succesfully changing /system to apply updates?

After it updates, remove it from system and see if the reverted version is the updated version, or the original shipped version.
[G2 and Desire Z Android Development] xile6:
If I was htc I would make a prem readonly backup. That y no matter what happens u can always restore day 1. Now on updates maybe they store the update on a normally read only partition the can only be writen to when there is a update. Which then makes a live os on each boot?

So the main os is on a read-only part and it makes a live os of itself after its been flashed. When we get an ota update we will learn more. This is just my thought on it.
[G2 and Desire Z Android Development] xile6:
I removed system/app

Just for the fun of it an it all came back on boot
[G2 and Desire Z Android Development] lbcoder:
Updates to system applications are installed to DATA partition -- original remains on the system partition.
[G2 and Desire Z Android Development] lbcoder:
There is nothing "live" about it. The system partition is mounted, you can read and write to it, it just gets all wiped clean after rebooting. A livecd creates a RAMFS overlay to a read-ONLY filesystem. There is no RAM used when you change the system partition, so it can't be a ramdisk.
[G2 and Desire Z Android Development] grankin01:
Has anyone found a program or script that acts like squashfs on the live linux distros like backtrack 4. If this is what is happening there should be a partition that is referenced by the os every bootup sequence to apply user updates. This type of "persistent changes" os layout has been used for years. I was wondering when they would wise up, modify it and use it to "lock down" the os.
[G2 and Desire Z Android Development] lbcoder:
All updates are handled by recovery. A dump of recovery would be nice ;)
[G2 and Desire Z Android Development] grankin01:
I was under the impression that recovery handled system (rom) updates not app (user) updates. If someone deletes app and/or app data then reboots would rewrite the changes to the apps given recovery controlling app updates as well. Somewhere the system is accepting writes. Finding this within this unusual os/hardware layout might give us a loophole or two to explore.

And yes, a dump of the recovery would be great. :)
[G2 and Desire Z Android Development] ace42588:
what software methods are available for dumping recovery?

what i mean by that is: are there any feasible methods that wont involve having a manufacturer connection or hardware hacking?
[G2 and Desire Z Android Development] sarconx:
I've tried to mount the recovery partition but have been quite unsuccessful.
[G2 and Desire Z Android Development] grivad:
Just a thought..

The G2 and Desire Z are pretty much identical hardware-wise, right? The G2 doesn't have the Fast Boot that HTC has shown off with the Desire Z though..

Could it be that, with the G2 being a T-Mobile branded device, T-Mobile didn't want hax0rs rooting the device and enabling tethering to use their superfast HSPA+ network, for example, so HTC made it tricky to root by using this second partition to restore the system on boot? And the Desire Z is actually using this second partition for the Fast Boot..?

I might be way off. Just curious ; )
[G2 and Desire Z Android Development] philosophics:
Old news mang. Just continue trying to figure things out for us and people like me will continue to stay on the sidelines making sure things are going as they should.

Carry on and no more get back games. We're all adults here, I hope...

Sent from my HTC Vision
[G2 and Desire Z Android Development] theclueless:
recovery does not handle user app updates (even apps once stored in /system, they are gonna be stored in /data once updated via market)

i do believe the none of the sections on the nand are in perm read mode, it defeats the point of if the loaded rom gets modified, it should revert to last official updated state, not the original shipped state

htc likes to give bootloader updates; it is just a matter of finding how ota are giving signals to the csd to enable write