yashsingh1/xdaforums huggingface dataset:
[HTC Pico (Explorer)] no.human.being:
HBOOT runs on aARM so it should see the same "virtual" Radio that Android (also running on aARM) sees. There's a partition dumping feature in HBOOT but it will refuse to dump "radio", most likely because it doesn't have access to it. I assume that Radio updates aren't directly installed by HBOOT, but instead somehow "transported out" and then installed by Iguana on mARM.
[HTC Pico (Explorer)] #define:
Ok... so the talking between hboot and radio is done by a "3rd Party"(Iguana in WFS) right?? Ok... So if we run HBOOT inside an ARM emulator, what about the references that it makes in the misc and radio partitions?? Can we use the radio dump we get as a kernel image with qemu??

Also, we've mapped the radio, but dumping it gives a read error after writing some 6 Mbytes to file... what could be the reason there??
[HTC Pico (Explorer)] Sachin Thomas:
If we can dump at least a part of the radio, where HBOOT begins (by searching for the byte sequence "12 00 00 EA 31 2E 30 33"), we can extract it and chainload it with NHB's exploit and see whether it locks up.
[HTC Pico (Explorer)] #define:
The hboots would still be the same.. so it'd still lock up... i've tried.. it locks up even with a stock, completely unchanged hboot...
[HTC Pico (Explorer)] Sachin Thomas:
But that was an HBOOT from an RUU. I think it's more realistic to test with an HBOOT from our phone because there won't be any issues with partition layouts. Also, what if Iguana makes some changes to the RUU HBOOT image before flashing it?
[HTC Pico (Explorer)] #define:
Possible.. But does it have a reason to change a stock hboot provided by htc?? Only 1 way to find out.. we get the radio dump...

@sachin - what about dd instead of cat?? To dump radio?? Acc to munjeni, that should work...
[HTC Pico (Explorer)] #define:
Wow!! Definitely much better than any of my professors!!! :D

Ok... so on the emulator, we just need to focus on running the hboot... right?? When i do that on qemu(qemu-system-arm) it asks me for a kernel image... now since this is arm, a linux kernel wouldnt work... right??
[HTC Pico (Explorer)] Sachin Thomas:
Is there any way to "simulate" the physical button presses (volume and power) in QEMU because if we don't press volume down while booting the HBOOT will try to boot directly into ROM (of course, only if it reaches this point without "dying")
[HTC Pico (Explorer)] #define:
If it reaches the point of booting into the rom, then there's not gonna be the need for a vol down button, as we can just replace or eliminate the routine that makes the phone reboot into the rom...
[HTC Pico (Explorer)] Sachin Thomas:
OK... BTW I tried to dump radio with dd and it stops after writing 1 MB. I tried with various block sizes 2048,4096....131072 and parameters conv=noerror,sync (they're supposed to skip bad sectors) etc and it still doesn't work
[HTC Pico (Explorer)] #define:
So dd for android is in fact, broken..

But now that we have a working debugging facility, we should first try and see if it even needs radio, whether if we can make the radio references skip...
[HTC Pico (Explorer)] #define:
Has anyone else tried the emulator for the Pico?? I'm having some trouble initializing it at the right place.. i made the "qemu.bin", using the two chainloaders, and when I use gdb, the breakpoints 2 & 3 seem to be "stuck" at 0x10068 : andeq r0, r0, r0...

Some problem with the outer chainloader, I suppose..
[HTC Pico (Explorer)] Sachin Thomas:
Did you set the length of outer chainloader to 0x000C0088 and that of inner chainloader to 0x000C0000? "andeq r0, r0, r0" is actually a NOP instruction in ARM
[HTC Pico (Explorer)] BeciMester:
Great progress! This is truly a big step forward! I'll be following this curiously. I hope the hboot won't stall and qemu can emulate the coprocessor calls from offset 0x410 onwards.

Looks like you've hit a bunch of 0x00's.

(I'm on my phone...)
[HTC Pico (Explorer)] #define:
Yes... one of the first things i vross checked.. and the length and other addresses are set correctly....
[HTC Pico (Explorer)] #define:
So whats the way out??
[HTC Pico (Explorer)] #define:
Because of these 2 inconsistencies, i dont get the 0x50 at the beginning,i get 0x0...
[HTC Pico (Explorer)] Sachin Thomas:
How did you create them? NHB said that we should use the plain binary only, not an Android boot image
[HTC Pico (Explorer)] no.human.being:
I'm gonna upload my binary so that you can cross-check. Password is "as always" but I'm gonna send it via PM nonetheless. ;)

0x00000000 - 0x00000087 is the "outer" chainloader.

0x00000088 - 0x0000010f is the "inner" chainloader.

The HBOOT image begins at offset 0x00000110.

The 0x0 - 0x10f code should be almost identical, just that you should have the following byte sequences:

@0x0000000c: 88 00 0c 00 (= 0x000c0088) instead of 88 00 08 00 (= 0x00080088)
@0x00000097: 00 00 0c 00 (= 0x000c0000) instead of 00 00 08 00 (= 0x00080000)

And of course your "hacked" Explorer HBOOT image instead of the WFS HBOOT image I used. Rest should be fine.

And yes, QEMU seems to be able to handle the coprocessor calls, but GDB doesn't seem to handle virtual memory. As soon as the MMU comes on, you cannot debug anymore. I hope that there's some workaround for this, otherwise this won't be too helpful.
[HTC Pico (Explorer)] #define:
I have 00 00 0c 00 (= 0x000c0000) instead of 88 00 0c 00 (= 0x000c0088)

The other sequence matches.. I tried correcting it directly in the hex editor, but using that and running :

$ arm-none-eabi-gdb
(gdb) info registers pc
The program has no registers now.
(gdb)

I've checked the addresses in the outer_chainloader.s file, and they seem to be correct.. I've ofcourse adjusted the values for Explorer...
[HTC Pico (Explorer)] #define:
Using "arm-none-eabi-as" and "arm-none-eabi-objdump"..

Like we do with the chainloader exploit... But with three files, instead of the usual 2..
[HTC Pico (Explorer)] Sachin Thomas:
Three files? I meant there's no need to use mkbootimg_wfs here
[HTC Pico (Explorer)] no.human.being:
Yep three files.

cat outer_chainloader.bin inner_chainloader.bin hboot.img > qemu.bin

Right, you run QEMU with "-kernel qemu.bin".

@bgharz16: You should use 0x000c0088 on the outer chainloader as it moves the concatenated "inner chainloader (0x88 bytes) + HBOOT (0xc0000 bytes). It probably won't go too wrong if you get a bit shorter since the last instructions of HBOOT are probably no-operations, but 0x000c0088 would be the correct size for the outer chainloader's payload (that is "inner chainloader + HBOOT") while 0x000c0000 is the correct size for the inner chainloader's payload (that is HBOOT only). Just for correctness.

We're that close to the machine now, we shouldn't start getting sloppy. ;)

---------- Post added at 04:24 PM ---------- Previous post was at 04:17 PM ----------

Your debugger doesn't seem to have a target to debug. You need to connect the debugger to the ARM emulator for remote debugging first. To do that type "target remote :1234". This will only work when the ARM emulator was started with the "-s" (remote debugging server) option.

When you do that, the debugger should respond like this.

(gdb) target remote :1234
Remote debugging using :1234
0x00000000 in ?? ()
(gdb)

Now you can use debugging commands.

(gdb) info registers pc
pc 0x0	0
(gdb)

Hope this helps.
[HTC Pico (Explorer)] #define:
Damn.. Forgot to give target... Pretty stupid..

EDIT : I cross checked the values again, just to make sure, and they seem to be correct at my end, on both the inner and outer chainloader.. Same output as before..

0x10068...
[HTC Pico (Explorer)] #define:
Alright.. I got to the "8xi.." command successfully.. Means its getting loaded and initialized correctly...

EDIT : When I use "run" in gdb, it goes into some sort of infinite recursion.. And says it "cannot access memory at address 3fffffff8".... So I'm guessig this is related to the virtual memory as NHB stated earlier..
[HTC Pico (Explorer)] no.human.being:
So you now have an HBOOT loaded at physical zero and the debugger trapping there? Nice. :)

You can debug (set breakpoints, continue, break, single-step, inspect registers, etc.) until the MMU is set up. Virtual memory seems to mess things up. I'm not exactly sure WHY things get messed up though. I assume that the ARM emulator does everything right and that the problem is on the debugger side "only", even though I'm not exactly certain yet.
[HTC Pico (Explorer)] #define:
So now we have to solve the virtual memory problem on the debugger side.. It seems that it doesnt take the full virtual memory into consideration...

After that it'll be pretty easy(i hope) and soon enough we'll know exactly where we're having problems in the hboot.... on both the phones!! Now thats what you call an almost joint effort!! ;)
[HTC Pico (Explorer)] no.human.being:
Well "easy". Let's see. ;)

There won't be any peripherals (and no Iguana ;) ) that HBOOT can talk to. We can principally specify NAND flash and an SD card and even serial and USB controllers, but it's quite unlikely that they appear at the same address as they appear on an MSM7227/MSM7225A, right?

I've searched the QEMU documentation and source code for a memory map but couldn't find any yet.
[HTC Pico (Explorer)] Sachin Thomas:
The memory map for default configurations are specified in "/qemu-1.3.0/hw/versatilepb.c" and "/qemu-1.3.0/hw/integratorcp.c"
[HTC Pico (Explorer)] no.human.being:
Both boards are ARM926EJ-S boards, but we specified "-cpu arm1136". Are you sure that QEMU uses a board that has a different CPU than is currently emulated?

EDIT: Ah, I found it.

188 if (!args->cpu_model) {
189 args->cpu_model = "arm926";
190 }
191 cpu = cpu_arm_init(args->cpu_model);

So CPU only "defaults" to arm926 but is "replacable".

So seems like if we wanted to emulate the MSM7227 or MSM7225A memory map, we'd have to modify the QEMU sources. Sounds complicated and error-prone.

EDIT2: Btw, for the MSM7227:

0x00000000 - 0x3fffffff is reserved for DRAM on EBI1, but only 0x00000000 - 0x1fffffff is actually used, this is basically aARM's RAM.
0x40000000 - 0x7fffffff is completely unused.
0x80000000 - 0x8003ffff is IMEM, a fast on-chip SRAM for communication between aARM and mARM.
0x80040000 - 0x87ffffff is completely unused.
0x88000000 - 0x9fffffff is for peripherals (not memory) on EBI2, most of this is probably unused.
0xa0000000 - 0xafffffff is for other on-chip subsystems. (*)
0xb0000000 - 0xbfffffff is mARM exclusive subsystems.
0xc0000000 - 0xcfffffff is aARM exclusive subsystems.
0xd0000000 - 0xefffffff is NAND on EBI2, since this is 0x1fffffff in size, all of it is actually used.
0xf0000000 - 0xfffeefff is completely unused.
0xfffef000 - 0xfffeffff is mARM IRAM.
0xffff0000 - 0xffffffff is mARM boot ROM.

(*) The on-chip subsystems are as follows.

0xa0000000 - 0xa00fffff: GRP (??)
0xa0100000 - 0xa01fffff: TSIF (transport stream interface)
0xa0200000 - 0xa03fffff: UART (universal asynchronous receiver transmitter)
0xa0400000 - 0xa07fffff: SD controller
0xa0800000 - 0xa08fffff: USB controller
0xa0a00000 - 0xa0bfffff: NAND controller
0xa0c00000 - 0xa0cfffff: CLP (cryptographic lookaside processor)
0xa0d00000 - 0xa0dfffff: LCD controller
0xa0e00000 - 0xa0efffff: Peripheral bus controller
0xa8000000 - 0xaaffffff: Other peripherals
0xac000000 - 0xafffffff: aDSP

Open questions:

1) Where's the framebuffer?

2) How similar is the memory map for MSM7225A?

3) Note that NAND starts at 0xd0000000 physical. This brings me the idea that once we know where HBOOT is in NAND (should not be too difficult to find out), we might simply write there using assembly language (without a bootloader). However, I don't quite understand how to write to NAND for two reasons.

3a) ARM only has "load" and "store" operations, but NAND memory needs to be explicitely erased before it can be written. How is this erasure performed on the machine level?

3b) ARM "load" and "store" operations reads and writes a single machine word (4 bytes, 32 bits) at once. NAND memory needs to be programmed in 128 KiB pages. Mind that programming a page is an atomic operation. Once it is programmed, you must no further write to it without erasing it first. How to write 128 KiB in an atomic operation when "store" only writes one word at a time?
[HTC Pico (Explorer)] Sachin Thomas:
Does a bootloader and a recovery use the same technique to write to NAND? Is there any difference at the machine level?
[HTC Pico (Explorer)] no.human.being:
The actual "erase"/"write" procedure is probably the same. However, the "msm_nand" driver in the Linux kernel (CWM recovery is actually a small but complete GNU/Linux system) refuses to "erase"/"write" NAND pages that are flagged as "locked" (which the entire "unmapped area" is) and the "lock"/"unlock" feature is not implemented in the Linux kernel so the Linux kernel cannot remove that flag.

We didn't have a chance to test what would happen if the kernel were to ignore the "locked" flag and set off an "erase" command even though the page is flagged as "locked", since we don't have kernel devs that could perform such a modification. However, it would make sense that such a request were blocked by the MMU.

We also didn't have a chance to test what would happen if the kernel were to set off an "unlock" command to such a page, since we don't have kernel devs that could properly implement that command. However, the lock to the "unmapped area" is probably controlled by Iguana, so the MMU will probably only let mARM (not aARM) control these locks.

Therefore we strongly suspect that, in case HBOOT has to flash some partition that lies in unmapped area ("radio", "sp1", "sp2", "hboot", "mfg", "wifi"), HBOOT somehow "authenticates" with Iguana first, then Iguana clears the "locked" flags on these pages, then notifies HBOOT that it may now perform the update, then HBOOT erases the partition and writes the new content, then tells Iguana that it's finished and Iguana locks the pages down again.

However, this is all suspicion. We actually don't know what happens, but we believe that Qualcomm, being as paranoid about someone modifying their chipset's firmware as they are, have in fact implemented some sort of "authentication mechanism". If the problem were "just" that the kernel was paranoid about erasing protected pages or that it didn't have the "lock"/"unlock" feature implemented, someone would probably have implemented it and security would have been broken. However, this has not happened yet, so I suspect that aARM isn't permitted to "lock"/"unlock" NAND pages at all and therefore the Linux devs have decided not to implement these commands in the kernel. I don't have any proof though.

EDIT: Also mind that the "erase"/"write" implementation in the kernel might not be suited to be directly integrated into our exploit code since it probably has cross-references to many other functions and data structures in the kernel, while the bootloader usually uses much simpler, more standalone, lower-level code to accomplish these things. The latter will have less dependencies on other code and thus be a lot easier to integrate into our programs.
[HTC Pico (Explorer)] Sachin Thomas:
If the actual "erase"/"write" procedure is the same, we could "port" CWM recovery to QEMU and single step through the code while it is flashing to (virtual) NAND, and then try to write our own low level code. But it's of no use if the region is "locked" by Iguana :(
[HTC Pico (Explorer)] #define:
It seems to me that it would be locked... thats kinda the whole point of S-ON/S-OFF isnt it?? To prevent unauthorized write access to the locked regions?? Unless we get our exploits signed by HTC, or get our hands on an ENG HBOOT, successfully writing to NAND without S-OFF would be next to impossible... atleast i would hope so...
[HTC Pico (Explorer)] #define:
So, the best thing for us would be to work through the hboot itself.. as we have no jtag access, and qemu needs some serious modifications to run the hboot..

Just thinkng out loud here... what if we dont specify the cpu in qemu?? Let it "assume" the platform on its own, and see where that lands us.. i dont know if its been tried or not..
[HTC Pico (Explorer)] no.human.being:
If we can work around the virtual memory issue (which is "just" an issue with the debugger, not an issue with QEMU), at least I'd like to step through it and see how far it gets.

As far as I can see it will "default" to ARM926 and Versatile/PB. What would be the advantage (apart from being "even more unlike an MSM7227/MSM7225A")?

EDIT: We really have to find the point where HBOOT "reverts" (on the MSM7227) or "locks up" (on the MSM7225A). If we get an actual HBOOT image running we have the advantage that we don't have to care how NAND is unlocked/erased/written, because it'll "just work".

EDIT2: I've just found that UBOOT is not designed to be loaded from RAM, it must be loaded from NAND, which means it's an "install via JTAG and set up your system completely from scratch" type of thing.

This sounds a lot like: "If you want to run UBOOT from RAM, you have to adjust such a big amount of code that you could just start over writing your own bootloader from scratch." :D

Installing UBOOT via JTAG does not solve the problem. If you have JTAG access, you could even write a new Iguana image to Flash, so there will be easier ways to achieve S-OFF. :D

So, unfortunately, UBOOT is probably not the way to go either. Damnit. :( :( :(
[HTC Pico (Explorer)] #define:
My point exactly... If we can just run an HBOOT on qemu, successfully, that'll solve almost all our problems..
[HTC Pico (Explorer)] no.human.being:
You forgot one thing. If we can debug an HBOOT on QEMU with GDB, that'll solve almost all our problems. :D
[HTC Pico (Explorer)] #define:
Ofcourse... I stand corrected... ;)
[HTC Pico (Explorer)] Sachin Thomas:
I only meant that running CWM in QEMU would be easier, because it is portable across architectures, and doesn't need Iguana and the "class one hypervisor" stuff. But if it comes with a downside ("abstractions" and "indirections") then it's not a solution :(
[HTC Pico (Explorer)] no.human.being:
I don't think that the CWM binary is portable as there's a separate binary for each device. It's just that it supports many different architectures and can be built for a lot of devices using "conditional compilation". I don't think that Versatile/PB is among the supported architectures as this is an evaluation board only. ;)
[HTC Pico (Explorer)] #define:
Hey.. Been cold for a few days...

BTW, Merry Christmas everyone!!!! :D
[HTC Pico (Explorer)] #define:
http://xdaforums.com/wiki/CASUAL

Could this be a possible future project after S-OFF??
[HTC Pico (Explorer)] #define:
Hey... Did anybody try the "-mem-path" and "-mem-prealloc" tags for "qemu-system-arm"?? Might help solving the "memory" problem...
[HTC Pico (Explorer)] no.human.being:
This is just to specify that it should allocate memory from files on the drives. The problem is not a problem with QEMU's memory. The problem is that GDB is not aware of the page tables. Like I said it's a GDB-only issue. You can still "halt" and "continue" from GDB and if you do that, you'll see that the HBOOT code is still running (program counter value changes). QEMU can and does still run the code, it's just that GDB is no longer able to debug it and I bet that this issue would occur with OpenOCD on an actual target (phone with JTAG) as well.
[HTC Pico (Explorer)] #define:
Dang.. so what now??
[HTC Pico (Explorer)] no.human.being:
I don't know. I'm researching. There's probably a way around this issue somehow. GDB is a GNU tool and these were built for debugging GNU/Linux. A Linux kernel makes extrensive use of memory virtualization. Remember that a Linux kernel image usually has the file name "vmlinuz". The "vm" is used for historical reasons and designates that the kernel makes use of "virtual memory".
[HTC Pico (Explorer)] Sachin Thomas:
Today I extracted an HBOOT from my own phone (it starts at offset 0x02B00000, just like WFS HBOOT). I expected that an HBOOT from my own phone will match my partition layout. But, to my surprise it also displays LOCKED instead of UNLOCKED :confused:

If the physical address where it reads the "locked/unlocked/relocked" state is hardcoded in the HBOOT itself, it should read the flag at the correct address in the misc partition and display UNLOCKED. But this doesn't happen in our case.

A LOCKED bootloader won't even allow us to use fastboot commands :(
[HTC Pico (Explorer)] #define:
I was thinking, that if the bootloader in the phone's memory is hardcoded to be Locked by default, then a possible explaination could be that it loads the bootloader, and then checks in the misc partition for the lock state(whether unlocked, relocked, or locked) and then displays it on the screen... And since you pulled it from the phone, and then chainloaded it, it doesnt get to make the misc partition check.. And hence displays the default hardcoded state Locked state..

Also, some people wanted me to ask you how you pulled the HBOOT from the phone.. As they have newer versions and wanna post them here..

Is it the same way we mapped the radio??
[HTC Pico (Explorer)] Sachin Thomas:
Why doesn't it get a chance? Does it get locked up before that? The chainloaded HBOOT overwrites the default boot code in memory and should definitely read the misc partition

Yes... They should calculate the starting addresses of all partitions, because they are probably different for newer revisions of the phone. The offset where HBOOT begins may also be different. I think it's "Size of Radio"- 0x004c0000 but I'm not sure

It's 0x02b00000 in my case so I used

fastboot -c "mtdparts=msm_nand:0x00100000@0x1ff00000(misc),0x00500000@0x02fc0000(recovery),0x00400000@0x034c0000(boot),0x0eb40000@0x038c0000(system),0x03b00000@0x12400000(cache),0x09600000@0x16900000(userdata),0x00a00000@0x15f00000(devlog),0x000c0000@0x02b00000(hboot)" boot recovery.img

cat /dev/mtd/mtd7 > /sdcard/hboot.img

EDIT:

Use the attached C program to generate the parameters for mapping radio and HBOOT. Note that mapping HBOOT is experimental because I'm not sure whether it always starts at "Size of Radio"- 0x4c0000

View attachment calc.c
[HTC Pico (Explorer)] #define:
Actually, the why isnt answered yet.. If we solve the virtual memory allocation problem in GDB, then we'll know why and how we can avoid it.. NHB is working on it...

The HBOOT locks up because the processor enters some sort of infinite loop, before it can properly initialize the HBOOT...
[HTC Pico (Explorer)] no.human.being:
The infinite loop is in the HBOOT code itself. Of course it's not designed to be an infinite loop, but it's busily waiting on some condition that is never met. I remember someone saying in the WFS thread that synchronization on the MSM7227 (and probably also on the MSM7225A) is often done via spinlocks. A spinlock that can never be acquired (because another thread/process never releases it) will be busily waiting forever, e. g. if the ressource is still "locked" by the "old" HBOOT.

You're right and on the WFS it actually does just that. The WFS HBOOT displays "unlocked" (or "injected" since we've replaced the "unlocked" string with that to distinguish a "hacked" bootloader from a "stock" one). If the locking should actually turn out to be problematic (stay after the other issues have been fixed) we might consider patching the routine that checks the lock status to always return "unlocked", just like we patched the routine that checks for the secu_flag status to always return "S-OFF". It's a bit "dirty", since it doesn't allow you to lock the bootloader again. I think you need a locked bootloader for installing RUUs, right?
[HTC Pico (Explorer)] Lt.Win:
Yes, you do. A locked or relocked anything will do.
[HTC Pico (Explorer)] Sachin Thomas:
Where does an HBOOT store information about locked resources? Can't we force it to release all resources through the exploit code before starting to execute the boot code?