yashsingh1/xdaforums huggingface dataset:
[HTC Wildfire S] no.human.being:
Exactly.

Of course I'll do it in C. :D

No real other option, really. Don't want to supply the Dalvik VM with the OS. I'm not even certain if the APIs are accessible via Dalvik, probably not as they're considered "too low level".

Yes it's bad. There's an API that's a lot better and supports random access to NAND (instead of just sequential) allowing us to read/write single pages (2048 bytes) and to overwrite "in place" (the driver does the "read-modify-erase-write"-cycle), but the CWM kernel does not seem to implement it (CM7.2 doesn't implement it either).

Right.

No we're not. HBOOT has "moved" too so I'd not be too certain that anything remains in the same place between two Radio versions.

Perhaps I should mention that the Radio firmware will contain the device's IMEI (and possibly other "identifying information") one could use to "clone" the device. Certainly I wouldn't want to have anyone running around with my IMEI, having the police knock at my door saying: "Mr. no.human.being, we found your mobile phone being used for arranging drug deals." :D

You'll find recovery images here.

@theq89: Your "unmapped memory" actually seems to be smaller. At 0x026c0000 you'll find that your recovery kernel begins and at 0x02b40000 it ends so your complete recovery partition is included as well. *lol*

Then at 0x02bc0000, there's another Android kernel header, so is this already your main kernel? :D

Then at 0x02f00000, it looks as if a file system begins. Is this already the beginning of your "/system" partition? :D

Oh man, seems your partitions are really really small for some reason.

However, the general layout always seems to be: Radio firmware - endless wall of 0xff - HBOOT - wall of 0xff - WiFi firmware

Now at 0x02c401d5 (with me still "unmapped memory") I find the string "ccode=ALL". In the HBOOT code there's a "table" that maps the values "ALL" and "11111111" together. As we know "11111111" is the SuperCID. So this might be where the CID is stored, but that is just a guess. :D
[HTC Wildfire S] theq86:
thanks for making me 3 years younger :D

how can my image include cwm kernel and android kernel? I mapped my radio just as you did, same kernel params. and I'm sure to have only dumped the radio. also, our radio files are the exact equal size. so wtf ?

EDIT:

how can I find out how my partition layout really is?
[HTC Wildfire S] :
I don't want it, should I get it?

It's available for me.

Right. You think you could find the rest of the signature recognition stuff and delete it. That way flashing ROM_01 would be easy!

BTW, what country are you in?

And what's your name?

Hmmm.. You're Scottish. I need a Scottish accent. Scottish accents are cool. And so are Welsh!

Anyway, when we get S-OFF, will we have to downgrade our bootloaders so that HTC can't see we've upgraded them and unlocked them; when getting it fixed?
[HTC Wildfire S] no.human.being:
Yes, you've dumped the same physical address space, but it seems that your Radio is smaller, therefore all your other partitions begin earlier. And it's actually so much smaller that 2 entire partitions + the beginning of the third were included in the dump. :D

You can do a "cat /proc/mtd" while booted into the ROM that's normally running on your phone. Obviously, this one has to know the "correct partitioning", otherwise it would mess up your phone.

The memory layout (by ascending physical address) is: "radio" --> "recovery" --> "boot" --> "system" --> "cache" --> "devlog" --> "userdata" --> "misc" (I really hope this is universal :D )

Remember the memory layout has nothing to do with the partition numbers. The partition numbers if ordered by ascending physical address are actually: mtd7 --> mtd1 --> mtd2 --> mtd3 --> mtd4 --> mtd6 --> mtd5 --> mtd0 (Lol! This is just crazy! Who was it that designed this sh*t? If I were to create the partition layout for a phone of course I'd number the partitions by ascending physical address, but of course the software engineers at HTC do it differently. :D )

However, when changing the mapping, you have to pass the parameters by ascending partition number, not ascending physical address. So at least the kernel devs were logical, however, this means you have to do a bit of additional work and reorder the list by partition number before passing it to the kernel. :D

The highest physical address on the NAND is 0x1fffffff (this is definitely universal :D ). Then you can do "backwards chaining".

(0x1fffffff - "size of misc") + 1 = "where misc begins"

"where misc begins" - "size of userdata" = "where userdata begins"

"where userdata begins" - "size of devlog" = "where devlog begins"

"where devlog begins" - "size of cache" = "where cache begins"

"where cache begins" - "size of system" = "where system begins"

"where system begins" - "size of boot" = "where boot begins"

"where boot begins" - "size of recovery" = "where recovery begins"

"where recovery begins" = "size of radio"

Radio always begins at 0x00000000 and ends at ("size of radio" - 1).

Remember this is all hexadecimal (the numbers in "/proc/mtd" are also hexadecimal).
[HTC Wildfire S] heavy_metal_man:
Nope not quite :D I'm actually northern Irish lol. Moved here for the view and the women lmao, but that's a different story ;)lol
[HTC Wildfire S] seaskyways:
Guys , i got this in the fastboot oem boot :

this clearly means my radio is still unmapped , how am I supposed to dump it since i don't have it mapped , shouldn't i have your "currently being used" kernel to map it ? and if i have it mapped can i dump it using adb pull /dev/mtd/mtd7 ??
[HTC Wildfire S] BeciMester:
Whoa, different partitions in different radios? Interesting. If I remember well, it's up to the hboot if the radio can be dumped or not. I'll just update my hboot then first, HTC-unlock and dump my old radio, just out of interest, to see the differences. Then upgrade radio and dump that too. :) Later, XTC-clip the newest radio, dump and compare.

(I'm on my phone...)
[HTC Wildfire S] seaskyways:
Hey , if I am S-OFF , could I use the fastboot oem eraseconfig , because my misc partition is messed up and it is causing terrible problems at least for me ???
[HTC Wildfire S] no.human.being:
Actually I'm using a very old CWM release (I think it's a 4.something iirc) for this. It'll probably work with every CWM image. It's just that I have this CWM release in my SDK folder so it's quick to boot. Don't ask me why I have it "lying around" though, I did never install it. I'm having a much more recent CWM installed on the device. :D

The mapping is done by passing parameters to the kernel via fastboot. There's absolutely no "magic" here. Here I'm explaining how to map and dump it and here I explained how I got to the parameters in case you want to calculate them for yourselves.

No, you cannot "pull devices". You have to cat it into a regular file and then you can probably pull the regular file. I did cat it to the sd since there's a great amount of storage available so most of the time you don't have to worry about whether there's enough free space available (the Radio is actually quite large).
[HTC Wildfire S] Ko_Ka:
I see your in Dublin, if its any help I got mine done in Chinese shop/net cafe (kind of wooden interior) facing Fibber Magees on Parnell street, 20 squid....
[HTC Wildfire S] pihug12:
Here is my phone (HBOOT-1.09.0099 + RADIO-7.54.39.28M) :

$ cat /proc/mtd
cat /proc/mtd
dev: size erasesize name
mtd0: 000a0000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00340000 00020000 "boot"
mtd3: 10d60000 00020000 "system"
mtd4: 02300000 00020000 "cache"
mtd5: 09600000 00020000 "userdata"
mtd6: 00a00000 00020000 "devlog"
$

I hope theq86 has the same addresses... :-x

Here is no.human.being's phone (HBOOT-1.08.0099 + RADIO-7.53.39.03M) :
[HTC Wildfire S] BeciMester:
Cheers, that's where I'm heading, the place was recommended on Boards (was it your post? :D)

Guys, should I let the phone boot into Android right after clipping? Will that make any unwanted changes? I'm fairly sure it won't be a problem, but I'm ready to just boot it into fastboot and do the dump straight away if necessary. I don't want to screw this one up as it's a one-way street. :)

(I'm on my phone...)
[HTC Wildfire S] heavy_metal_man:
Already failed my dump let's not waste another eh ;)
[HTC Wildfire S] theq86:
Yes, exactly the same.

But I'm no good HEX calculator so I don't have the start addresses :(

EDIT: I now calculated the addresses for the latest configuration

/misc start:0x1FF60000 size:0x000a0000
/userdata start:0x16960000 size:0x09600000
/devlog start:0x15F60000 size:0x00a00000
/cache start:0x13C60000 size:0x02300000
/system start:0x02F00000 size:0x10d60000
/boot start:0x02BC0000 size:0x00340000
/recovery start:0x026C0000 size:0x00500000
/radio start:0x00000000 size:0x026C0000

this leads us to this starting command

./fastboot -c "mtdparts=msm_nand:0x000a0000@0x1FF60000(misc),0x00500000@0x026C0000(recovery),0x00340000@0x02BC0000(boot),0x10d60000@0x02F00000(system),0x02300000@0x13C60000(cache),0x09600000@0x16960000(userdata),0x00a00000@0x15F60000(devlog),0x026C0000@0x00000000(radio)" boot recovery.img

gonna test mapping now !

dev: size erasesize name
mtd0: 000a0000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00340000 00020000 "boot"
mtd3: 10d60000 00020000 "system"
mtd4: 02300000 00020000 "cache"
mtd5: 09600000 00020000 "userdata"
mtd6: 00a00000 00020000 "devlog"
mtd7: 026c0000 00020000 "radio"

Yeah, it works. Now I gonna verify that these are the real boundaries.

@ nbh down there: thanks but I got it myself ^^
[HTC Wildfire S] no.human.being:
Ok, let's see.

0x1fffffff is the highest physical address on the controller, so ...

"radio": 0x00000000 - 0x026bffff (size: 0x026c0000)
"recovery": 0x026c0000 - 0x02bbffff (size: 0x00500000)
"boot": 0x02bc0000 - 0x02efffff (size: 0x00340000)
"system": 0x02f00000 - 0x13c5ffff (size: 0x10d60000)
"cache": 0x13c60000 - 0x15f5ffff (size: 0x02300000)
"devlog": 0x15f60000 - 0x1695ffff (size: 0x00a00000)
"userdata": 0x16960000 - 0x1ff5ffff (size: 0x09600000)
"misc": 0x1ff60000 - 0x1fffffff (size: 0x000a0000)

Ordered by partition number:

mtd0: size 0x000a0000, physical 0x1ff60000 ("misc")
mtd1: size 0x00500000, physical 0x026c0000 ("recovery")
mtd2: size 0x00340000, physical 0x02bc0000 ("boot")
mtd3: size 0x10d60000, physical 0x02f00000 ("system")
mtd4: size 0x02300000, physical 0x13c60000 ("cache")
mtd5: size 0x09600000, physical 0x16960000 ("userdata")
mtd6: size 0x00a00000, physical 0x15f60000 ("devlog")
mtd7: size 0x026c0000, physical 0x00000000 ("radio")

So in kernel parameters that's:

mtdparts=msm_nand:0x000a0000@0x1ff60000(misc),0x00500000@0x026c0000(recovery),0x00340000@0x02bc0000(boot),0x10d60000@0x02f00000(system),0x02300000@0x13c60000(cache),0x09600000@0x16960000(userdata),0x00a00000@0x15f60000(devlog),0x026c0000@0x00000000(radio)

For comparison, the table for my phone (older Radio) was:

0x00000000-0x02fbffff : "radio" (mtd7)
0x02fc0000-0x034bffff : "recovery" (mtd1)
0x034c0000-0x037fffff : "boot" (mtd2)
0x03800000-0x13bfffff : "system" (mtd3)
0x13c00000-0x15efffff : "cache" (mtd4)
0x15f00000-0x168fffff : "devlog" (mtd6)
0x16900000-0x1fefffff : "userdata" (mtd5)
0x1ff00000-0x1fffffff : "misc" (mtd0)

So my kernel was parametrized with:

mtdparts=msm_nand:0x00100000@0x1ff00000(misc),0x00500000@0x02fc0000(recovery),0x00340000@0x034c0000(boot),0x10400000@0x03800000(system),0x02300000@0x13c00000(cache),0x09600000@0x16900000(userdata),0x00a00000@0x15f00000(devlog),0x02fc0000@0x00000000(radio)

EDIT: Damn, someone was faster! :D
[HTC Wildfire S] theq86:
nhb, our calculations do not seem to match. ?!?
[HTC Wildfire S] no.human.being:
Y not? Both tables and command line look identical. Just that you use capital letters for hexadecimal digits greater 9 while I use lowercase. Doesn't matter, I just find it easier to read with lowercase letters.
[HTC Wildfire S] theq86:
yes, I saw it now.

I'll do following now just to be REALLY sure

cat every partition to the sd, and compare them with the cwm recovery made .img files doing a simple md5 compare

misc, recovery and boot are already matching now I try the larger partitions

edit: I'm stupid. obviously /data /cache /devlog are changing partitions. :D so no need to compare
[HTC Wildfire S] no.human.being:
As far as I see the newer Radio has less "misc" and "radio", but more "system" than the old one.

Size of "recovery", "boot", "cache", "userdata", "devlog" remained the same, but obviously even these partitions were "moved around" as they all come "after the Radio" and the newer Radio is smaller.
[HTC Wildfire S] pihug12:
You two doublecrossed me. You're too fast. :p

I made a small table :
[HTC Wildfire S] theq86:
but you got a nicer presentation, so you're good, too :D :D :D

7zipping my radio shrinks it from 40.632.320 Bytes to 526.859 Bytes lol

@nhb: I definately have NO kernel in my radio anymore, yeah!
[HTC Wildfire S] no.human.being:
My radio.img was 50.069.504 bytes original, 948.990 bytes gzipped, 783.514 bytes bz2ed, 528.991 bytes 7zipped. :D

And yeah we'll have to supply different pagetables and probably different offsets into the Radio to patch for different Radio firmwares. So this is gonna be quite some amount of work if we want to cover a broad spectrum. :D

But I think other than HBOOT 0.90.0000, 1.08.0099 and 1.09.0099 there's not a lot "floating around", right? So 3 versions of the hack is probably ok. Afaik the page tables can be integrated into the boot image, so the "end user" won't ever come in contact with it. He'll just download one of 3 images and execute it. Would be nice if we could somehow "detect" if we're "on the wrong firmware", as this could prevent some "bricking accidents" with people accidently downloading the wrong version of the exploit for their firmware.
[HTC Wildfire S] theq86:
can HBoot 0.9 take boot params when locked? o.O

We can indeed detect a wrong firmware. Same way RUU does, by looking up the software version in /misc.

Or we just read out the radio and hboot version from the radio partition itself. that would not exclude users with a changed /misc partition (like me, I always set software number to 0.0.0.0 to flash any RUU I want :D)

---------- Post added at 10:15 PM ---------- Previous post was at 09:55 PM ----------

@nhb

you found the android recovery source yesterday. can't we learn from that code how to talk to a mtd device in C? Just a thought

EDIT: I found something interesting

https://github.com/packetlss/amonra_bootable_recovery/tree/mahimahi/mtdutils

the function names in mtdutils.c sound quite promising and useful

mtd_erase_blocks, write_block etc.
[HTC Wildfire S] :
Nice work!

Sent from a Time Lord, via his TARDIS.
[HTC Wildfire S] MrTaco505:
Yea sure, that would be helpful
[HTC Wildfire S] :
Since you need a dump, instead of using a clipped phone, use a stock S-OFF phone.

Sent from a Time Lord, via his TARDIS.
[HTC Wildfire S] theq86:
"Security" is a bad joke

Using stock ROM and kernel ...

:D:D:D:D:D:D:D

---------- Post added at 11:41 PM ---------- Previous post was at 11:37 PM ----------

LOL know what it means? We could even make an S-OFF APK and upload to market.

Hell HTC has to bite in their asses :D
[HTC Wildfire S] no.human.being:
:eek::eek::eek::eek::eek:

WTF did you do to get the mtd7 mapped on a STOCK ROM? :eek::eek::eek:

Also: mtd0, mtd1, mtd2, mtd3, mtd4, mtd7?

Where did mtd5 and mtd6 go? :eek:
[HTC Wildfire S] theq86:
it's not that kind of magic, and of course the bootloader is unlocked, without it's definately not possible.

what I did is to take my boot.img from a ruu. instead of booting a recovery image I booted my boot.img with the respective parameters.

Yesterday I stated, that a standard kernel would NOT do. But of course it didn't, I used a crappy mtdmap

Now there I have the right mtdmap according to hboot and radio version I just tried it again.

That's not much of a big deal, I just replaced recovery.img and boot.img and started. now there the phone is fully booted it is the proof that our above mapping is 100% correct.

But, in fact this is a great step in usabiliy. If we are ready making the C exploit, I will write a native android APK using this exploit, giving the users the possibility to "switch" S-ON/OFF, (and maybe also SuperCID and SimLock) from within Android, with a fancy GUI.

EDIT: Well.I can not explain why those are not displayed. They are there. Maybe my screenshot app is crappy. gonna make a new.
[HTC Wildfire S] no.human.being:
Lol ok. We could also customize CM so that it doesn't display the phone's "desktop" but only "our app", then ship that OS image. Boot it, you can select CID (well .. why only SuperCID on/off, actually we could do a dropdown list where you could select any CID :D ), and S-ON/S-OFF.

But you're gonna have to do the GUI work. I'm not familiar with app development at all. Never compiled anything for Dalvik VM, just native code. :D

I'm don't think that our kernel exposes the required functionality. I think it only exposes the minimal API that can read/write sequentially. I think it doesn't even expose random access (fseek) for the NAND to the userland. It only uses the functionality internally (of course the file system driver doesn't overwrite the entire NAND when changing a single file) but I don't think it's exposed as an API.

Try opening "/dev/block/mtdblock*", it'll fail with an I/O error since there's no driver behind the device special file. And the "/dev/mtd/mtd*" only allow to read/write sequentially so we can't write to block 1000 without overwriting the 999 blocks before so we'll have to read it all, put it somewhere else (SD card?), erase it all, change whatever we want, write it all back.

If we want to have the other APIs exposed we'll have to build our own kernel. Neither CM7.2 nor CWM exposes the APIs and the Linux kernel devs consider them "evil" as it seems there was some "abuse" done via these APIs. They even consider removing the API calls from the kernel sources entirely so that you no longer have an option to enable support for them during compile time.

I think HBOOT 0.900.0000 can't boot via Fastboot at all. It can't flash via Fastboot either, that's why you had to do it via the PG76IMG.nbh on the SD. I could be wrong though.
[HTC Wildfire S] theq86:
So, the better one here.

Yes, I have some experience with the android sdk and native java apps.
[HTC Wildfire S] no.human.being:
Well I can code Java for the PC, but not for Android. I'm not familiar with the APIs (the native APIs are just Linux APIs but the Java API is "special").

I've written a tracking software for satellites in Java (was for the "big interdisciplinary research project" at our university), which displays the predicted satellite position on a mercator projected map (even the 2D library for rendering this was completely custom by me :D ), calculates angles for the antenna and communicates them to a "daemon" I implemented in C using a socket interface. The daemon then drives stepper motors via a microcontroller connected to the PC via RS232 for moving the antennas.

This enables us to "track" the satellite with our directional antennas while it flies over and keep the "RF beam" focused on it so communication is optimal.

Sadly, the orbital mechancs turned out to be too complicated to be implemented in the semester, so we left that a bit open. We'll probably take "GNU Predict" (an orbital mechanics "daemon" from the GNU project) for that purpose.

So now we're "waiting" for the address of the secu_flag for RADIO-7.53.39.03M and RADIO-7.54.39.28M and then we can begin writing some exploit code I'd say.
[HTC Wildfire S] theq86:
Well, I feel much more familiar on object oriented code. although I know C and C++

But nowadays they lack of some concepts making user applicactions to be written with less effort.

e.g. try to write a Windows, GTK+ or Qt Application.

Windows uses C and C++ and an API that never ever anyone could learn completely.

A new Win32 project just showing 1 window with Hello World needs 400 lines of code.

GTK is also C, has some C++ bindings (yes, and even mono libs)

but the framework itself is not better than the winapi :D

Only Qt is quite comfortable even in C++ :D

But that would now lead to a completely off topic discussion ^^
[HTC Wildfire S] no.human.being:
Well I'm not at all into "GUI applications". Normally I'm just writing "servers" and "daemons" and command line utilities. GUI code is such a mess. And when you have to do it, you better have a concept to keep it really really separated from the rest of the application.

Best is GUI in a separate process communicating via socket with the "actual application".

If that doesn't work, at least have it in a separate thread with a very small and well-defined interface between.

I'm actually doing most of my work in C#, but obviously for security-related and embedded stuff, C is the way to go. And for university stuff I stick with Java. It's almost exactly the same as C#, but it's GPL licensed and at our university you have to pass two exams in Java in the first two semesters as a computer scientist. So it's a language you can expect every computer scientist to be familiar with.
[HTC Wildfire S] theq86:
ok, back to topic.

Wouldn't it be much less effort to flash all the radio instead of maybe still having to write a kernel?

I mean, NAND wearing back and forth, S-OFFING is something you probably do just once, but not every now and then.
[HTC Wildfire S] no.human.being:
Yes but you suggested "mtdutils" and they use APIs that are probably not exposed on "Android" (and CWM is also "Android" in this case).

For example they open "/dev/block/mtdblock*" on some occasions, but these devices are just "dead" on the Android kernels. There's no "real" driver behind. Try to open any of them and you get an I/O error straight off. What's supported is "/dev/mtd/mtd*", but we can't just take the "mtdutils" and replace all the "block/mtdblock" by "mtd/mtd" since the two device drivers expose different functionality and "mtdutils" makes use of functionality that the Android "mtd" driver does not expose. So when we stick with the CWM kernel we probably can't use "mtdutils".
[HTC Wildfire S] theq86:
Well, but the flash_image utility (which depends on and uses mtdutils) works. It works on the android kernel, it works on the cwm kernel. I have tested both.

Sure, you can not just fopen, fseek, fwrite, fclose. But the mtdutils should work. otherwise I can not explain why flash_image does it's job. btw cwm uses flash_image probably to restore it's backed up files
[HTC Wildfire S] no.human.being:
But that's the semantic that a block device should expose. I can "fopen, fseek, fwrite, fclose" on the drives on my computer, why can't I do it on an embedded system?

Since you can't "cat /dev/block/mtdblock?" or "dd if=/dev/block/mtdblock? of=/dev/null count=1 bs=2048" either, I think the "mtdblock*" devices are just "dead" (not implemented).

If I could just "fopen, fseek, fread, fwrite, fclose", I could read exactly the pages that require patching from NAND into RAM, patch them, write them back. The "mtdblock*" devices don't even require explicit erasure, they can just "overwrite" and handle the rest by themselves. And they allow random access which means we don't write back everything, but only the pages we changed so less possibility for desaster (if we write the entire Radio back chances are higher that we hit a "bad page" that cannot be written to and then have a "hole" in the Radio).
[HTC Wildfire S] theq86:
Well, but we know that it somehow must be possible to access only chosen locations.

The XTC clip e.g. mimics a eng sim, probably a smartcard containing a key or even it's own os and prepares the sdcard with a diag utility. and this one tells the hboot to make s-off.

so how does that work? if you are right they probably do not use block devices