yashsingh1/xdaforums huggingface dataset:
[XPERIA X10 Android Development] mayank.1love:
Figured out some more info today. The ser file which is actually compiled java bytecode is sent from SE server to SEUS when we click confirm for firmware upgradation on SEUS. "It tells SEUS what to do next". This ser file tells SEUS to download appropriate firmware files from SE server, to unlock or lock a phone, to activate the phone.

This ser file is importing some important classes from jars present in SEUS plugins folder. I digged deep into the listed classes in ser decompiled java and found out every single method is inside the plugins folder.

Keys are sent/received to/from server using the methods in SEUS plugins only. I think I can find out a way to have a valid key to sign our custom firmware. Or even have test firmware before its release to public as the methods dynamically read the model number and type from our mobile which I can set as a static value as developer model inside the jars.

Still have a lot of work to do. The java inside plugins jar are somewhat difficult to understand as they are doing some assembly level coding using direct JVM calls. Need to learn assembly language a bit more to know exactly what the plugins are doing.

The priority for now is to understand the important classes in plugins folder and spoofing a localhost server as SEUS server so that we can inject our own ser file into SEUS and make it do whatever we want to do.

I'll keep you updated of every info I get.

I am using DJ Java Decompiler.

Its working all good for jars/classes.

Hex Workshop Hex Editor comes handy for situations when the files cannot be identified easily.
[XPERIA X10 Android Development] brett19:
Hey,

What was it that you were looking at that needed an understanding of Assembly? I may be of some use there.

P.S. I noticed some people here seem to be thinking that we might be able to find the certificate they use to sign the system through SEUS, but would it not make sense that SE would be signing the new system package before they put it online for us to download? Not have it signed AFTER being downloaded? I can't believe SE would be stupid enough to drop the certificate locally.

Cheers, Brett
[XPERIA X10 Android Development] zdzihu:
Guys, I need some help here (regarding chroot):

I'm currently stuck in boot loop with dalvik crashing every few seconds:

http://pastebin.com/pcqTucni

http://pastebin.com/hfDmJVbH

Any clues? I have tried 3 different framewoks (1.6, 2.2CM, 2.2AOSP), but no joy. I wiped /data, /dalvik-cache etc; paths and permissions seem to be ok. May it be related in any way to files being deodexed (stupid, but original rom works fine...)?

I'm running out of ideas :D

Cheers!

PS. Do I need to wipe /cache? It appears to be empty...
[XPERIA X10 Android Development] Stryker777:
Same here. I have done a lot of asm over the years. Time is a constraint but I will help when I can if needed.

About the signing, I would think it would be done ahead by SE also. Have not looked into it though so I will retain more input till I get some time to look. :D
[XPERIA X10 Android Development] biktor_gj:
Yepp, I know what that is :)

The chroot reversing and putting back the system partition to /system instead of leaving it on /... That's what I'm with, If you connect through adb, and do a chroot from there, it will keep on booting (unless servicemanager died more than 4 times)...
[XPERIA X10 Android Development] zdzihu:
Disregard whatever was written here before ... I wasn't sober enough haha :D

Just realised that mtdblock2 was mounted already as system and remounting it could lead to our problem.

Anyway, commenting out those lines _seems_ to solve the "root reverting" issue for me:

# Mount /system rw first to give the filesystem a chance to save a checkpoint
#mount yaffs2 mtd@system /system
#mount yaffs2 mtd@system /system ro remount

Need more testing now.

Cheers!
[XPERIA X10 Android Development] drdownload:
@stryker777, I think the signing is done ahead by SE but if the sig gets verified by SEUS and not by the bootloader there a good chance to flash custom ROMs.

Bad luck, it would be if both would be the case.
[XPERIA X10 Android Development] biktor_gj:
I added one more line:

mount yaffs2 mtd@system / rw remount

And it seems it goes a little further (even mantains the RW attribute), but it feels like when the drm script finished the chroot goes back to where it was, maybe we need to force an endless loop on it?

BTW how are you preparing the environment? You copy it all at once or made a package you untar to the system partition?

Edit: i think Im starting to figure out this mess, i think my problems are related to adb and some things more that I have to kill before chroot..dont know if it will work though... ill send you a pm tonight with my last package so you can check it out..
[XPERIA X10 Android Development] Stryker777:
So if we find where SEUS is checking the keys, we can trace it, find the key, modify, and try to push it to the phone and find out if it is checking or not. Or am I not following correctly?
[XPERIA X10 Android Development] zdzihu:
I got rid of all /system remounts in init.rc (I remount rw in drm.sh before chrooting) and it seems I am booting fully from the new environment now.

I prepare a package with init files, dir structure and system (around 50 megs), and then unpack it with basic install.sh.

I tried to kill the "old" adb in drm.sh, but didn't make any change for me...

Take a look into my files (based heavily on yours haha) to see what I kill/delete/umount. Should be in your PM box already. Don't laugh! :)

BTW, if you uncomment all services in init.rc, this package should boot SE's 1.6. But other roms still give Dalvik errors. I'll try to investigate.

Cheers!
[XPERIA X10 Android Development] jerpelea:
finaly got some time to play

kernel boots and initrd is loaded then it loops

i tried to use storage card for system.ext2 and did not touch /system partition

this way we can boot using our own kernel
[XPERIA X10 Android Development] ddewbofh:
Nice, being able to boot custom kernels has become my biggest wish after PSFreedom was released. :)
[XPERIA X10 Android Development] jerpelea:
http://www.sendspace.com/file/7owluu

thanks again my friend (biktor_gj) for bringing my phone back from death

all this work was done with help from goroh_kun who modified initial files

Bin4ry's ideea was to boot an system.ext2 from sdcard instead of using /system partition so i did this

it crashes but .......can be fixed
[XPERIA X10 Android Development] zdzihu:
This is... awesome! Thanks a lot jerpelea
[XPERIA X10 Android Development] maxrfon:
No way!

Can you please explain the process?
[XPERIA X10 Android Development] Bin4ry:
This is true. But it's a minior problem. The initrd has some permission issues which should be solved easily in a unix enviroment. Just extract the ramdisk correct the permissions and put it back. Then the loop SHOULD stop ;)

Regards

Bin4ry
[XPERIA X10 Android Development] Bin4ry:
This is not about flashing (YET). It's like jerp said. We can load a kernel and its ramdisk out of the original image (only the .prop file must stay from original image), after that we can mount a image from SD to get it started. If all works correct we can switch from mounting to extracting it to internal flash. But mounting is the better way to test, because you don't mess the system and you can fix problems faster without flashing back to stock all the time.

So the idea is like i written before. Get some images on SD and mount it from there.

If all works correctly I can image that we can write a little Bootmenu so that you can decide what image you wanna boot and the Bootmenu extracts the desired image to internal ram and boots it from there. But first of all, this method needs some improvements so that it is fully functional.

Regards

Bin4ry
[XPERIA X10 Android Development] mixxy:
dont take this the wrong way, but after we're able to mount an image from the SD card, wont running an image from the SD card make the system run extremely slow (unless u have a class 4+ microsd)?

or will it mount it to system memory? in which case wont we be lacking in that category?
[XPERIA X10 Android Development] biktor_gj:
Yes, but it's useful for testing since you can 'safe boot' from nand when you break the SD :)

@Jerpelea: Welcome back my friend, glad to see you here again ;)

So what's the problem with the initrd? not getting the partitions? atags? or simply looping for no apparent reason? if someone can drop the ramdisk sources I'd like to take a look...

Also I did not get to mount the yaffs2 images directly, not sure if I'm missing something... did you guys have more luck than I did?

PS: Oh! better than using loopback images should be better to have 3 partitions : fat32/ext_data/ext_system. That or disable vold for the moment or it will unmount the sdcard and cause a bootloop ;)
[XPERIA X10 Android Development] Bin4ry:
Read my post that you quotet carefully :D I wrote in the last chapter that we can extract the image to internal memory if we want. So booting from SD is the first step to test everything. If all work is done we can port it like this :

1.) Extract the image from SD to internal memory

2.) Boot from there

But now its much better to boot from SD, because you must not reflash the whole phone if you fail ;)

I attached the ramdisk my friend. I think we just have perm. issues.

So take your look and hexedit it back inside boot-new.img

Regards

Bin4ry
[XPERIA X10 Android Development] zdzihu:
init

All answers inside :D

Cheers!

EDIT: Bin4ry is too quick :P
[XPERIA X10 Android Development] Bin4ry:
Hrhr :P

Regards

Bin4ry

- Sorry for Spamming ^^ -
[XPERIA X10 Android Development] JamesBarnes:
Im not able to flash my phone atm so i cant help and test, but if you can give me a specific account of the errors/symptoms that you are having i will fiddle with the script and hopefully we shall make some progress ;)
[XPERIA X10 Android Development] biktor_gj:
Thank you guys, taking a look to it now :)

I think I might do a different ramdisk using 2 partitions on the SD to ease my life a little bit...

Let's see what can we do!

EDIT: why is it missing the init scripts?
[XPERIA X10 Android Development] zdzihu:
They should be placed manually on the / together with real (binary) init
[XPERIA X10 Android Development] ddewbofh:
IIRC the name of the init-script isn't hardcoded anywhere and can be passed as a boot parameter. So creating a custom script and passing that with a boot parameter ought to work.

Edit: I've gotten the hang of the boot procedure and how to do proper testing. Turns out that going from creating custom Linux distros to creating Android "distros" needed more mental adjustments than I anticipated. If anyone needs something tested or feedback feel free to give me a shout. :)
[XPERIA X10 Android Development] biktor_gj:
Afaik it runs init.rc, then init.machine.rc, where machine is the board model (es209ra, trout, sapphire...) which is passed by the kernel, which gets it from bootloader...

And since we reboot through kexec an init script needs to be run from ramdisk's init so it can mount partitions to actually start running on the whole system...

Im trying to repack the boot inage, but I must be corrupting the image, Im only removing the old ramdisk and hex copying the new one over it... Should i do it some other way? Mkbootimg maybe?
[XPERIA X10 Android Development] zdzihu:
The idea of this approach is to launch fake init (prepare environment, populate /dev, mount rootfs from card) and finally switch_root into our image (where init and initscripts are).

That's how I see it.

Do you reckon we could bypass switch_root thingy and boot the usual way with init+scripts on the ramdisk?

I'm using mkbootimg and it works fine.

http://xdaforums.com/showpost.php?p=7419264&postcount=100

By corrupting, do you mean that you're stuck at the black screen right after "Booting Linux"? Because that's exactly where we are atm :)
[XPERIA X10 Android Development] Bin4ry:
Nope you are doing right.

Maybe you are corrupting the ramdisk. Have you used newc option?

Or simply use mkbootfs to build ramdisk.

$ mkbootfs ./ramdisk | gzip > ramdisk.gz

Everytime i use mkbootimg the bootimage gets corrupted. Thats because Xperia is not using standard bootimage structure like its described in mkbootimg.h and maybe i miss a option.

For me hexediting is working good. But if i build ramdisk manually its corrupted. If i use mkbootfs its good. :)

Regards

Bin4ry

EDIT:

Argh this time i was too late :D
[XPERIA X10 Android Development] biktor_gj:
To create the ramdisk I do it old school :)

find ./ | cpio -H newc -o > ../ramdisk_new ; gunzip ../ramdisk_new

What it does is start the loader, attempt to position on the entry point for the bootimage, say invalid bootimage, and bypass the entry point and restart. It goes on an endless loop. It happened with some broken kernel images and with some versions of the miniloader. Haven't attempted to boot jerpelea's image though, if kernel versions are too different there might be a problem somewhere I don't know... When I reflash the phone tomorrow I'll dig into it launching it manually without the autoboot script (god I felt like a newbie ;))

Must also say that the ramdisk had some broken end or something, but I'm doing it with SE's original ramdisk dump as a base, so it should work...

I'll redo everything from scratch tomorrow with a decent hex editor (Sorry GHex developers, but it's horrible to use your program...) and see if I can at least manage to start the kernel...

@zdzihu: That is the idea, we don't stop just init with this, we stop the kernel, init, environment and everything on the second the miniloader starts. So, when we boot a secondary kernel, we reset almost everything (except the modem), hence we need to start init from the ramdisk and it needs it's scripts to work out. That would be like booting a different boot image from fastboot, for example.
[XPERIA X10 Android Development] Bin4ry:
Exactly :D when i type this my ramdisk is ****ed up. So i decided to use mkbootfs and it works. I have no idea why oO

@zdzihu:

Thanks for the link to goroh's post. Now i know why my bootimg is corrupted. I forgot the base option ^^

Regards

Bin4ry
[XPERIA X10 Android Development] zdzihu:
@Biktor_gj

Do you still happen to have serial2usb adapter?

Just a thought, SE kernel boots with console=null, but now you could change it to ttyMSM0... or am I wrong?
[XPERIA X10 Android Development] biktor_gj:
Yes I have it, but ttyMSM0 can perfectly be the bluetooth serial device, and even if it was a normal serial port we don't know where are the headers to connect it :(

I think it would be easier just to remove the console parametter (I think it defaults to the framebuffer if it's not specified) or otherwise force it (console=/dev/fb0)

Worth a try!

@jerpelea: could you upload your zimage somewhere? I want to be sure I'm not cutting the kernel when I hexcut it for mkbootimg...
[XPERIA X10 Android Development] zdzihu:
You may try this script to unpack it (just for reference) http://android-dls.com/files/apps/unpack-bootimg.zip

Regarding previous post:

EDIT: Nevermind I worked it out :)
[XPERIA X10 Android Development] biktor_gj:
On a side note, we have a problem between kernel / amss / original kernel

My phone booted just fine goroh's bootdiag image, running now on 026 it gets stucked. I'm going to try to downgrade the radio to see if it makes a difference...
[XPERIA X10 Android Development] zdzihu:
I checked with 1.0.14 but no difference.

I played a little with available ramdisks and images, but I can't get it to work. The only way to get into Android was to use Goroh's bootkit.

PS. I tried to force console to /dev/graphics/fb0 but no joy :(
[XPERIA X10 Android Development] biktor_gj:
Hi, while I reinstall my windows machine, I was thinking on something. Seeing the way Samsung has done the init sequence on galxy series phones (tested on a i5700 and i9000 and both look the same), we can replace SE's init, with one modded to allow recovery from there. I took a look and doesn't seem difficult, simply catch a keypress on the main function and use the filename "recovery.rc" instead of "init.rc". That way we could get something similar to clockworkmod recovery on ours with the boot images.

About the radio and stuff, then it will be the kernel (will check when my windows laptop is up and running again...) hence we might have a problem if we need to use the same kernel version when doing the jump. It will probably have something to do with the rpc driver crashing on boot. Because it attempts to, but doesn't get it right.

@zdzihu: I tried hex editing the bootimage and alter the console value, and removing it from the image but don't work. BUT, goroh's image is able to show the console on framebuffer, so not everything is lost. I first need to be able to boot again goroh's image, then I'll chroot manually and start it up by hand, then build a different boot image. Or at least that's my plan. I only need time. And don't have much lately...

I'm also trying to make a fresh froyo build using the sources, but don't have any ideas on which are the critical required libraries for SE to boot fine. Any ideas on how to gather which is what?
[XPERIA X10 Android Development] zdzihu:
That sounds like a great idea! Hope you'll find some time to look into it.

I modified ramdisk on latest Goroh's image and it boots fully into android. With severe problems like no gsm, no sound... But it boots! If we could get the sources, we would eventually solve those issues.

@Goroh_kun - if you could upload your recent sources that would be much much appreciated! I'm not too good with kernel hacking but I'm willing to learn :)

I also compiled my own kernel (with old rpc patches), but I can't get it into fb console - I'm stuck at Tux logo. I must have overlooked something, but it's still WIP.

Considering platform alone (without kernel) I'd say there's nothing really critical (seek for clues in Jerpelea's CM images).
[XPERIA X10 Android Development] jerpelea:
please post your boot back

i want to compare

thanks
[XPERIA X10 Android Development] zdzihu:
Hi Jerpelea,

Here is it:

http://www.mediafire.com/?99gj94pbhlo983v

Cheers!
[XPERIA X10 Android Development] jerpelea:
thanks

lets see what can be done ;)
[XPERIA X10 Android Development] zdzihu:
Here's why:

http://pastebin.com/qUbJ3cBy

Shame on you SE :D
[XPERIA X10 Android Development] biktor_gj:
We need a way to reboot the mcpu without rebooting the acpu at the same time on kernel init. That and we can do whatever we want. That or close the rpc chats before kexec and pray :). The console code can be patched...hmm maybe call some of the shutdown routines before the jump through splboot?
[XPERIA X10 Android Development] zdzihu:
Hi Biktor,

I'm playing a little with resetting mcpu / modem, but obviously everything dies miserably. Even halting modem leads to crash.

Could you tell me what exactly happens when I'm sending mcpu reset request?

(like: acpu gets notification of mcpu death and commits suicide etc etc)

or anything else I should look for?

Another question is: do I definitely need to reset mcpu before kexec or it's good enough to do it early from the new kernel?

Sorry if those questions sound silly, but I'm ARM/GSM dummy :) Learning the stuff, but really short of time now.

Cheers!
[XPERIA X10 Android Development] biktor_gj:
Well, I'm quite a gsm dummy too, no problem ;)

if you only send the modem reset, the mcpu resets, and it seems the acpu resets itself too. I'm trying to fix this problem (don't get much hopes), but I don't even know if I'm using the correct splboot sources.

If you try to reset the modem on the bootloader or on the new kernel, its firmware will already have crashed (because smem is trashed and no one was there to reply since original kernel died). So I'm trying to recompile splboot module with some little modifications to make it close the rpc channels before jumping to the loader. It's a shame it doesn't compile. Any ideas on which splboot sources are the last? the last one I have is "splboot_5.zip", but it stops compiling saying:

arch/arm/kernel/head.S:42:2: error: #error KERNEL_RAM_VADDR must start at 0xXXXX8000

The problem is on the physical kernel address, but don't know what should I hardcode there...
[XPERIA X10 Android Development] Bin4ry:
I have to check which one was the last that i compiled successful, but i THINK it was this one i attached.

Regards

Bin4ry
[XPERIA X10 Android Development] zdzihu:
KERNEL_RAM_VADDR >= PAGE_OFFSET + 0x8000

from memory.h :

#define PAGE_OFFSET	UL(CONFIG_PAGE_OFFSET)

and from .config:

CONFIG_PAGE_OFFSET=0x80000000

Don't blame me if it explodes etc :)
[XPERIA X10 Android Development] zdzihu:
Guys, is it possible to umount /system somehow?
[XPERIA X10 Android Development] ucaboys:
If unmounting is not posible, how about mounting an empty partition to /system.
[XPERIA X10 Android Development] zdzihu:
Ok, so after having a lot of fun with my X10 I hit the obstacle that seems to affect both chroot and kexec approaches.

Basically when /system is replaced by anything else than SE's stuff, the init crashes and reboots before reaching the point we could start our own code (or even connect with adb). So even if we get fully working kernel for kexec, we are stuck with the current system, aren't we?

In my understanding, the guilty must be either SUF or CMZN thingy. If there's a way around them, we would be able to chroot to Froyo exactly the way Milestone guys did.

Can someone correct me if I'm wrong and/or suggest possible solutions?

Cheers!
[XPERIA X10 Android Development] MeZmeriZe_:
Curious, if you copy the SE /system elsewhere and chroot to that, does it still crash?

(I mean copy the entire se system unchanged, if it still fails it should give some pointers as to why yes? )
[XPERIA X10 Android Development] ucaboys:
zdzihu,

Congratulation on the effort, it seems that you guys are narrowing down the solution to the X10 dilemma. I'll gladly help out in any way I can, I'm a developer my self - I just need some orientation regarding the Android System.

I guess that it must be looking for a File or some property on SE's /system. Is there any way to log what is being read/written from the /system mount.
[XPERIA X10 Android Development] zdzihu:
It doesn't crash anymore. But unfortunately this is different issue.

There's no logging available at the time of boot. I'll try to investigate for prioprietary files missing.

For now I'd say it's a SDK incompatibility problem (4 vs 8).

When I replace system, everything (including libs) is compiled with SDK 8, but those few files from ramdisk (suf, cmzn, adbd) aren't, and they might fail to load.
[XPERIA X10 Android Development] Bin4ry:
Yep true.

Also The build.prop must stay if not the Signature Verification fails.

For myself i havent found a way to boot it from this way. And i'm still looking into Miniloader sources and tring to fix it, but for now i have not made progress.

The other hope could be the new 2.1 kernel, maybe it gives us some more options ;)

Regards

Bin4ry