yashsingh1/xdaforums huggingface dataset:
[Gen8 General] sodjas:
@scholbert:

Thank you very much one more time :) , hope we can bring out something interesting

@fzelle:

The answer for this is the circumstance when you are at a software development deparment surrounded with high level software guys and have an interesting and impressing task to build an RFID enabled industrial terminal with WIFI capability and a platform where you can easly build fancy touch based UI-s in just 3 months then you made choices like these -> to have a proof of concept thing -> to prove that your software is unique and feasible to develop -> to be able to go to exhibitions to make this interesting to investors -> later it is more then possible that we will say somebody who has the experience in hw building: OK we have this HW build us a similar one from OEM parts bundled in an industrial box...

@both:

Thank you guys for replying, this is such a nice community! :cool:

br, sodjas
[Gen8 General] sodjas:
Hey Guys!

This is what I've done:

1. studied the linked threads by scholbert

2. used the extended aos-tools and successfully extracted .aos file

3. now I'm seeing a:

-rw-r--r-- 1 root root 432 2011-10-10 17:21 digest
drwxr-xr-x 2 root root 4096 2011-10-10 17:21 raw
-rw-r--r-- 1 root root 10 2011-10-10 17:20 repack.sh
drwxr-xr-x 3 root root 4096 2011-10-10 17:20 root
-rw-r--r-- 1 root root 10 2011-10-10 17:20 unpack.sh

structure.

If I understood right scholbert the ramdisk is under root/data/androidnerged.squeashfs.secure

How can I manipulate this part?

How can I rebuild the ramdisk?

If I understand correct the GPL released part is only the initramfs and the kernel itself...

Another question is that the most interesting thing here for me is the aos-fix tool

What is the:

--clear-signature	Clear the signature out of a SIGN block, or a flash segment header.

part excatkly good for?

Sorry for the lots of questions, I'm still very green on low level :)

br, sodjas
[Gen8 General] sodjas:
One more thing:

I have the following structure under raw:

-rw-r--r-- 1 root root 264 2011-10-10 17:21 11_EXT2
-rw-r--r-- 1 root root 264 2011-10-10 17:21 15_EXT2
-rw-r--r-- 1 root root 8 2011-10-10 17:20 4_EXT0
-rw-r--r-- 1 root root 3264456 2011-10-10 17:20 7_MMCF
-rw-r--r-- 1 root root 2878728 2011-10-10 17:20 8_MMCF

I guess 7_ and 8_ MMCF files are the interesting ones...

Can I find a desc about these because in vitalif's thread about tweaking the bootloaders he tells about mmcblk0 / 1 ...

I also wasn't be able to find the hex parts found by scholbert when he linked vitalif's thread...

Still very confused but keep on learning ;)

br, sodjas
[Gen8 General] fzelle:
Of course i understand that, i'm a sw dev for 25 years.

I was lucky to see that board before we had some similar needs as you have.

What do you use for the rfid part?
[Gen8 General] scholbert:
Hi sodjas,

it's late but i'll try to answer your questions :o

So this is the firmware structure before it is installed on the device.

The update process is done by the bootcode and there's not that much information about it and not much we could actually do with these files.

So you'd better have a look at the installed partitions and the way to modify thoose.

No, the squashfs file is the final rootfs mounted as a loop device by the stock firmware.

It's possible to modify this file, but if you do the checksum won't match and you'll have to tweak bootcode to ignore security checking.

So in fact this is not very comfortable, but doable.

The ramdisk is the the cpio file archive also beeing updated and included in the update. Don't know the structure right now because it's on my linux laptop.

I don't know this tool and i guess it has been made for the older platforms.

Yeah, as i pointed out:

By extracting the update file you got an "external view" of the system.

After the firmware is placed on the device these files are flashed to eMMC and got mounted as block devices.

So it's little different then.

It's a bit complicated to explain... but quite easy in the end.

You'll have to read out the partitions as raw blocks.

All the low level stuff is mounted as /dev/block/mmcblk0p1

Please read out first using the dd command.

It will give a file at about 32MByte, ands it's all cryptic binary format (rawfs and below).

So if you like to enter this, there's not much information here and you'll have to digg out pieces.

Maybe it's better to modify the higher level parts...

Anyway, the very first bootlogo is called banner. The format is unkown, at least to me, and you'll have ot understand the rawfs file structure used by Archos.

Attached you'll find the mounts of stock and Uruk firmware.

I know that this is little abstract right now, but i'll have to take some sleep :p

Have fun!

scholbert
[Gen8 General] sodjas:
@fzelle:

the rfid part is for workflow monitoring in glass houses. we would like to deploy N terminals in each house and the workers can check-in, define a task they are working on and after they've finished can check out... simple but all info is gathered on a central place and is in order to make easier planning/observing/and calculating financial/investment related things...

the only problem is the musb_hdrc.ko -> it is not picking the FTDI chip if its plugged-in with the OTG cable at the same time. It is a known issue but I still have a lots of problem here:

1. if a change musb_hdrc.ko to my instance musb_hdrc.ko in /lib/modules it is not working at all, if I check it with modinfo it is different than the one in the GPL release. I guess these files under /lib/modules are also proprietary

2. I degbugged around and saw that the state changes for musb only work well if you first attach the OTG micro which has a grounded 5th pin ->then the state goes to a_idle and then attach the FTDI chip -> this is a working scenario

3. If you attach the OTG with already connected FTDI then it stuck at b_idle mode...

I started a thread about this:

http://xdaforums.com/showthread.php?t=1288522

but it seems the proprietary drivers will be the root of the problem here...

@scholbert:

a mininum thing I owe you is a beer :) a lot of useful infos here again I'll check around and come back if there are some updates.

thank you guys!

br, sodjas
[Gen8 General] scholbert:
Hi,

thanks for appreciating.

But stupid me, i forgot one thing...

Quoting myself:

So there's a kernel driver for rawfs of course :p

The rawfs parts got mounted to /mnt/rawfs and present themselfs as files (avboot, banner,...).

No need to fiddle with the raw binaries to investigate the structure.

You'll have to be root to access this directory.

I'll do some research about the file format of the banner file :cool:

EDIT:

O.K., here we go... it is a gz compressed file representing raw pixel data.

So in fact this is the very first boot logo!

Should be possible to import this file to gimp or something.

Don't know if it breaks the avboot security check if it get's replaced. So please be careful!

EDIT2:

I got a Archos 101 so filesize will vary on other platforms.

The pixel data is put into 32bit format so each pixel requires a 4-byte value.

On A101 the resulting file size is of the uncompressed banner file is 2457600 bytes -> 1024x600x4

The simple framebuffer inside avboot uses only the lower 24bit of 32bit pixel data.

The most significant byte of each pixel is always set to 0xFF (0xFF000000 written as MSB first)

To completely blank screen (delete archos logo) you'll need to blank out all pixel.

Pixel data should look like this:

0xFF000000 x 1024 x 600 (for A101)
0xFF000000 x 800 x 480 (for A70)

You might create such a file with a hex-editor or use some scripting.

Good luck with your USB driver stuff!

Regards,

scholbert
[Gen8 General] sodjas:
Update :)

I modified both mmcblk0 and mmcblk0p1, my tab is still booting.

The modification of /mnt/rawfs/banner directly was my idea too, but I'm still afraid of some parts :)

I pulled the banner file and on ubuntu I can list the content, but can't extract it, and there are lots of questions like:

1. The file inside banner is named i240x400 -> this means that it is a 240x400 image but it would be more logical that it is 400x240 based on how "Entertaiment your way " looks...

2. Ubuntu's file roller can't extract it

3. When I open it in hex editor it has some meta info at the start of the file

and a question at the end:

How did you uncompress your banner file? I always get errors, maybe I used wrong syntax...

br, sodjas
[Gen8 General] letama:
Hi Sodjas,

Let me jump in on this to complement your study:

First, a BIG WARNING: touching any of this without patching signature check will result in bricking device.

Second BIG WARNING: Vitalif's mmc offsets for bootloader patches are valid for his device only. Mine has completely different values, I presume there are multiple bootloader versions out there. If patch is incorrect, or not done in the right order, instant brick.

7_MMC and 8_MMC are indeed interesting parts, these are the kernel+initrd images for android and recovery.

As stated by scholbert, to get a better understanding, you should take a look at /mnt/rawfs:

You'll see few files there:

avboot (second stage bootloader)

init (normal boot android, kernel+initrd)

recovery (recovery boot, kernel+initrd)

custom (sde boot, kernel+initrd)

banner (boot logo as it seems)

7_MMCF is flashed to init

8_MMCF is flashed to recovery

Splitting these files into zImage+initrd.gz is quite easy, I can give you some hints if you want. But I don't think it would serve you much for what you're trying to do: what you want is changing bootloader logo, so your goal is avboot+banner.

But again, don't modify anything on mmc0 until you're sure of what you do. First bootloader checks avboot signature and avboot checks init and recovery signature for sure. Without patch, you'll brick your device.

I don't know if avboot checks banner file though.
[Gen8 General] sodjas:
Hi Letama,

Thank you for joining.

Yes sure you are right :) the values vitalif mentioned were on different addresses BUT occured only once in the whole binary for both binaries, so I tried my luck and I think it is working because after reboot I have mmcblk0 and mmcbl0p1 still with the patched content and the tab boots without any problem

If I'm right then now I can start playing with altering mmcblk0p1 content because I have patched verify_hash function. Am I right?

Two big questions still open for me:

1. How to modify banner in an efficient way?

2. I understand now I have two patched bootloader binaries and can dd them on any A70IT, am I right or the patching process needs to be done/ device? I suppose not.

Update 2 :)

In the meantime I uncompressed it on win with 7-zip I see the content I should modify to 0xFF000000 but I'm still not sure how flush this to all 0xFF000000 because there is some meta info at start. And as second, how to compress it, maybe the gzip -9 i240x400 will do its job as it was described at Gen5 boot logo tutorial...

br, sodjas
[Gen8 General] sodjas:
Sorry AFAIK, no metainfo, just the pure 0xAABBCCDD 32bit values as scholbert told me before, sorry for that :)

br, sodjas
[Gen8 General] sodjas:
I realized again I'm a fool, or just archos guys wanted to make a joke :)

1. the banner file is named i240x400 BUT

2. when you extract it you got a 1 536 000 bytes sized file

3. I used my awsome maths skills and found out that: 1 536 000 / 4 <byte> = 384000 / 800 <pixels> = 480

So the raw image size is 800x480 and no 240x480 as the file name indicates...

I just need to flush 0xFF000000 384000 times in a file I think.

br, sodjas
[Gen8 General] letama:
Yes, search pattern was the same for me too, only different offsets. Good thing that code didn't change.

I only modified init and recovery, but I guess they wouldn't implement a specific signature check for boot logo. I'm suspecting that there is no signature check at all on it but I never checked avboot for that.

To modify init and recovery, I wrote a small app that directly open /dev/block/mmcblk0p1, seek for the right position (with some file signature check to be sure I'm doing it right) then write. I make sure that I don't write a bigger file than current ones and it works fine for init/recovery. I believe that archos flasher does the same as existing file sizes are bigger than the firmware dump file sizes.

For your needs, I don't know if you will have to adjust size in the rawfs directory entry and/or avboot. Rawfs is quite straightforward, you can take a look at linux driver if you need to adjust directory entry.

A dd of mmcblk0 (not mmcblk0p1, you need also first bootloader) should do the trick if the hardware doesn't change and doesn't need a different bootloader. I think that there is multiple revisions of hardware out there, improper bootloader for a specific hardware revision could not boot.

Safest bet would be to write a small patcher that would check pattern and stop if you don't have a known bootloader.
[Gen8 General] sodjas:
Okay I'm successfully created and compressed a similar 1536000 byte sized file fully wiped with 0xFF000000 values.

The interesting thing now is that /mnt/rawfs is mounted as "rw" I have su but I can't alter the banner file, actually I can't do anything with it...

Should I mount the altered mmblkc0p1 on my host machine alter there the banner and then flash the mmcblk0p1 again?

I tried to mount it with losetup but with no success...

Keep on trying... :)

br, sodjas
[Gen8 General] scholbert:
Hi again!

I see there'd been some efforts and letama joined in, cool!

Be careful with altering the partition externally...

There's a checksum in the rawfs header section.

If you simply alter the banner at the given offset, checksum won't match.

I examined the linux driver sources but could not locate the checksum section right now. Please be a little patient and do not brick your device.

There are still too many unknown functions under the hood.

Perhaps there's some functionality to write to rawfs using recovery system.

I know that Urukdroid installer uses this feature to replace kernel and ramdisk.

I'm little limited in sparetime right now... so please don't hurry and be careful.

Your operating on very low level.

Cheers,

scholbert
[Gen8 General] letama:
Hi Scholbert!

You're sure of this ? Checksum check in kernel ? As long as bootloader 1 and 2 are patched I think nothing is checked at kernel level. I changed kernel and initrds without altering kernel, it works fine. What is also checked is signature on the squashfs.secure, but check is done with a toolbox command in the initrd scripts. There is also some checks in the recovery scripts on kernels, but that's it.

Sjodas patched checksums check function in bootloaders, so he should be fine regarding this.

However, I'd be cautious and try to check in avboot what happens if the bootloader can't gunzip the file because of size or whatever. If bootloader crashes because of invalid gunzip or invalid size, you're in trouble.

AFAIK, no. There is kd_flasher (from toolbox) that let you flash a zImage and initrd.gz for sde, but it only does sde (custom file in rawfs). I presume this is what is used in Urukdroid. But to write normal kernel (init in rawfs), I had to use my own tool.
[Gen8 General] sodjas:
Yes scholbert, you're right :)

I just wondered, that the most elegant way could be to mount - if it can be done - the mmcblk0p1 on my host machine, alter the banner file there and then flash it like I did during the boot loader patch.

letama mentioned that he uses a small utility for seeking and writing mmcblk0p1 maybe this is the only way, but then th tricky part is to find out where the banner file starts...

@letama:

If you read this could you please tell us more about your utility? It would be great...

I't is exciting but you are right scholbert I must be careful and patient :)

Another thing what is not 100% clear for me:

We are talking now about the very first logo, but during every boot there is a blink then comes up same -unanimated image again- this is the logo of the second loader, but where is it exactly?

The animated is already done, fortunately if you place proper zip under /data/customization it works perfectly as I told.

So much things to do but still exciting...

Thank you guys!

br, sodjas
[Gen8 General] scholbert:
No, no, you got me wrong.... i was speaking of rawfs as filesystem.

There's a header definition with chksum inside the kernel driver source code.

By comparing two raw images from different devices, i also found different values of the header files inside the rawfs partition.

Like in every other filesystem there seems to be something like an integrity check.

There's also some kind of file table at the beginning of rawfs (araw).

The begining of each file inside the partition is starting with a header (fraw).

So if we break this structure by tweaking single bits, there might be problems.

O.K. thanks for the info :)

Yeah i guess this should be the way.

The stock firmware uses the same file inside the cpio ramdisk.

See attachment!

The file init_lib.sh calls a tool which grabs the banner file.

It is located under /etc/scripts inside ramdisk.

To override, the script could be changed (see ramdisk of urukdroid for example).

Then the ramdisk has to be re-created and flashed to the device.

Regards,

scholbert
[Gen8 General] letama:
You can't mount easily, there is no rawfs driver in "regular" linux kernel. You could add a rawfs driver, but it's probably simpler to write a tool to work on an image or on the device itself.

Well, finding where the file start is quite easy, just dd mmcblkp1 and search with hex editor. That's what I did for finding kernels in it. However, I didn't have to adjust size of the file in the directory entry, so I didn't bother. In your case, I don't know if you have to, my bet is that you have to adjust size.

Well, this is really basic:

fd = open("/dev/block/mmcblk0p1", O_RDWR);
if(verify_signature(fd, initkernel_sig, initkernel_offset, 8) <0) {
printf("unable to find kernel signature on mmcblk0p1\n");
exit(-1);
}

fdn = open(argv[1], O_RDONLY);
if(fdn < 0) {
printf("unable to open kernel to patch\n");
exit(-2);
}

kernel_size = lseek(fdn, 0, SEEK_END);
printf("kernel size=%d\n", kernel_size);
if((kernel_size > 7490927) || (kernel_size < 1000000)) {
printf("kernel size invalid, aborting\n");
exit(-3);
}

kernel = malloc(kernel_size);
if(kernel == NULL) {
printf("unable to alloc memory for kernel\n");
exit(-4);
}

lseek(fdn, 0, SEEK_SET);
if(read(fdn, kernel, kernel_size) != kernel_size) {
printf("unable to read kernel in memory\n");
exit(-5);
}

if(verify_signature(fdn, initkernel_sig, 0, 8) <0) {
printf("unable to find kernel signature on newkernel\n");
exit(-6);
}

printf("flashing new kernel\n");
lseek(fd, initkernel_offset, SEEK_SET);
kernel_wrote = write(fd, kernel, kernel_size);
if( kernel_wrote != kernel_size) {
printf("write failed, %d written, %d wanted\n",
kernel_wrote,
kernel_size);
exit(-7);
}

close(fdn);
close(fd);

verify_signature is a basic function that seek the offset and verify that what I find there is matching what I expect. After that, I re-open mmc and check that I still have a kernel signature, but it's only a paranoid check.
[Gen8 General] letama:
Oh! Right, there is a structure in front of the kernel+initrd files, there is probably one in front of the banner. Structure is described in libaos.h in aos-tools, I rebuild it with correct sizes for kernel and initrd when I prepare the file to be flashed.

struct flash_header {
uint32_t magic;
uint32_t bits;
uint8_t signature[AOS_SIGNATURE_LENGTH];
uint32_t unk_88; // data is signed from here to eof
uint32_t filesize; // size of the file: always populated
uint32_t entrypoint; // address of the file's entry point: in boot1 this
// is typically 0x87e80100, in cramfs files this is 0x100,
// in init/recovery kernel this is not populated
uint32_t cpio; // offset to the .cpio.gz file: populated only in init/recovery kernel
uint32_t cpio_size; // size of the .cpio.gz file
uint32_t unk_9C[25];
uint8_t data[];
} __attribute__((packed));

Extract of my code to repackage kernel files:

#include "../libaos/libaos.h"
...
struct flash_header header;
...
read(fd, &header, sizeof(header));

printf("sizeof(header)=%ld\n", sizeof(header));
printf("magic : 0x%08X\n", header.magic);
printf("bits : 0x%08X\n", header.bits);
printf("filesize : %d\n", header.filesize);
printf("entrypt : 0x%08X\n", header.entrypoint);
printf("cpio : %08X\n", header.cpio);
printf("cpiosz : %d\n", header.cpio_size);
...
header.filesize=sizeof(header)+krnsz+cpiosz;
....

The header has effectively a signature (header.signature), but it's bypassed with the bootloader patch (at least for kernels, I trashed original value to be sure of that).
[Gen8 General] sodjas:
So guys, scholbert I made a little investigation in init_lib.sh.

And the first thing I tried is to alter the script.

I made my own banner file and put it under root altered that one row in init_lib.sh:

from /mnt/rawfs/banner to /banner

The result was a blank screen :) so probably something is not good with my banner file but at least I only have the very first one.

Keep on hacking to get a working self created banner not just a blank screen.

Maybe here is a checksum verification too?

br, sodjas
[Gen8 General] letama:
No, there is no checksum verification in script and fb_write command. fb_write basically takes the uncompressed input and put it to frame_buffer. I was able to do it directly from android: pushed abcbox to my build, re-did the link abcbox fb_set, uncompressed the file (as zcat does) and then push it, logo is properly displayed.

To do your own image, photoshop is able to open raw file but doesn't process well the ff part (alpha channel I presume), it believes it's a cmjn instead of alpha+rgb and therefore messes up colors. The easiest would be to do raw rgb in photoshop then write a little tool to add alpha.

I checked mmcblk0p1, there is no flash_header for this one. Offset is at 0x18c0400 of p1 on mine.
[Gen8 General] sodjas:
1. I pulled /mnt/rawfs/banner

2. Extracted it and got i240x408 raw image

3. I created a 800x480 blank bmp

4. made a dd if=i240x480 of=blank.bmp seek=1 bs=54 command

and voila :) I was able to open archos's image with gimp, it was upside down as mentioned in the Gen5 boot logo setting

One thing I'm thinking about is that at this boot stage / (root) is not set so my first idea to put my custom banner under / is not really good (because I tried it with the unmodified archos banner just to check it works)

My second was put it in the same dir as init_lib.sh is in initramfs and to modify /banner to ./banner but still no success...

I think I must write it in mmcblk0p1 and re-flash it to work proper... Then I don't need to alter init_lib.sh...

Keep on trying...

br, sodjas
[Gen8 General] sodjas:
Another thing:

@letama:

As you suggested I searched in the dumped mmcblk0p1.

My theory was to dump mmcblk0p1 abd pull the banner file (from the mounted rawfs) and to open both with hexeditor and try to find a specific pattern from banner in mmcblk0p1 but I couldn't...

Is this approach right or I'm doing something nonsense? :)

br, sodjas

Update:

I studied the piece of the utility you sent me and yes it nice, I need to seek in mmcblk0p1 find the start of the banner fill it with 0xff000000 and that it :)

But again you have ther ther kernel_offset etc etc values... I don't know how to determine the banner offset... :(

But I won't surrender :)

br, sodjas
[Gen8 General] scholbert:
Yes that's the way to do it manually. I did the same and found it.

BTW, i dumped /dev/block/mmcblk0 (including inital page)

Should make no difference though.

Banner starts with:

0x1f 0x8b 0x08 0x08 0x85 0xb5 ...

Alternatively use text search for "i240x400" and go back to starting line for the offset.

All files start at full pages.

In fact any gz compressed file could easily determined by 0x1f 0x8b :D

Do it manually!

See above.

Using the dd tool you should use bs=512 (fits to internal structure of eMMC).

Good luck!!!

scholbert
[Gen8 General] sodjas:
I couldn't find it because I left in the count=257 param in dd command and it read only 128K (vitalif used this command because it is enough to patch bootloaders) :eek: how stupid I am :rolleyes:

ok round two starts... :cool:

br, sodjas