yashsingh1/xdaforums huggingface dataset:
[Galaxy Tab Q&A, Help & Troubleshooting] Jarmezrocks:
Thats the thing though, I believe that CM 9 has changed several times since the beginning. Both the latest and stable didn't work. There is another stable version but really got put off the whole ordeal because I was reading so many posts of people confused as to which version they should be flashing and which one they made the stable version in the end.

Something else also thrown in was that some guys where flashing Overcome and not allowing conversion to ext4 and then upgrading to CM10?

Should flashing Overcome kernel over the top of standard Gingerbread allow the device to boot and run stock GB with ext4? How would this be different to running stock and using OCLF4 on stock kernel with CWM Recovery? Say CF root Recovery?

Sorry for all the questions but the device isn't mine and I am now looking for the simplest solution to making it run normal and stable. Seems like normal GB is so unstable its not funny! Is it normal to get random reboots on stock Gingerbread?
[Galaxy Tab Q&A, Help & Troubleshooting] priyana:
It is not normal for GB to reboot by itself. There must be something not right.

About overcome kernel on top of normal GB with ext4, yes that is possible to run it that way.

About allowing overcome kernel to convert to ext4, or not to allow that, it doesn't matter.

CM kernel will repartition the internal storage and everything will dissappear once a new partition scheme is in.

My suggestion is usually for people to not wait for overcome kernel to convert since it is not needed.
[Galaxy Tab Q&A, Help & Troubleshooting] Jarmezrocks:
Hey I got some good news!:good:

And....I will need you [MENTION=546063]priyana[/MENTION] to update your OP for the Overcome ROM thread too, this will apply to all Australian P1000 Tabs I guess...well at least the Optus ones anyway.

Essentially the GB_Stock_Safe_v5.tar and gt-p1000_mr.pit are NOT safe like we are lead to believe....well not for the tablet I have just repaired. It is now running CM10 no problems!:D

I have created a new firmware package that incorporates a new PIT and various combos of files from the old GB_Stock_Safe_v5.tar and the standard firmware package.

What I found was flashing the completely stock firmware the device would not boot or function what so ever, which is really strange as most people would expect that standard firmware would at least be stable enough to use. Not even with the stock PIT or the gt-p1000_mr.pit.

You can down load the files here >>> GB_Stock_AUST_Jarmezrocks+P1.pit

This will have a firmware package and pit file combination that is suited to Australian P1000 TABS

Containing

GB_Stock_AUST_Jarmezrocks.tar

P1_add_hidden.pit

These are the ONLY files that will allow the Tab to boot, run and function normally. Before the thing was just so unstable you could barely do anything with it.

So from here Australian user who are having issues/problems with the Overcome ROM guide and getting continual reboots or bootloops should flash this package.

I will start a new thread in the dev section where it will be seen by others who are wanting to salvage their device.
[Galaxy Tab Q&A, Help & Troubleshooting] priyana:
That is good to hear.

BTW, the new overcome thread is sos_sifou's, not mine ?

Regarding the stock ROM, I remember that Stimpz0r, one of the great devs here, used to own a P1000 from optus and have no issue with overcome GB stock safe v5.

Makes me wonder if there are different variant of optus P1000.
[Galaxy Tab Q&A, Help & Troubleshooting] Jarmezrocks:
Sorry yeah you are right, you're just the heavy promoter of it (with good reason too ;) eh?)

I will PM him and see if he can include it in the OP?

All I know is that without a doubt there was no way that package was saving this Tab, and that the one I posted most certainly did....so it would suggest to me that there is a difference between some models...I mean why else would they create a different firmware with a different .pit file? It makes no sense to do it without reason when really it is much easier to maintain the status-quo...don't you think?

Thanks for your valuable input too by the way. I really appreciate it :good:
[Galaxy Tab Q&A, Help & Troubleshooting] maihuuquan:
can you re-upload ? i cannot download it
[Android General] tal.aloni:
About:

fastboot stores each file it receives in RAM before flashing it to your Android device.

For this reason, large partitions (such as the /system partition) must be splitted using the "compressed ext4 file system sparse image format",

this format is defined by the AOSP (Android Open Source Project) and was designed for the single purpose of flashing a large partition.

If you tried to flash system.img from backup and received the "Invalid sparse file format at header magi" error, you have come to the right place.

The Solution:

SparseConverter is a tool that can create / decompress compressed ext4 file system sparse image format (e.g. system.img_sparsechunk1).

Note for Motorola phone owners:

The factory images from Motorola contains a 128KB motorola header and a 4KB motorola footer, if you decompress those images you may want to remove the header and footer.

I had no problem using the standard Android fastboot with my unlocked Moto G and flashing images without the Motorola header / footer.

(If you keep the header / footer, then you must use Motorola's fastboot)

Usage Examples:

SparseConverter.exe /compress D:\system.img E:\ 256MB

( will compress D:\system.img to 256MB sparse files starting from E:\system.img_sparsechunk1 )

SparseConverter.exe /decompress E:\system.img_sparsechunk1 D:\system.img

( will decompress E:\system.img_sparsechunk1, E:\system.img_sparsechunk2 and etc. to D:\system.img )

The software may contain bugs and/or limitations that may result in damage to your phone, I take no responsibility for any damage that may occur.

For additional information about the "compressed ext4 file system sparse image format" see libsparse/sparse_format.h

p.s. Source code in C# is provided, Windows XP \ 2003 users must install the .Net Framework 2.0 before running this tool.
[Android General] gouster3:
How can i flash system.img when is too large. i have compressed it using this tool, now how to flash?
[Android General] tal.aloni:
Everything is explained in the original post, You need to use fastboot.exe which is part of the ADT bundle (Android Development Tools).
[Android General] gouster3:
fastboot flash system system.img_sparsechunk1 ?

Not working for me. :/

It will be downloaded to device, but not flashed. :(
[Android General] tal.aloni:
yes,

fastboot flash system system.img_sparsechunk1

fastboot flash system system.img_sparsechunk2

fastboot flash system system.img_sparsechunk3

make sure the system partition size match the image that you're trying to flash, to flash a bigger image you'll have to reflash the partition table and all the partitions (boot, recovery etc.)

p.s. not sure what "not working" means, what error do you get exactly?
[Android General] gouster3:
Here is my error code, after attemp to flash system system.img_sparechunk1:

FAILED (remote: flash_cmds error!)

partition size match, it is system dump from another same device.
[Android General] tal.aloni:
Sorry, I'm not familiar with this error.
[Android General] gouster3:
I have Rooted my ASUS Tablet, and deleted something in system, now i am soft-bricked.

Now i cannot do somethnig, only use fastboot, butt i cannot flash system.

So i must waiting to developers, to port CWM/TWRP for my device. :)
[Android General] dark_samus:
Hi, I'm trying to convert a userdata.img file (60GB!) so I can flash it with fastboot onto my phone, when I run the tool I get this

C:\Users\Matt\Documents\SparseConverter_1.0.0>sparseconverter /compress C:\Users
\Matt\Documents\userdata.img "C:\Users\Matt\Documents\userdata sparse" 256MB
Image size is not a multiple of 4096 bytes

Does this mean that it won't convert my image? Thanks

EDIT: Also I know this image is valid as under linux

matt@Chell:~$ sudo mount -t ext4 -o loop /home/matt/userdata.img /home/matt/ext4
[sudo] password for matt:
matt@Chell:~$ sudo su
root@Chell:/home/matt# cd ext4
root@Chell:/home/matt/ext4# ls
anr bugreports dontpanic media resource-cache user
app cam_socket1 dpm mediadrm security
app-asec cam_socket2 drm misc shared
app-lib connectivity fdAlbum nfc system
app-private dalvik-cache local opponvitems time
backup data lost+found property tombstones
[Android General] tal.aloni:
Hi Matt,

The sector size for most flash devices is 4096 bytes.

The "compressed ext4 file system sparse image format" assumes that the image is going to be flashed to a flash device, and thus expects its size to be a multiple of 4096 bytes.

I'm not sure how you created this user data partition image, the size of a proper image (for flashing purposes) should be a multiple of 4096 bytes.

Edit: BlockSize can be set to 512 instead of 4096 in SparseConverter source code (SparseCompressionHelper.cs).
[Android General] hazard666:
I ran into this issue as well. I am trying to compress the userdata image from my phone that was previously bricked and I'd like to find a way to hold onto this data.

Thank you!
[Android General] tal.aloni:
How did you create this image? are you sure it's a raw image?

if it's indeed a raw image, you could append a the necessary amount of zero bytes to the image (e.g. using HxD) and then you could compress it.

I'm not sure what you mean by "hold onto this data", this tool is to prepare an image for flashing, if you're trying to access the files there are other methods (e.g. linux).
[Android General] juliospinoza:
Awesome, I use this tool to compress system file of Motorola X2014 of 2.7GB that I was not able to flash it... works great, thank you so much
[Android General] Azlun:
Great progress! In my case

I generate my system.img.raw by

adb root

adb wait-for-device

adb shell "dd if=/dev/block/platform/f9824900.sdhci/by-name/system of=/sdcard/system.img.raw"

adb pull -p /sdcard/system.img.raw

1st Trail below will NOT works, but at the end you will see my solution:

Windows cmd log:

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 536870912

...

pause

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_

sparsechunk1

target reported max download size of 536870912 bytes

erasing 'system'...

OKAY [ 0.591s]

sending 'system' (524284 KB)...

OKAY [ 1.350s]

writing 'system'...

OKAY [ 13.953s]

finished. total time: 15.897s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_

sparsechunk2

target reported max download size of 536870912 bytes

erasing 'system'...

OKAY [ 0.532s]

sending 'system' (524287 KB)...

OKAY [ 1.351s]

writing 'system'...

OKAY [ 8.678s]

finished. total time: 10.565s

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_

sparsechunk3

target reported max download size of 536870912 bytes

erasing 'system'...

OKAY [ 0.524s]

sending 'system' (411515 KB)...

OKAY [ 1.075s]

writing 'system'...

OKAY [123.069s]

finished. total time: 124.671s

Please note even we pass the fastboot flashing, but it does NOT works after flash is finished, the phone can reboot to Linux logo, but block there, can not start Android Animation and go to desktop.

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell

- exec '/system/bin/sh' failed: No such file or directory (2) -

SO, SOLUTION is:

as an conclusion, it seems fastboot can not manage the multi input images well, BUT it can handle a single compressed image files, so set the max image side to the partition size to 2684354560 can make fastboot works well.

I need to find my original dd output file system.img.raw in 2.5GB, then:

Windows cmd logs:

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>sparseconverter /compress system.img.raw .\ 2684354560

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>fastboot flash system system.img.raw_sp

arsechunk1

target reported max download size of 536870912 bytes

erasing 'system'...

OKAY [ 0.767s]

sending sparse 'system' (521371 KB)...

OKAY [ 4.530s]

writing 'system'...

OKAY [ 9.577s]

sending sparse 'system' (511116 KB)...

OKAY [ 6.465s]

writing 'system'...

OKAY [ 10.833s]

sending sparse 'system' (445892 KB)...

OKAY [ 6.274s]

writing 'system'...

OKAY [139.377s]

finished. total time: 177.831s

that will bring my system.img.raw back to my phone WELL

you can see my rooted su is there, ^_^:

E:\AzlunData\DEV Tools\ImageBakupsCMD\SYS GMS+Root+AzAppsMDP>adb shell

shell@msm8994:/ $ su

su

root@msm8994:/ #
[Android General] tal.aloni:
Matt, hazard666,

I was wrong, there is no restriction on BlockSize , you're welcome to download SparseConverter source code and set BlockSize to 512 (in SparseCompressionHelper.cs),

please let me know if this helps.
[Android General] JulesJam:
Hi, I used your tool to decompress sparsechunks into a single system image for the Moto X 2014 XT1097, AT&T. This is needed as a preliminary step in order to get root.

The next step is to convert the image into an ext4 format. However, the instructions we have to do that require using linux command simg2img and it doesn't like the header. The errors are:

"Invalid sparse file format at header magi

Failed to read sparse file"

"Bad magic"

or

"Error reading sparse file header"

How do we get rid of this header? Thanks!
[Android General] tal.aloni:
I usually do it manually with HxD (hex editor).

I think you're confused, once the image is decompressed, it's no longer a sparse file, you shouldn't use simg2img (to convert it to raw image, because it's already been done).

instead, you can just mount the image, for ext4 it would be:

sudo mount -t ext4 -o loop,offset=131072 system.img mnt/

and when you're done customizing:

sudo ./make_ext4fs -l 840M -a system mysystem.img mnt/
sudo umount mnt

(note that I've used offset=131072 instead of removing the header)
[Android General] JulesJam:
Yeah, I am definitely confused, lol!

Ok great, I will have someone who knows linux try that. What we want is a single file system image in ext4 format so hopefully that will work!

Thanks!
[Android General] G4 Man2:
And I'm someone who's running a Linux distro!

After the second command I'm getting a memory allocation error though

"error: do_inode_allocate_extents: Failed to allocate 6524 blocks"
[Android General] JulesJam:
Also, just to be complete, the way I am going to modify/root the ext4 image is by using TWRP on a bootloader unlocked device according to these instructions:

http://xdaforums.com/showpost.php?p=60389310&postcount=177

It worked for the XT1096 so it should work for the XT1097. Once we get the rooted ext4 image, we have a tool to flash it to the XT1097 that bypasses the signature check. Our bootloaders are locked so we have to do it this way.
[Android General] G4 Man2:
After adjusting the length flag to 3 gigs I was able to get it to work (I'm clueless but I assume that's what I was supposed to do). I'm hesitant to start uploading until I can get verification though.
[Android General] tal.aloni:
It should be the same as the system.img that you're mounting (don't forget to subtract the header length).

p.s. 840MB is for the Moto G GPE, for the Moto X it should be 2768MB.
[Android General] G4 Man2:
Unfortunately I'm just the middle man who runs linux so I don't know exactly what I'm doing with the image.

So you're saying I should use the commands you gave earlier (replacing 840 with 2768) and I should be golden? Or do I need an extra step to "subtract the header length"?
[Android General] tal.aloni:
Take the length of the system.img you've decompressed (with the Motorola header and footer), subtract (128KB + 4KB) from it, and that's the size of the raw image that you should end up with.

example: say your system.img is 2,902,593,536 bytes, you remove the header and footer from that and are left with 2,902,458,368 bytes,

that's exactly 2768MB.
[Android General] G4 Man2:
The system image I have (with the header and footer) is in fact 2,902,593,536 bytes. So I do need to set the length to 2768M and I should be good?
[Android General] tal.aloni:
Yes.

note that it would only make sense to use make_ext4fs (which would create a new ext4 image) if you have made changes to the raw image.

if you're just sending the decompressed image to someone else, then just remove the header and footer with HxD, and you'll have an unmodified ext4 raw image.
[Android General] G4 Man2:
To my knowledge, all I'm doing is using some tools I have available to follow directions from JulesJam.

Here's what he said.

"I was able to make a single file but as the OP of that thread indicates, there is a header in the files.

When I try to flash it to my XT1096, it says invalid sparse file format at header magi. So obviously the header is noticed. However, it then proceeds to try to flash it. But it gives me a signature error - invalid signed image - and the flash is aborted. I can boot normally from bootloader mode so no issues there. "

He then asked for someone with the tools available to " convert the file into ext4 format"

I'm not knowledgeable in all this, I'm just trying to help a guy out.
[Android General] JulesJam:
She!

Girl!
[Android General] G4 Man2:
Sorry! I default to he, I'll start calling everyone "it" now.
[Android General] JulesJam:
With your help [MENTION=4049305]G4 Man2[/MENTION] was able to make an ext4 image that flashed successfully. Thanks for all of your help!!!

I was able to root the unmodified ext4 image and then I could use the tool we have - MoFo - to flash it to my phone with a locked bootloader. Next someone has to bake some useful features into the ext4 image to make it, well useful!

Our (Verizon and ATT Moto X 2014) bootloaders are locked so we have to use MoFo to flash unsigned, modified ext4 images to have root, Xposed, busy box, etc.

Thanks again for all of your help!!!
[Android General] JulesJam:
How do I do this? Is this what I use?

http://mh-nexus.de/en/hxd/

[MENTION=4049305]G4 Man2[/MENTION] - does this require linux?
[Android General] G4 Man2:
Any hex editor, even ones on Windows that can edit system images will work. Open your image with the editor and to get rid of the Motorola header and footer, delete the first 131072 bytes and the last 4096 bytes. Then save the image and that's that.
[Android General] JulesJam:
This is what I see:

Image is 2,834,564 and I need it to be 2,834,432.
[Android General] G4 Man2:
Say that I wanted to delete a 3 byte header and a 4 byte footer.

I would delete the first 3 FF FF FF

And the last 4 FF FF FF FF

So you need to delete the first 131072 pairs of digits and the last 4096 pairs of digits.
[Android General] JulesJam:
Each row has 16 pairs and 131072 pairs divided by 16 = 8192. So I have to delete the first 8,192 rows?

If I convert 8192 into hex, that is 2000 so I delete all of the pairs through row 2000 including row 2000? Or not including row 2000?
[Android General] G4 Man2:
I'm just gonna be honest with ya. On the software I use, I just entered 131072 and it took me to that byte. I'm no hex expert. I can tell you though you should see a difference in what the bytes contain where you need to separate the header and footer from the contents.
[Android General] JulesJam:
How did you do the footer?
[Android General] G4 Man2:
Searched for the 4096th byte from the end
[Android General] JulesJam:
I have a system.img that is flashable as is by fastboot. I want it to be an ext4 image. Can this tool do that for me? When I run that command above, it tells me Image size is a not a multiple of 4096 bytes.

---------- Post added at 06:01 AM ---------- Previous post was at 05:59 AM ----------

I think I figured it out. thanks!
[Android General] tal.aloni:
if you are suspecting your system.img is compressed, you can try to decompress it (/decompress).

it's also possible that it's just a raw ext4 image.
[Android General] JulesJam:
FYI - for anyone who wants a step-by-step on how to use a hex editor on a motorola image to remove the header and the footer:

1) Download the hex editor here:

http://mh-nexus.de/en/hxd/

2) Click Edit-> Select Block and in the pop-up select dec at the bottom. Then put 0 in the Start-offset and 131072 in the length. Click OK to select the header. Then click Edit-> Delete to delete the header.

3) At the top where it says hex, change it to dec using the pull down. Click Search-> GoTo and in the pop-up select dec and end (backwards) and in the offset put 4096. The blinking cursor will be on the last pair of the row ABOVE the one you want. Write down the number of the row right below where the blinking cursor is.

4) Click Edit -> Select Block and in the pop-up select dec, in Start offset put the row number from step #3 and in Length type in 4096. Then click OK then click Edit-> Delete to delete the footer.

5) Save the file.
[Android General] Fedex15:
Hello, I have motorola xt1068 I drew system_motog.img to edit and convert deodex rom. Deodex entire rom (app, framework, priv-app) and converted the system folder system.img with this command "make_ext4fs -s -l 1024M system_new.img system_mnt". Then when the attempt by fastboot flash fails because files have to be 256MB max.I try using the sparceconverter and pulls me error "Image size is not a multiple of 4096 bytes". I try to follow his tutorial HEX EDIT but I still the same, probe "sparseconvert / compress 256MB system_motog.img system" and the command works well, but not with the new image.Where you can be the error. When I follow the tutorial do not understand a part (my language is Spanish), when "The blinking cursor will be on the last pair of the row ABOVE the one you want says. Write down the number of the row right below Where the blinking cursor is . ", what would the code / line, I could put a picture, I will leave one to have if it is correct. Thank You
[Android General] JulesJam:
Did you try using mfastboot to flash it?

http://xdaforums.com/attachment.php?attachmentid=2427667

mfastboot can flash the single file system.img. That may be all you need to do. Please let me know if it works.
[Android General] Fedex15:
Yes, i try with fastboot and mfastboot, and its the same. I put the folder (framework, app, priv-app) in one zip and flash by recovery. Thanks
[Android General] JulesJam:
Is this the step you don't understand?

3) At the top where it says hex, change it to dec using the pull down. Click Search-> GoTo and in the pop-up select dec and end (backwards) and in the offset put 4096. The blinking cursor will be on the last pair of the row ABOVE the one you want. Write down the number of the row right below where the blinking cursor is.
[Android General] fenixus:
I don't understand how that set of commands might work if each consequtive fastboot flash system command starts with erasing system partition (and whatever chunk was uploaded before), so in the end user ends up with empty partition and only the last chunk uploaded. I think there was a fastboot switch -u to prohibit erasing partition before formatting, not sure if it works for flashing like that.

thanks for the idea!

:good::laugh:

worked for me too, set the chunk size to my system.img file in bytes (3221225472 in my case) to create 1 sparse file and then fastboot flashed it properly, finally got the device working after 1-2 hours of fiddling with it lol
[Android General] tal.aloni:
[MENTION=4474827]fenixus[/MENTION]:

Previous chunks are not erased when you flash the next chunk.
[Android General] fenixus:
well, it says "erasing system" before flashing each one ... anyways, that approach didn't work at all for me, phone was stuck on android logo after flashing the last chunk, factory reset, and reboot to system. maybe it differs between phones, mine is zenfone 2 laser ZE551KL.