yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] lebigmac:
@ronlkh1965

This seems like a superunpack specific problem and I would suggest you to post it in the official superunpack thread and wait for the author of the tool (munjeni) to answer it. He should know what's wrong.

Sorry I can't help you with this issue. It doesn't seem related to my script at all.

Apparently it is failing at extracting the partitions from your super.img file for some reason... Have you tried it on a real Linux machine? Maybe it has to do with the virtual machine settings? Do you have full read-write access to the current folder in the virtual machine? Or maybe your super_original.img is corrupted for some reason?

If you can't get superunpack tool to work maybe you can try with lpunpack but then you would have to adjust the script accordingly as superunpack creates raw .ext4 files and lpunpack seems to create sparse .img files

What happens if you execute this command:

./tools/lpunpack ./super_original.img

Does it successfully create your 4 partitions as sparse .img files?
[Android Software/Hacking General [Developers Only]] uvzen:
I tried to do it again on a set up system without any virtual machines. Successfully created super_fixed.img and successfully flash to my device but it is still bootloop. I also give a photo of twrp when I restore the super partition.
[Android Software/Hacking General [Developers Only]] lebigmac:
@uvzen sad to hear it causes bootloop on your device :(

Did you disable dm-verity and verification?

Looking at the first screenshot it seems like all went well. I don't see any errors. Even the lpmake commands looks like it's supposed to. And it seems you are also on Android 10 MIUI 12 just like me...

I don't know what could be the problem on your device...

What you can try to do is restart from scratch by doing a clean flash of the super.img from the Xiaomi firmware folder using

fastboot flash super super.img

And then run the script again but this time redirect the output to a text file and then send me the text file so I can look through it. Maybe there is a hint somewhere on what went wrong.

For redirecting the script's output to a text file you can do this:

./makesysrw.sh in=./super_original.img>./mylog.txt
[Android Software/Hacking General [Developers Only]] Blackwe3l:
My device is realme 6i, the firmware was downloaded from the manufacturer's website, I unpacked super.img using imjtool because the script does not recognize it, then I ran the script, at the end there was an error due to 1GB of extra space, not to mention 100MB. removed # from resize2fs -f -M $ i and still an error! I just need rw, how can I remove it from the script? Can I process an image without connecting a device? I want to flash via SPFT.
[Android Software/Hacking General [Developers Only]] lebigmac:
Thanks for the log. I will look into it.

You have lots of partitions nested inside your super.

Maybe your issue can be fixed simply by adding another if statement to the script that checks the name of the partition and only modifies it if it's equal to system or vendor or whatever.

And then shrink every partition back down to its original size before merging into new super and flashing

lpmake E 03-05 13:41:05 3133 3133 builder.cpp:698] [liblp]Not enough free space to expand partition: system
Not enough space on device for partition system with size 6035427328

I wonder what this lpmake error means... Does it mean there is not enough space on the harddrive? Or not enough space in the new super image file or what?!
[Android Software/Hacking General [Developers Only]] Blackwe3l:
I use a translator, honestly the translation is stupid. I think that the problem is in the system section, I unpacked super, mounted vendor, common_preload they mount without problems, but I can’t mount the system. It looks like the system cannot be resized while others can be resized. Do not think that the super.img image is corrupted, the image is intact. There is enough free space on the hard disk.
[Android Software/Hacking General [Developers Only]] munjeni:
Looks like you're out of space on your hard disc!
[Android Software/Hacking General [Developers Only]] lebigmac:
Over 130 downloads later I decided to make a quick update to the script. Please check the first post for download link!

I hope this version works a little bit better for some folks. Please report back if it works for you or not!

Good luck!
[Android Software/Hacking General [Developers Only]] kory-vadim:
Adding 1 GB to the size is naturally a lot.
[Android Software/Hacking General [Developers Only]] lebigmac:
The extra space is necessary for the unshare_blocks command to go through (see twitter post of @topjohnwu). The new version 1.03b resizes each partition back to its original minimum size after the unshare_blocks command is executed. Give it a try!
[Android Software/Hacking General [Developers Only]] kory-vadim:
[QUOTE = "lebigmac, post: 84604349, member: 8039604"]

Дополнительное пространство необходимо для выполнения команды unshare_blocks (см. Сообщение в твиттере [USER = 4470081] @topjohnwu [/ USER]). В новой версии 1.03 размер каждого раздела возвращается к исходному размеру после выполнения команды unshare_blocks. Попробуйте!

[/ЦИТИРОВАТЬ]

На кухне unshare_blocks не использую, в принципе не надо, достаточно увеличить размер, чтобы было свободное место 20 мб и более.

Если в образе нет свободного места, то перепаковывать нет смысла.
[Android Software/Hacking General [Developers Only]] kory-vadim:
Тем не менее, "unshare_blocks", вероятно, понадобится при изменении размера с помощью resize2fs.

У меня на кухне "unshare_blocks" не используется, потому что образ распаковывается и перестраивается с увеличенным размером и без "share_blocks".
[Android Software/Hacking General [Developers Only]] Blackwe3l:
V1.03

makesysrw: Joining all extracted images back into one single super image...

lpmake I 03-05 20:07:57 3069 3069 builder.cpp:1012] [liblp]Partition common_preload will resize from 0 bytes to 1393737728 bytes
lpmake I 03-05 20:07:57 3069 3069 builder.cpp:1012] [liblp]Partition oppo_engineering will resize from 0 bytes to 1074003968 bytes
lpmake I 03-05 20:07:57 3069 3069 builder.cpp:1012] [liblp]Partition oppo_product will resize from 0 bytes to 1081790464 bytes
lpmake E 03-05 20:07:57 3069 3069 builder.cpp:698] [liblp]Not enough free space to expand partition: system
Not enough space on device for partition system with size 6065672192

makesysrw: Error! failed to create super_fixed.img file

Reduced from 1024MB to 50MB in bytes too

makesysrw: Joining all extracted images back into one single super image...

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition common_preload will resize from 0 bytes to 372424704 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition oppo_engineering will resize from 0 bytes to 52690944 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition oppo_product will resize from 0 bytes to 60477440 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition system will resize from 0 bytes to 5044359168 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition vendor will resize from 0 bytes to 699637760 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition version.19677 will resize from 0 bytes to 52690944 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition version.19678 will resize from 0 bytes to 52690944 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition version.19746 will resize from 0 bytes to 52690944 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition version.19747 will resize from 0 bytes to 52690944 bytes

lpmake I 03-05 23:05:08 8851 8851 builder.cpp:1012] [liblp]Partition version.19748 will resize from 0 bytes to 52690944 bytes

lpmake E 03-05 23:10:04 8851 8851 images.cpp:338] [liblp]Image for partition 'system' is greater than its size (5364744192, expected 5044359168)
[Android Software/Hacking General [Developers Only]] lebigmac:
Where does this 300 MB size difference come from?

This definitely requires some more thorough investigation and research...

Have you tried to run lpmake command manually with the extracted images using the awesome Brepro1 tutorial? Also check out the very helpful AndyYan tutorial while you're at it and check the helpful comment by gabrielfrias

Maybe if you do it manually we can figure out what exactly is causing this error on your machine and fix it
[Android Software/Hacking General [Developers Only]] Lossyx:
Have you done it successfully? I was never able to resize the partition back to its original size, i had to delete stuff in it first or resize the partition in fastbootd.
[Android Software/Hacking General [Developers Only]] lebigmac:
Well after the e2fsck unshare_blocks command the extracted partition is being resized back down to its minimum size in version 1.03b. That's what I meant.

In version 1.0 this resize to minimum function was disabled (resize2fs -M partition) and each partition had 1 GB of extra space so you can write stuff to it later. Maybe 1 GB extra space for each partition was a bit too much. Some people have 10 partitions embedded inside their super! I only have 3 : system, vendor and product.

I think the act of reduping the partition itself already increases its original size by a few bytes.
[Android Software/Hacking General [Developers Only]] lebigmac:
To the 6 people who downloaded version 1.03 today please download version 1.03b !!!

1.03 had a little bug that is now fixed in 1.03b !!!
[Android Software/Hacking General [Developers Only]] Blackwe3l:
No I changed your script and ran, then copied the result at the end
[Android Software/Hacking General [Developers Only]] Blackwe3l:
Happened

Spoiler content hidden.

Thanks for helping
[Android Software/Hacking General [Developers Only]] Az Biker:
Please post in English. If a second language is required, post it below the English comment as I have here. Thank you.

Пожалуйста, размещайте на английском языке. Если требуется второй язык, опубликуйте его под комментарием на английском языке, как у меня здесь. Спасибо.
[Android Software/Hacking General [Developers Only]] gearmaker:
Thank you for your work on the new script.It ran and completed without errors.It also flashed the super_fixed.img but unfortunately for me it resulted in a bootloop for my Oneplus 7T Pro.

I was able to fastboot flash the super original image and all was well again. I did not try the

disable verity and verification. I will try again running those 2 commands first and see what happens.
[Android Software/Hacking General [Developers Only]] kory-vadim:
I told you not to do "resize2fs -M", the image must have free space, otherwise nothing can be added to it. There is no need to convert to sparse either.
[Android Software/Hacking General [Developers Only]] lebigmac:
I know. Resizing partition back to minimum doesn't really make too much sense but apparently now the script works better on more devices.

You can always debloat lots of unnecessary system apps and free up space that way (always make backups first!). I'm sure there must be another way to increase partition size afterwards...

More research is required... stay tuned!

By the way the script converts the partitions to sparse to avoid that annoying lpmake invalid header magic warning ;)
[Android Software/Hacking General [Developers Only]] kory-vadim:
"invalid header magic error" - just lpmake says that the image is "not sparse" (send it to / dev / null if it interferes), and it doesn't affect anything, now the script is just wasting time on converting, and lpmake will write it anyway raw image.
[Android Software/Hacking General [Developers Only]] lebigmac:
@kory-vadim

Thanks for the suggestion. If it really makes no difference then feel free to remove that part if waiting 2 additional seconds bothers you so much. That's the beauty of open source software. You can modify it to your liking!

I'd personally rather not get that annoying warning and waiting 2 seconds more doesn't really bother me too much!

I guess it's just personal preference.
[Android Software/Hacking General [Developers Only]] kory-vadim:
Your script is not useful to me, I already said that I repackage super.img directly on the phone. Just suggested how to do it more efficiently.
[Android Software/Hacking General [Developers Only]] lebigmac:
Those folks at 4pda are a step ahead of us that's for sure ;)

@kory-vadim

If you have a better method please feel free to share it with us. The more people have read-write access to their devices, the better for the open source community.
[Android Software/Hacking General [Developers Only]] kory-vadim:
Are you laughing, what hackers, ordinary people who take knowledge from open sources, including from xda, well, you still need to connect your head.

There is no other method, except for rebuilding super.img (at least once, to remove readonly).;)
[Android Software/Hacking General [Developers Only]] d4ylight:
Thanks for pooling all of those resources together @lebigmac !

Unfortunately I've run into a problem while running your script.

Spoiler content hidden.

A little info about my device:

model: moto g stylus (XT2043-4)

If you need more information, don't hesitate to ask! :)
[Android Software/Hacking General [Developers Only]] lebigmac:
A new day, a new error :)

So it seems the superunpack successfully extracts your nested partitions from your super.img.

And another potential problem I just noticed is that superunpack is skipping dump of the empty partitions. This could lead to more problems further down the line when the new super_fixed.img is created using lpmake...

Can you run this command manually on any of the extracted partitions in terminal? Or is it only the product_a.ext4 that is giving you trouble?

resize2fs partition_name.ext4 3G

Another question : can you mount for example system_a.ext4 using this command on your computer? And do you see any files if you navigate with nautilus to /mnt/dir ?

By the way do you use virtual linux machine or real linux machine?

sudo mkdir /mnt/dir
mount -t ext4 -o ro system_a.ext4 /mnt/dir

That's a cute phone BTW ;)
[Android Software/Hacking General [Developers Only]] d4ylight:
It affects every partition extracted.

These are the current contents of the folder that I ran the script in:

makesysrw.sh

product_a.ext4

system_a.ext4

tools (directory)

myargs.txt

super_original.img

system_b.ext4

vendor_a.ext4

All the same error:
[Android Software/Hacking General [Developers Only]] dewettie:
Anyway to make the script work so you can edit partitions directly in Linux to delete files before they are rebuilt and packaged into a new super.img so you don't manually have to edit the system, product, vendor partitions on the phone itself? Maybe mounting them in Linux and an easy way to unmount after you edited?

Also Mediatek firmwares use a scatter file to load the sizes of partitions into it's flash tool.

If super partition is bigger in size or different, the flash tool throws an error message that super.img can't be loaded. Anyway to make super.img compatible with flashing through Mediatek flash tool for Mediatek devices? I think flashing through fastboot will also cause problems if it doesn't match with sizes int the scatter file.

Flash Tool

SP Flash Tool v5.2104 for Windows - MediaTek Flash Tool

Download SP Flash Tool v5.2104 to flash or install Stock Firmware ROM (Flash File) on the devices powered by Mediatek Chipset.

spflashtools.com

SP Flash Tool v5.2104 for Linux - MediaTek Flash Tool

Download SP Flash Tool v5.2104 for Linux to flash or install Stock Firmware ROM (Flash File) on the devices powered by Mediatek Chipset.

spflashtools.com

Link to some Android 10 non a/b firmware

Just unzip and load the scatter file in the flash tool, then it won't be able to load the modified super.img file.

UMIDIGI_A9_PRO_128GB_V1-0_20210127

MediaFire is a simple to use free service that lets you put all your photos, documents, music, and video in a single place so you can access them anywhere and share them everywhere.

www.mediafire.com

I hope the script will also be able to work easily with Mediatek firmware in future.
[Android Software/Hacking General [Developers Only]] lebigmac:
Good news for the Windows and Mac users: universal version is immiment! Just had a major break-through in development! Stay tuned!

Anybody know how to create a flashable zip? Please contact me! I'm new to this whole concept of flashable zip's
[Android Software/Hacking General [Developers Only]] lebigmac:
I like your idea of having the option to manually edit all the interesting files on the computer before flashing them back to the phone. I think @giubenez had this idea before as well. This would probably require some kind of graphical user interface which shouldn't be too difficult to create.

I'm not too sure about Mediatek SoC's. Keep in mind this whole project was developed on a Snapdragon device (Xiaomi X3 NFC)
[Android Software/Hacking General [Developers Only]] lebigmac:
@d4light thanks for sending me your log. It looks like on your Motorola device you have this option in your parameters when you run the mount command : errors=remount-ro

/dev/block/dm-3 on / type ext4 (ro,seclabel,relatime,block_validity,delalloc,barrier,user_xattr,errors=remount-ro)

Maybe this is what's causing resize2fs to fail. Great. Another one of those read_only 'features"... Anyone know how to disable this one? @topjohnwu ? Anyone?
[Android Software/Hacking General [Developers Only]] lebigmac:
Hi @Lossyx

Is there a difference between regular fastboot mode and fastbootd mode? My fastboot command doesn't have the resize-logical-partition option...
[Android Software/Hacking General [Developers Only]] d4ylight:
According to this: (link here)

In order to enable Fastbootd on any Android device:
You need to Add fastbootd to PRODUCT_PACKAGES in device.mk: PRODUCT_PACKAGES += fastbootd.
Make sure that the fastboot HAL, boot control HAL, and health HAL are packaged as part of the recovery image.
You need to add any device-specific sepolicy permission required by fastbootd.

I will say though that it mentions booting into stock recovery...
[Android Software/Hacking General [Developers Only]] lebigmac:
@d4ylight the new version 1.04 is almost ready!

I can't guarantee that it will fix your resize2fs problem though. Give it a try!
[Android Software/Hacking General [Developers Only]] Lossyx:
Yes there is. Fastbootd is for managing dynamic/logical partitions. Only devices launched with Android 10 has this.

You can boot into it with fastboot reboot fastboot from regular fastboot (bootloader interface) or adb reboot fastboot from OS.

The text d4ylight quoted is just for people building custom recoveries.

Also be sure you have up to date platform tools.
[Android Software/Hacking General [Developers Only]] lebigmac:
Good news : the new version 1.04 is ready! I just uploaded it to the first post.

Now you can set a custom amount of free extra space (in megabytes) for each partition using the size=x argument.

You can also specify the input and output file name as requested by @giubenez using both in=x and out=x parameters.

The usage arguments are a little bit different this time, similar to the dd command. For more usage details check the first post!

Also cleaned up the code a bit.

I didn't have the chance yet to implement the adb exec-out suggestion of @Alecxs to speed up the dumping process. Maybe in the next version!

This is not the universal Windows / Linux / Mac version just yet. The universal version should be released this week though so stay tuned!
[Android Software/Hacking General [Developers Only]] Bob_Howard:
Hi lebigmac,

Complete noob with Android so I apologise.

I'm trying to make my Pixel 3 RW.

Rooted the device and everything is fine with adb but I'm getting an error when running the script:

View attachment 5244613

I confirmed it was running Android 10 as it had me doubting myself!

Can you suggest where I'm going wrong or what the issue might be?

Many thanks!

Bob
[Android Software/Hacking General [Developers Only]] Lossyx:
Your device needs to be **launched** with Android 10 or newer. Although I do believe Google repartitions their phones over OTA.

Manually navigate to /dev/block/by-name and see if you have a super partition.
[Android Software/Hacking General [Developers Only]] Bob_Howard:
Hi Lossyx, Thanks for a quick reply!

Just checked and there's no super there...

View attachment 5244623

I'm guessing I'm SOL on this then?

Hmmmmm...
[Android Software/Hacking General [Developers Only]] lebigmac:
Hi @Bob_Howard thanks for giving the new version a try and for your valuable feedback.

Apparently you don't have a super partition.

Are all your partitions mounted as read-only? You can check this by running this command on the phone:

mount

Have you tried to manually remount your partition as read-write using this command?
[Android Software/Hacking General [Developers Only]] Bob_Howard:
Hi @lebigmac, thank you for helping here!

Tried the remount command:

View attachment 5244663

mount

Gives me 57 entries. The only ones that are read-only are:

View attachment 5244667
[Android Software/Hacking General [Developers Only]] lebigmac:
Thanks for the feedback @Bob_Howard

Can you dump /dev/block/dm-0 to your internal storage and then successfully resize it?

su
dd if=/dev/block/dm-0 of=/data/local/tmp/system.img
resize2fs /data/local/tmp/system.img 3G

Or is resize2fs going to complain about unsupported read-only feature(s) error just like on the Motorola of @d4ylight
[Android Software/Hacking General [Developers Only]] Bob_Howard:
That all worked fine:

View attachment 5244691
[Android Software/Hacking General [Developers Only]] lebigmac:
Okay Bob thanks for confirming this. I'll try to write another additional function this afternoon for devices such as yours that don't have super image but still need a solution to make their system read-write-able
[Android Software/Hacking General [Developers Only]] Bob_Howard:
dupe post!
[Android Software/Hacking General [Developers Only]] Bob_Howard:
That will be awesome! Many thanks!
[Android Software/Hacking General [Developers Only]] ronlkh1965:
Hi, I would like to know if phone do not have the super.img any way to make it works?
[Android Software/Hacking General [Developers Only]] d4ylight:
@ronlkh1965 Referring to this post, he is going to try to make it work. :)
[Android Software/Hacking General [Developers Only]] ronlkh1965:
Yes, i am running ubuntu on VMware in Win10. I will try again.
[Android Software/Hacking General [Developers Only]] ronlkh1965:
Thanks, looking forward to hearing from him
[Android Software/Hacking General [Developers Only]] lebigmac:
Hello @ronlkh1965 thanks for your message.

Can you please connect your phone to your computer and run these 2 commands and show me the output? Thanks!

adb shell ls -al /dev/block/by-name

adb shell mount
[Android Software/Hacking General [Developers Only]] lebigmac:
To all those that have no super image on their Android 10+ device, which partitions do you want to make read-write? I'm assuming system, vendor and product? That's it?

And all these partitions reside in one of those /dev/block/dm-0 locations right? dm-0, dm-1, dm-2, dm-3 ...

I always thought that every device that runs Android 10 or above always has a super image. I guess I was wrong...