yashsingh1/xdaforums huggingface dataset:
[Android Software Development] djsubterrain:
Hoping someone can help with my TWRP test build for OnePlus 9 Pro, I'm following the guide below :

How to Create and Port TWRP Recovery to any Android Device

In this tutorial, we will show you detailed steps to create and port TWRP Recovery for any Android Device using Android Image Kitchen tool.

www.droidwin.com

I have the Linux working on my Windows 10, downloaded AIK for Linux 3.7 as advised, I also downloaded the TWRP for LG G3 as it's the closest I could see to the correct resolution.

unpackimg works fine with the already built TWRP for LG G3 but when trying to unpack the boot.img from the latest EU build for the OnePlus 9 Pro (There's no recovery.img from Payload Dumper) the unpacking fails with the following :

No folders are created.

I also tried with the latest Windows based AIK from here, the split_img and ramdisk folders get created but the only thing that gets unpacked is a "boot.img-origsize" file inside the split_img folder with a size of 1KB, the contents show just one line with "100663296"

There's no log shown to confirm what has happened either. Not sure if there's a verbose flag to see the log but trying to launch with either -v or /v just shows the same error "No image file supplied."

Could it be a similar issue to previous posts?
[Android Software Development] osm0sis:
Yeah you can clearly see it says header version 3, and the errors are exactly the same as the post one page ago. Read my last reply.

I'm finishing regression testing AIK v3.8 today I hope. Went down a rabbit hole tracking an xz 5.2.5 regression that only manifests on Android arm64, then my Ubuntu VM broke itself upgrading to latest LTS since I wanted to make sure distro xz 5.2.5 was fine there too. 😅
[Android Software Development] djsubterrain:
Thanks so much for the confirmation and thanks for putting in your own time for this. More so appreciative given your "day job", I'm guessing tinkering with this stuff maybe takes your mind off what you see there.
[Android Software Development] Postman1year:
At first thank you very much for new AIK 3.8, I have use AIK Win32 version 3.7 for about one year.

But I found a problem for Today's New AIK 3.8 (Android.Image.Kitchen.v3.8-Win32.zip), When I try to unpack stock Boot.img by using AIK3.8 unpackimg.bat,

I just get this message "Setting up work folders . . ." , and Nothing extract found, I wait for 1 hour but still nothing.

The Same boot.img unpack and repack in AIK 3.7 (Android.Image.Kitchen.v3.7-Win32.zip) is very perfect and NO any problem Every Times.

Please CHECK, thank you very much.
[Android Software Development] FearNaiim:
Hello

Thank you for your AIK 3.8. I'm trying to download AIK for linux but the link is not work. Can you please check

Thanks again
[Android Software Development] osm0sis:
If you read what it says you would already have your answer 🤨:

AIK-Linux-v3.8-ALL.tar.gz (waiting on xda to allow .tar.gz uploads again :rolleyes:)

Note the file link goes to "blame-xda" 😉

Anyway, I just got a reply they've fixed it, so I'll be uploading it and making the release post shortly.

Edit: Nope, still won't accept .tar.gz for some reason... so the wait continues. I'll release as soon as xda will allow. 🤞
[Android Software Development] FearNaiim:
Same as here except I'm getting two files in Split_img folder View attachment 5297981
[Android Software Development] FearNaiim:
Thank you and sorry for now understanding the problem
[Android Software Development] osm0sis:
I didn't make any changes to that part of unpackimg.bat: https://github.com/osm0sis/Android-...c7fd7537793fee48ffd37025432bb6b62e153c3f5fb07

It works fine for me here in multiple locations; I just tested in Documents and on my Desktop, and I also tried into a random Cygwin directory yesterday while testing the --local command line option.

Unzip AIK v3.8 zip into a different folder and try again.
[Android Software Development] Postman1year:
Sorry, I have try Unzip Android.Image.Kitchen.v3.8-Win32.zip into several different folder, C, D drive, Destop, etc..

The problem are the same, "Setting up work folders . . ." , but wait long time

just getting two files in Split_img folder
[Android Software Development] osm0sis:
So it's clearly hanging for you guys right here at the first Cygwin commands it encounters: https://github.com/osm0sis/Android-Image-Kitchen/blob/master/unpackimg.bat#L67

Remove any AIK v3.7 folders you've got and reboot since the different Cygwin dll versions might be conflicting.

Again, it works totally fine for me both right now and throughout the last several weeks of development.
[Android Software Development] Postman1year:
Because get your hint for ( Cygwin ), SO

I replace 13 cyg*.dll FROM old AIK 3.7 to new AIK 3.8, same folder(android_win_tools)

the problem SOLVE
[Android Software Development] osm0sis:
Now you're using old dlls with executables built using the new dlls, which could lead to other problems for you.

Try my suggestions first. See my above edit in my previous post.
[Android Software Development] Postman1year:
I have Remove all AIK v3.7 folders and reboot for three times, problem are the same, hang....

old dll could lead to other problems, I know, but now

the only WAY to SOLVE is replace the 13 cyg*.dll from 3.7 to 3.8

OR return to perfect 3.7
[Android Software Development] CXZa:
It stops to cut. You could use "for" instead...
[Android Software Development] osm0sis:
Here's a hint, it's not perfect or we wouldn't need v3.8 😛

Since I can't reproduce the issue can you at least experiment with reverting only one dll at a time and find out exactly which one is causing you guys problems?

I guess we'll call this the public beta now. 😑

Not really a solution where there are a number of complex cuts and pipes, and it used to work fine for everyone. (Though still also does work fine for me.. 😕)
[Android Software Development] Postman1year:
ONLY Replace just ONE File ( cygwin1.dll ) from 3.7 to 3.8 , CAN Solve the HANG problem.
[Android Software Development] CXZa:
Yes, I noticed. The scripts calls the same quite many times...

I probably would just edit androidbootimg.magic so it gives the right output without cutting or forring(?).
[Android Software Development] osm0sis:
Now try replacing every dll except that one and see if it works. 👍

Nah, the additional output is used elsewhere, and yeah batch is hacky..
[Android Software Development] CXZa:
Well, if it's too complicated, use another one elsewhere...

That dll checking is ridiculous.. is a newer dlls even needed "elsewhere"...
[Android Software Development] osm0sis:
What? Cygwin dlls belong with the executables built using them or thar be dragons.

Updating and rebuilding the binaries with the latest Cygwin dlls has been my practice since the beginning of AIK 9 years ago and there's never been a problem before; newer Cygwin usually means less bugs, not more.

If you have a problem with how I'm trying to troubleshoot this you can feel free to rejoin the thread later.
[Android Software Development] Postman1year:
cygwin1.dl must replace from 3.7 to 3.8, those other 12cyg*.dll not need,

same result whether 12cyg*.dll are 3.7 or 3.8

but cygwin1.dl must be 3.7 otherwise it HANG
[Android Software Development] osm0sis:
Thanks for looking into it. I'll investigate more later today and get back to you via PM hopefully with a working test build.

You're Windows 10 x64 presumably?
[Android Software Development] Postman1year:
Yes, Windows 10 x64 20H2 , 19042.685, Pro, and I come from Macau so I use the Traiditional Chinese Win10
[Android Software Development] CXZa:
No problem. Just tried to help. Have fun doing it in the hard way...

(BTW. it sticks when using win7 32-bit too...)

Have you new exe(s) for 3.8 as cut is the same as in 3.7.

If it matters...
[Android Software Development] osm0sis:
Those are pulled from Cygwin's /bin directory. I have a script that diffs them and updates them in AIK at release time as needed. Sometimes the Cygwin devs update them at the same time they update Cygwin's core, sometimes they don't. I guess logically that means there's at least some basic cross-compatibility across dll versions.. 🤷‍♂️

Weird. I'm on Win10 32bit. No clue how I'm somehow unaffected. Maybe since I have an actual Cygwin installation? 🙁🤦‍♂️

My plan is to properly roll back the cygwin1.dll in my Cygwin installation if it'll let me, then rebuild the AIK tools from that, just to be safe and keep everything matching.
[Android Software Development] osm0sis:
Looks doable!

Judging by the cygwin and cygwin-devel package histories AIK v3.7 would have been built on Cygwin 3.1.4-1.

Latest, which is in AIK v3.8, is 3.2.0-1, so I'll downgrade and try building with 3.1.7-1 and we'll go from there later today after I get some sleep. 👍

I'll PM the three of you.

Edit: Sleep is for the weak! Results from the PM testing seems like that did the trick! Thanks again to you all! Hell of a showstopper bug to find on release day and not even be able to reproduce it myself! 😅🙌
[Android Software Development] djsubterrain:
I'm a little confused on how to use this, I downloaded the updated 3.8 (win-master) but it's pretty barren and has the same result as the previous. Are all the links on the original post pre-requisites? I had tried installing CygWin but there's tons of different sections, had no clue which are/are not required.

A how-to in the original post would help immensely (I know from experience they're a ballache to do)
[Android Software Development] osm0sis:
Not sure what you mean "barren" 🤔

Instructions are already in the OP. Everything needed is self-contained here; the OP talks about not needing full Cygwin.

v3.8 unpacks hdr_v3 images without any issues:

Android Image Kitchen - UnpackImg Script
by osm0sis @ xda-developers

Supplied image: aosp-hdr3-avb2-rawzimg-gzrd-oneplus_9pro-boot.img

Setting up work folders...

Image type: AOSP

Signature with "AVBv2" type detected.

Splitting image to "split_img/"...
ANDROID! magic found at: 0
BOARD_KERNEL_CMDLINE
BOARD_PAGE_SIZE 4096
BOARD_OS_VERSION 11.0.0
BOARD_OS_PATCH_LEVEL 2021-02
BOARD_HEADER_VERSION 3
BOARD_HEADER_SIZE 1580

Unpacking ramdisk to "ramdisk/"...

Compression used: gzip
81778 blocks

Done!

If you're sure you've got the latest v3.8 zip from the OP and it's still not working then upload your boot.img and I'll see if I can reproduce this issue. Everyone else is reporting success now.. 🤨

Edit: Just reread your post, and you mention something about "win-master" and it being empty/barren, making me think you just downloaded the script files from GitHub without any binaries? The zips in the OP here are the only way to download a full, working AIK package, so try those and I have a strong feeling it'll work for you now. 🙃

Alright! Let's do this release! 🤘
[Android Software Development] osm0sis:
Hey all!

[ Release removed ... ]

Edit: So, uh, we'll call this a "soft launch" I guess and do a proper release once these regressions are fully worked out. :(
[Android Software Development] djsubterrain:
100% correct, Downloaded the updated FULL zip and will give it another go, thanks once again for all your hard work

View attachment 5298699

That's more like it :)
[Android Software Development] djsubterrain:
How big a problem is this? Unpacked an existing TWRP for LG G3 which supposedly has similar resolution to the OnePlus 9 Pro, got these messages during the unpack:

Got no errors or warnings at all while unpacking the boot.img for the OnePlus 9 Pro though. Do those messages mean those files have already been extracted so wouldn't be replaced?

Actually, saying that, I'm getting 0KB files using both the Windows and Linux version :

View attachment 5298719

Most importantly the default.prop which won't let me open it either. The Linux unpack was way quicker than the Windows one, I'm using the Linux script under Ubuntu subsystem for Windows 10 but I've updated it with apt to latest, got the following from the Linux script :
[Android Software Development] osm0sis:
That exact warning is already mentioned in the v3.8 release notes above. 😉

0kb "files" = symlinks. :p

default.prop is just a symlink to /system/etc/prop.default on modern devices. On something a little older prop.default is still in the ramdisk, but be aware in a boot-as-recovery device that ramdisk prop.default might just be for recovery only. 😬

Downgrade your distro to cpio 2.12 or you'll have a bad time. 🙂
[Android Software Development] raystef66:
@osm0sis works fine now for hdr_v3 support - thnx for your hard work :cowboy:
[Android Software Development] Postman1year:
Yes, for my twrp352.img, not only default.prop be zero kb, but all symlink file be zero kb now after use AIK 3.8 to unpack, there are many symlink important Functionable file change to NO Function by using AIK 3.8 !!!!!!!

but, using AIK-3.7 are always no problem till now, SO I want to know how to Solve this 0kb problem if I continute to use AIK 3.8, now I only can return AIK-3.7 to get functionable TWRP.

the picture show the diff of 3.7 and 3.8, AIK 3.8 set all the symlink file to 0kb, so no function for these files, but the files are very important !! (most important files in SBIN folder)
[Android Software Development] CXZa:
That's one way until O fixes it. Same dll problem.... try this .. ;)
[Android Software Development] Postman1year:
OH, you are RIGHT!!!!!

after ONLY Replace just ONE File ( cygwin1.dll ) from 3.7 to 3.8 , problem Solve !!!

all files are the same !!!!!
[Android Software Development] osm0sis:
Oh they're actually non-functional in Windows? I assumed he just didn't understand the symlinks, since AIK-Linux is for sure working fine.

🤦‍♂️

Turns out my downgrade to 3.1.7-1 I think also nuked my Cygwin NDK build environment too, so uh, ****. 🙁

I'll PM you guys again once I get it working here.

Edit: Ah, good news! Downgrading Cygwin further to 3.1.4-1 still reproducibility provides the working dlls for AIK, so I'll see what the last working Cygwin update was and we'll probably freeze it there. 🤞

Edit 2: Aaannd it's 3.1.4-1 after all. Even 3.1.5-1 results in 0kb symlinks. I'll rebuild the binaries from there and we should be good again. I may have to report this bug to Cygwin. Seems like pretty serious breakage of cpio/symlink functionality. 😕
[Android Software Development] Postman1year:
Yes, they're actually non-functional in Windows.

So, I can only trouble you, sorry, I just want AIK-3.8 be perfect
[Android Software Development] osm0sis:
Haha no no I'm glad you guys are finding stuff this fast. Here I thought I was so fancy and thorough with all my automated regression testing and yet missed this basic breakage when I updated the Cygwin libraries 2 days before release. 😅

Working night shift at the hospital tonight but should be able to upload a fixed release build tomorrow in the day at some point. 👍

Edit: Aw sweet! Chrome Remote Desktop'd into my build PC from work and it seems like downgrading Cygwin back to 3.14-1 also fixed my NDK build environment in Cygwin, so uh, damn, the Cygwin devs ****ed up wildly lately I guess but at least I'm back in action. 🤷‍♂️💪
[Android Software Development] Postman1year:
These are not urgent. There will be no problems after this weekend or beyond. I am not able to help. I can only say that pay attention to rest, body is most important.

sorry for my poor chinese english
[Android Software Development] scarlet-glass:
Android Image Kitchen-UnpackImg script

By osm0sis @ xda-developers

Image provided: boot.img

Set the working folder. . .

Unrecognized format.

error!

Hi @osm0sis

Android Image Kitchen-UnpackImg脚本

由osm0sis @ xda-developers

提供的图像：boot.img

设置工作文件夹。。。

无法识别的格式。

错误！
[Android Software Development] Postman1year:
It may be the format of your boot.img file. Please upload it or give it a network connection when you have time. Let me study it, thank you.
[Android Software Development] osm0sis:
Yep, took the words out of my mouth. 🙂

XDA enforces English-only on their forums though, so I think you at least need to put both. 👍
[Android Software Development] Hen Ry:
Why does the size of the img change (become smaller)?

miui 12.5 boot.img - device venus - Mi11

before 196.608 KB -> after repack 67.040 KB
[Android Software Development] Hen Ry:
Sorry, post was double 🤷‍♂️
[Android Software Development] Postman1year:
before 196.608 KB -> after repack 67.040 KB

IT IS normal by repack boot.img or recovery.img, for exampe, xiaomi mi 9, original boot.img IS the partition size, 128mb, when I unpack and repacked IS 33mb， and the original recovery.img IS 64mb， unpack and repack IS 28mb only

But there are big problem for these REPACK image such as BOOT.IMG & TWRP-RECOVERY.IMG for AVB2.0, control hash & sha256 by other IMAGE call VBMETA.IMG, so after FLASH the new BOOT.IMG, will casue BOOTLOOP, so you must CRACK the VBMETA.IMG by using HXD and reflash to VBMETA Partition.

Crack VBMETA can use to flash MOD-NEWBOOT.IMG and NEW-SYSTEM, NEW-VENDOR

but Can Not function to get the DATA decrypt for YOUR TWRP-RECOVERY.IMG, just like many TWRP3.5.2 for MI 9 (such as all TWRP download from TWRP.ME, all file I try, NO DATA decrypt !!!!), FOR RECOVERY DATA DECRYPT, AT FIRST, you must ADD 00 by using HXD to ORIGNIAL RECOVERY PARTITION SIZE, and MOD the AVB2.0 FOOTER.
[Android Software Development] osm0sis:
It's because most of the boot/recovery partition is empty space. That question has been answered many, many times throughout this thread.
[Android Software Development] ineedroot69:
why does your window version tool output more files and folder?

the left one is this https://xdaforums.com/t/util-unmkbootimg.1877807/

the right one is your tool
[Android Software Development] osm0sis:
Because mine is better.
[Android Software Development] ineedroot69:
View attachment 5313787
[Android Software Development] osm0sis:
Jokes aside, that 9 year old thread/binary clearly doesn't extract symlinks or empty directories correctly, so I'd be surprised if a repack using it ever worked at all. 🤦‍♂️

I'm also stunned nobody ever linked those poor folks trying to get that sh*t working over the years to AIK. 😕

Edit: Also not sure why that thread or tool existed at all, since CyanogenMod's unpackbootimg has been around since 2010 🤷‍♂️: https://github.com/CyanogenMod/android_system_core/commits/eclair/mkbootimg/unpackbootimg.c
[Android Software Development] ineedroot69:
run into this when using Linux version of AIK

boot.img decompile without issue but i can't open ramdisk/sbin extracted by AIK it say i don't have permission i have tried

chmod a+x * operation not permitted

chown twrp: kitch operation not permitted
[Android Software Development] ineedroot69:
i fixed it ... it took me goddamn 1hours to figure it out

just do a CD command on AIK directory then do a

sudo chmod -R 777 *
[Android Software Development] osm0sis:
Your repacked ramdisk will have those 777 modes now. Bad idea. You should sudo nano or whatever instead.

If for some reason you can't edit files as sudo in your setup then you could try the unpackimg --nosudo argument, but note you will lose any special ownership files potentially have on OEM images. It's mentioned in the AIK-Linux instructions and --help.
[Android Software Development] ineedroot69:
i seen that nosudo argument when i tried reading unpackimg but didn't understand what it does 🙃 also i found the culprit sbin folder is owned by user named root and from group named root i guess they denied permission not coming from that group

also it's stupid of me adding execute permission (a+x) should've have added read permission instead since i can't read it 🙃
[Android Software Development] osm0sis:
No you need to either use sudo to edit the files, or use --nosudo to unpack, as I told you. Your way will probably break the ramdisk.
[Android Software Development] .RyanBH:
Hi! I`m not good on all this things, but okey

I have A5 2016(A510F, a5xelte, how ever you want)

I have LineageOS with 10 Android on it

I am rooted

I want to build a kernel for kali nethunter, build zImage and download AIK

I unpack my stock boot.img (I extract it from adb with "adb pull /dev/block/.....")

I change boot.img-kernel with my zImage and repack it

After boot i have KERNEL IS NOT SEANDROID ENFORCING and bootloop

UPD: If you need something say pls
[Android Software Development] .RyanBH:
pls heelp
[Android Software Development] osm0sis:
No I can't help you fix your zImage. Go look up how to compile and package a kernel for your device. There's probably a thread in an appropriate forum already.
[Android Software Development] .RyanBH:
Okey, but it is a paradox that only for this device nothing about kernel and ROMs, I check Russian PDA and all links are broken, and nothing on XDA
[Android Software Development] chronolz:
Hi,

I am using AIK v3.8 to unpack and repack a custom kernel.

When I take a boot.img and unpack it, the 64MB image file unpacks to ~ 28 to 29 MB where the ...cpio.gz file is about 1MB and the ...img-kernel is about 28MB

Once I replace the kernel file with my custom kernel the repack cmd gives two file: image-new.img (28MB) and ramdisk-new.cpio.gz (1MB)

I am assuming that the repacked image-new.img file doesn't have the ramdisk-new.cpio.gz in it and if I only flash the image-new.img on to my s21ultra exynos then it will not touch the ramdisk already on the phone?

Summary, 1) unpack cmd unpacks the stock boot.img (64MB) such that the unpacked files (29MB) are smaller in size than the stock boot.img

2) repacked img is 29MB while the stock boot.img is 64MB, is it still ok to flash on the phone?

unpacking output...

Android Image Kitchen - UnpackImg Script

by osm0sis @ xda-developers

Supplied image: boot.img

Setting up work folders...

Image type: AOSP

Signature with "AVBv2" type detected.

Splitting image to "split_img/"...

ANDROID! magic found at: 0

BOARD_KERNEL_CMDLINE androidboot.selinux=permissive

BOARD_PAGE_SIZE 4096

BOARD_OS_VERSION 11.0.0

BOARD_OS_PATCH_LEVEL 2021-03

BOARD_HEADER_VERSION 3

BOARD_HEADER_SIZE 1580

Unpacking ramdisk (as root) to "ramdisk/"...

Warning: Using cpio 2.13 may result in an unusable repack; downgrade to 2.12 to be safe!

Compression used: gzip

3785 blocks

Done!

repacking output...

Android Image Kitchen - RepackImg Script

by osm0sis @ xda-developers

Packing ramdisk (as root)...

Warning: Using cpio 2.13 may result in an unusable repack; downgrade to 2.12 to be safe!

Using compression: gzip

Getting build information...

kernel = boot.img-kernel

cmdline = androidboot.selinux=permissive

os_version = 11.0.0

os_patch_level = 2021-03

header_version = 3

Building image...

Using format: AOSP

Done!
[Android Software Development] greeshan:
Thanks for the great tool! But i have question:

When i repack recovery.img (TWRP) without changes - all is ok, TWRP works, but if i add file to ramdisk (example i was add "busybox" to "ramdisk\sbin\") TWRP is no work, i have bootloop or "System corrupted..." message when boot to TWRP.

Maybe i should change size image or add new record about new file?
[Android Software Development] osm0sis:
You probably added something that makes the image too large for the partition?
[Android Software Development] greeshan:
I think no. I was add busybox binary (1 577 040 bytes).

original size TWRP.img = 134 217 728 bytes, after unpack (ramdisk + split_img dirs) = 109 175 390 bytes.

P.S. if i move files in "TWRP\ramdisk" from one dir to other dir without changes = all is ok, but if add new file = problem.

I think maybe i should make changes in config files (for example in "split_img\twrp.img-origsize" or other...) after added new file?
[Android Software Development] osm0sis:
Shouldn't make a difference, new files get default permissions, and just adding a busybox binary shouldn't break anything in itself.

If you're using Windows try on Linux with root, to see if that helps.