yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] Boscep:
[MENTION=4896512]Alexey71[/MENTION], your attached Extractor_v2.6.zip unable to download. so pls mirror it to androidfilehost or etc, thank you.
[Android Software/Hacking General [Developers Only]] Boscep:
[MENTION=4896512]Alexey71[/MENTION], big thumb for great tool. repack tool?
[Android Software/Hacking General [Developers Only]] Nishidh:
No it does not I have tried repacking system new dat but gave me file contexts not present error so i too need help
[Android Software/Hacking General [Developers Only]] Swapnil Soni:
I wonder how other devs pack it :confused:
[Android Software/Hacking General [Developers Only]] puargs:
Trying to alter a stock ROM and resign with my own custom key just for myself. This process will allow me to use a device with custom changes (NOT root), but still allow me to use Android Pay.

This process involves using the "make_ext4fs" tool. However, this tool (as far as I can tell) requires using the old plaintext version of "file_contexts". Android 7+ uses a new binary version of this file instead ("file_contexts.bin").

So if anyone can solve either of these questions, I would be very grateful:

1) Is there a tool to convert my device's binary "file_contexts.bin" to the old plaintext "file_contexts"? -OR-

2) Is there a build of "make_ext4fs" that accepts the new binary format "file_contexts.bin"? I tried to build the new nougat version myself from AOSP, but after a successful build of the tool, I can't get the thing to read files on either 64-bit or 32-bit systems. It yields an error "No such file or directory" when trying to point to file_contexts. I am not a great C developer, so I don't know if I built it wrong. For reference, no, I am not missing the lib32stdc++6 libraries. Here is the link to the binary if you want to see if you can get it to work. -OR-

3) If anyone has the file_contexts text file for the OnePlus 3, ROM version 7.X Nougat, that would work too!​

Any help anyone can offer in any way would be fantastic! Thank you so much for your time to look at this!
[Android Software/Hacking General [Developers Only]] shoey63:
[MENTION=3785303]puargs[/MENTION]

This may help https://xdaforums.com/showthread.php?p=70511303

Haven't tested myself though
[Android Software/Hacking General [Developers Only]] puargs:
That did the trick - thank you so much for the link. Saved me after an entire week of frustration. :good:
[Android Software/Hacking General [Developers Only]] svandasek:
my system.new.dat is small after this command ./rimg2sdat system_new.img . it has only 618,5kB!! Why? please help me out [MENTION=5132229]xpirt[/MENTION] [MENTION=3892016]luxi78[/MENTION] [MENTION=6262977]howellzhu[/MENTION]
[Android Software/Hacking General [Developers Only]] xpirt:
use img2sdat..
[Android Software/Hacking General [Developers Only]] svandasek:
Thanks a lot For your answer and for being online/active! Will try it today... Hope it will be successful as I am porting freedom os from op3 to 3T :)
[Android Software/Hacking General [Developers Only]] jmwuxtfz:
I understand already！
[Android Software/Hacking General [Developers Only]] svandasek:
[MENTION=5132229]xpirt[/MENTION] I think I have tried img2dat but got an error like "....Should be 0x00000 but is 2x78492....) After choosing 4(nougat). What am i doing wrong?
[Android Software/Hacking General [Developers Only]] xpirt:
everything is wrote in first post..
[Android Software/Hacking General [Developers Only]] svandasek:
Don't see any solution in the first post...
[Android Software/Hacking General [Developers Only]] xpirt:
you didn't follow all the steps..
[Android Software/Hacking General [Developers Only]] svandasek:
Hm, i don't see any steps in the first post. Do you mean the second post?
[Android Software/Hacking General [Developers Only]] xpirt:
yes, I do mean the 2nd post
[Android Software/Hacking General [Developers Only]] svandasek:
yeah! now my system.new.dat is 1.9gb! thx a lot, had to do it from step 1 in second post :)
[Android Software/Hacking General [Developers Only]] xpirt:
img2sdat 1.2:

- [bcf5e7d] added command line option to set Android version (thanks to [MENTION=5787964]SuperR.[/MENTION])

- [7d5e031] added command line option to set custom output path (thanks to [MENTION=445262]peter23[/MENTION])
[Android Software/Hacking General [Developers Only]] vjgrace:
Decreased Sparse Img File Size

I have followed the guide specifically (Ubuntu steps 1-4) and continue to have the same issue even when trying a variety of parameters. No matter what I try, I get ~350K sparse img file size which leads to ~350K system.new.dat file size. The ROM i am porting is CM-13.0. I have seen others have the same issue throughout this thread and even a very recent set of posts but have seen no resolve. Once again, I have followed the steps precisely.

Any help would be greatly appreciated.
[Android Software/Hacking General [Developers Only]] Boscep:
[MENTION=5132229]xpirt[/MENTION], many people can't download your files. so pls mirror it to androidfilehost or etc, ty.
[Android Software/Hacking General [Developers Only]] kykint:
Hi, I'm getting some weird results when converting my GS7's system.img to sparse dat with your latest img2sdat.py.

The size of stock, raw system image converted with simg2img is 4404019200B. However when I convert it to sparse image with ext2simg, then convert it to sparse dat with img2sdat.py, and convert it again to raw image with sdat2img, the output is only 4369301504B in size.

Also i noticed that when I use img2simg instead of ext2simg, the output image size was normal; 4404019200B.

This didn't happen when I used rimg2sdat, but I'm not quite interesting in using that since it's quite outdated. Is it normal, or am I missing something?
[Android Software/Hacking General [Developers Only]] I'mTheMr:
Hi

im trying to work on a rom

it losses root access when edited ...

its turned on from setting > developer options

based on cm12.1 official nightly

i added some apps and frameworks and media folder

any solutions ? thanks :)
[Android Software/Hacking General [Developers Only]] sattarvoybek:
[CODE]sattarvoybek@sattarvoybek-HP-620 ~/dev/unpack $ ./make_ext4fs -T 0 -S file_contexts -l 1610612736 -a system system_new.img output/
file_contexts: line 1 is missing fields, skipping
file_contexts: line 22 is missing fields, skipping
file_contexts: line 72 is missing fields, skipping
file_contexts: line 73 is missing fields, skipping
file_contexts: line 74 is missing fields, skipping
file_contexts: line 78 is missing fields, skipping
Creating filesystem with parameters:
Size: 1610612736
Block size: 4096
Blocks per group: 32768
Inodes per group: 8192
Inode size: 256
Journal blocks: 6144
Label:
Blocks: 393216
Block groups: 12
Reserved block group size: 95
error: build_directory_structure: cannot lookup security context for /system/addon.d
sattarvoybek@sattarvoybek-HP-620 ~/dev/unpack $

[/CODE]

Help please, what's the problem?
[Android Software/Hacking General [Developers Only]] ale8530:
Hello [MENTION=5132229]xpirt[/MENTION],

./make_ext4fs -T 0 -S file_contexts -l 1073741824 -a system system_new.img output/

This command is ok when file_contexts is txt file, how you have resolve problem file_contexts.bin of android 7?
[Android Software/Hacking General [Developers Only]] Tejas tedi:
Use this tool to convert bin to text file

replace the file_contexts.bin with yours and open run.bat

link: https://goo.gl/v8awjR
[Android Software/Hacking General [Developers Only]] ale8530:
Thanks
[Android Software/Hacking General [Developers Only]] Mr Hassan:
is there any angle for explain me how to do that, more easy,

i have installed python 3.6 + java etc, and i already downloaded the attech file , but idk how and where i take start?

i have extracted system folder need to pack again in ext4

:D
[Android Software/Hacking General [Developers Only]] Mr Hassan:
how to open this file ?
[Android Software/Hacking General [Developers Only]] Q9Nap:
[MENTION=5132229]xpirt[/MENTION]

Thanks for the updated versions of the tools!

I am wondering what the difference is in the way the system image is converted to sdat among the different android versions?

I converted a 7.0 stock rom for moto g4/plus first using the "4" setting, and it wouldn't flash.

I then converted the 7.0 rom with the "3" setting, and it works fine.
[Android Software/Hacking General [Developers Only]] Boscep:
hello [MENTION=5132229]xpirt[/MENTION]

create imgunpacker.exe & mkuserimg.exe pls, ty.
[Android Software/Hacking General [Developers Only]] triptaminer:
hello all. i know, this is for lollipop, but i need something like this for nougat :/ googling half of day but still cant find it :/ maybe this one will work, but in my *.transfer.list are lot of new commands not covered in this script. is there any chance to find updated version of it?

if someone wants give me some hints, here is small overview of my list (btw, its OTA for nougat but "3" in first line means MM? :/):

3

92739

62

10650

bsdiff 0 313822 07468a13da00a8b25ae31e0675fcb1e098f11ec0 1cffbdaa3632b9365d0ce561db8d69797f7b8f24 2,105091,105597 732 2,104881,105613

move d5343d01f632a44a61a4705c12b48362e2ca9f46 2,153122,153123 1 2,153098,153099

imgdiff 3337316 36972 7d0db3891a86b476b345ae8c5c8707613c707210 14a25255b98fb6bab79dd5bd79bec6a93585d37c 2,133956,133992 36 2,133956,133992

stash cbeb477ece0fc5338b1d76b4840b9d773e3a69b4 2,31547,31549

free 4bddb5bd0bba5918db2654f8c1f5de8001b9ec61

...

new 2,128055,128058

bsdiff 13911732 339073 f3934b65aa43edf65bf2b110711290d35e49d4a7 737d08873af846c6db5bb94d92f374d4683384ef 2,129263,130011 1045 2,66017,67062

zero 7946,523,524,525,526,...,53200,65024,66050,97792
[Android Software/Hacking General [Developers Only]] xpirt:
otas are not supported.
[Android Software/Hacking General [Developers Only]] Mr Hassan:
hello, dear supporter with OP,

i found a tool to repack that system folder ,but its need system.transfer list

any body tell where i found this file
[Android Software/Hacking General [Developers Only]] black dark man:
Can you give the link to the tool
[Android Software/Hacking General [Developers Only]] shlape:
I had a problem calculating the [-I] argument value (bolded in the quote above). I'm using Ubuntu and when I ran this command I get something like the following at the end of the output:

error: ext4_allocate_best_fit_partial: failed to allocate 83 blocks, out of space?

I tried recalculating based on the reported blocks and the reported size of each block (as per the output that came before the error). It was a 'hit and miss' arrangement as my calculations didn't seem to cover this missing space. When I finally (apparently) calculated enough space the output reported a segmentation fault and core dump! :eek:

My question is how to calculate the -I value to reflect the correct size. I added some small files into the file structure which has increased the size somewhat but I don't think the number you provided will work for my ROM.
[Android Software/Hacking General [Developers Only]] topet2k12001:
Hi @shlape,

I just finished unpacking and repacking an official firmware/ROM for the OnePlus 3 with success. Maybe this will help:

If you're using Ubuntu, right-click on the system.img and you'll see its size in bytes, like so:

So in my case, this is the value that I entered. I have read in an older post somewhere in this thread the following:

1. The value of 1073741824 is about 1GB, expressed in "bytes".

2. If getting an error, try to set the "-l" value to the same size of system.img.

Not sure if that will work for you, but you might want to give it a try.
[Android Software/Hacking General [Developers Only]] fays91:
Img2sdat 1.2

hello [MENTION=5132229]xpirt[/MENTION] I do not understand why this error

Xiaomi RedmiPro

*******************************

*Creating the new image

*******************************

CMD:

paramake=`wc -c $workdir/systemimg/system.img| cut --delimiter=' ' -f 1`

make_ext4fs -T 0 -S $tools/file_contexts -l "$paramake" -a system $workdir/systemimg/system_new.img $workdir/final/tmp/rom/system/

cd $workdir/systemimg

Creating filesystem with parameters:

****Size: 2684190720

****Block size: 4096

****Blocks per group: 32768

****Inodes per group: 8192

****Inode size: 256

****Newspaper blocks: 10239

****Label:

****Blocks: 655320

****Block groups: 20

****Reserved block group size: 159

Created filesystem with 5060/163840 inodes and 614643/655320 blocks

*

*************************************

*Creating .dat and .list files

*************************************

CMD:

$bin/img2sdat.py system_new.img

Img2sdat binary - version: 1.2

************1. Android Lollipop 5.0

************2. Android Lollipop 5.1

************3. Android Marshmallow 6.0

************Android 4. Nougat 7.0

************

Choose system version: 3

Traceback (most recent call last):

**File "/home/Miui/0_HMPro_BETA/Bin-HMPro/img2sdat/img2sdat.py", line 82, in <module>

****Main (sys.argv)

**File "/home/Miui/0_HMPro_BETA/Bin-HMPro/img2sdat/img2sdat.py", line 72, in main

****Image = sparse_img.SparseImage (INPUT_IMAGE, tempfile.mkstemp () [1], '0')

**File "/home/Miui/0_HMPro_BETA/Bin-HMPro/img2sdat/sparse_img.py", line 51, in __init__

****Raise ValueError ("Magic should be 0xED26FF3A goal is 0x% 08X"% (magic,))

ValueError: Magic should be 0xED26FF3A but is 0x00000000

*

**************************************
[Android Software/Hacking General [Developers Only]] shlape:
I did manage to repack the ROM but unfortunately it didn't install. I'm not sure if there is an associated MD5 check that didn't work due to the mods.

I changed the install script to bypass this check but it still didn't install. Also for some reason any ROM other than Oppo Project Spectrum don't work, ie no WiFi or Bluetooth.
[Android Software/Hacking General [Developers Only]] topet2k12001:
Hi [MENTION=5499632]shlape[/MENTION],

When you said "...it didn't install", any additional information that we can go by?

I've been using these binaries from way back in my LG G3 days and I've had no problems. I was able to port some ROMs over (this is because my LG G3 was a dual-SIM variant and most custom ROMs are for single-SIM variants) successfully with these unpacking/repacking binaries/tools.

Or it could have been whatever modification you did that caused the problem (i.e. the issue could already outside of the scope of this thread), perhaps?

Also when you modify, notice that a ".Trash" folder gets created. Please make sure that you delete that folder right before you convert it back to a sparse system.img as it also causes the "out of space" error (this was explained somewhere in this thread, in the past).
[Android Software/Hacking General [Developers Only]] hiepgabacninh:
help me! (nougat rom)

I have unpacked file system.new.dat successfully but when I repack the error
[Android Software/Hacking General [Developers Only]] shlape:
I used the colorOS recovery to install, which, in hind sight was probably the issue... it's kind of flaky.

The mods I tried involved placing superSU files within the system directory. I was able to set the appropriate properties similar to exiting files of the folder they were placed however the install simply failed without error code or message. Again this was colorOS recovery so I'll try using the latest TWRP for my device as well as this and your suggestions.

I have a few resources for compiling android and would be keen to get an Android N for my device... but for now... small steps.

Thanks to all who helped.
[Android Software/Hacking General [Developers Only]] topet2k12001:
Hi [MENTION=5499632]shlape[/MENTION],

Yeeeaahhh...I don't have a device with ColorOS but I have never used "stock" recovery to flash a custom ROM. :)
[Android Software/Hacking General [Developers Only]] iykeDROID™:
[MENTION=5132229]xpirt[/MENTION]

please do you have any idea about this error :

i keep getting this error using this command :

./img2sdat.py system.img tmp 4

------------------------------------------------------------

convert-dat$ ./img2sdat.py system.img tmp 4

img2sdat binary - version: 1.2

Traceback (most recent call last):

File "./img2sdat.py", line 82, in <module>

main(sys.argv)

File "./img2sdat.py", line 72, in main

image = sparse_img.SparseImage(INPUT_IMAGE, tempfile.mkstemp()[1], '0')

File "/home/iyke/Desktop/xe/convert-dat/sparse_img.py", line 51, in __init__

raise ValueError("Magic should be 0xED26FF3A but is 0x%08X" % (magic,))

ValueError: Magic should be 0xED26FF3A but is 0x00000000

------------------------------------------------------------

i don't know how to go get it done.

all ideas are invited... thank you.
[Android Software/Hacking General [Developers Only]] xpirt:
did you follow step 3?
[Android Software/Hacking General [Developers Only]] iykeDROID™:
yes [MENTION=5132229]xpirt[/MENTION]

--------------

okay looks like it works now :

$ tools/./img2sdat.py convert-dat/system_new.img convert-dat/tmpsparse 4

img2sdat binary - version: 1.2

Total of 657408 4096-byte output blocks in 2078 input chunks.

Generating digraph...

Finding vertex sequence...

Reversing backward edges...

0/0 dependencies (0.00%) were violated; 0 source blocks stashed.

Improving vertex order...

Reticulating splines...

max stashed blocks: 0 (0 bytes), limit: <unknown>

Done! Output files: convert-dat/tmpsparse

---------- Post added at 05:55 PM ---------- Previous post was at 05:48 PM ----------

here are some changes that i did before it worked:

Original

tools/./make_ext4fs -T 0 -S convert-dat/file_contexts -l 1073741824 -a system -a system convert-dat/system_new.img convert-dat/system/

Modified

tools/./make_ext4fs -s -T -1 -S convert-dat/file_contexts -L system -l 2692743168 -a system convert-dat/system_new.img convert-dat/system/
[Android Software/Hacking General [Developers Only]] iykeDROID™:
Hi all, i have made a script for helping to unpack/repack :

here is link : https://xdaforums.com/android/software-hacking/dev-android-extractrion-repack-tool-t3588311
[Android Software/Hacking General [Developers Only]] abhis3k:
Its because nougat has file_contexts.bin which is a binary file and not file_contexts text file.

You need to convert the binary file to text file first.

Check Here

It works beautifully if you are on Windows

or Use This if you are on Linux
[Android Software/Hacking General [Developers Only]] kornelkahh:
Can someone help me? I can't unpack system.new.dat. :/ Tbh I never used Python, so pls somebody help me. I'm trying to use that sdat2img.py but I can't lol.
[Android Software/Hacking General [Developers Only]] gendouhydeist:
[MENTION=5132229]xpirt[/MENTION]

Hi, where can I can get binaries source code to compile for nougat? A.S._id already discontinued his work up to marshmallow only. What I need is img2simg and make_ext4fs for nougat.

Also do I have to convert file_contexts.bin to txt file? As I already have system.transfer.list is it mandatory to convert it?
[Android Software/Hacking General [Developers Only]] Focusos:
hi，you can see here .

https://xdaforums.com/apps/superr-kitchen
[Android Software/Hacking General [Developers Only]] nclwz:
Thank you for your contribution:good:
[Android Software/Hacking General [Developers Only]] nclwz:
resolved
[Android Software/Hacking General [Developers Only]] evangelico793:
How to extract the file system.new.dat
[Android Software/Hacking General [Developers Only]] nclwz:
What's the difference with “-T 0” and “-T -1”

-T 0 what means？

-T -1 what means？

thanks
[Android Software/Hacking General [Developers Only]] threehappypenguins:
For some reason, when I repack the files into the system_new.img, and I mounted this new system file to take a peek in it, and I noticed that the lost+found folder goes missing. Why is this? It appears to be empty, but is it needed? Why does it disappear?
[Android Software/Hacking General [Developers Only]] xpirt:
added a note to OP about

OTA's patching

system and how to reproduce that on your pc, using imgpatchtools by [MENTION=6298645]erfanoabdi[/MENTION], check it out.
[Android Software/Hacking General [Developers Only]] yattodettaman:
what about recompile dat files with new oreo file_context ?
[Android Software/Hacking General [Developers Only]] mark332:
error

syntax error please fix it
[Android Software/Hacking General [Developers Only]] CMKdaGreat:
How to extract images of navbar from lineage is using this? I'm a beginner so pls help
[Android Software/Hacking General [Developers Only]] ale8530:
[MENTION=5132229]xpirt[/MENTION] you have create system_new.img Oreo with this command?

./make_ext4fs -T 0 -S file_contexts -l 1073741824 -a system system_new.img output/

until nagout I did not have any problems.

I also used make_4fs updated at Oreo of Superr, but the problem remains
[Android Software/Hacking General [Developers Only]] ale8530:
I have resolve xpirt is necessary convert file_contexts.bin to file_contexts for Oreo
[Android Software/Hacking General [Developers Only]] greeshan:
maybe somebody have make_ext4fs for android 7 for windows?

When I repack "system" folder to system.img (or after to system.new.dat) and flash it i have high consumption of system of 60% and more, the battery very quickly discharges ... As if the rights to something are incorrectly exposed and the discharge is due to it. If i flash ROM.zip with "system" folder inside - consumption of system of 15% and small... and battery life is good.... I was told that I need to compile through a new make_ext4fs for a new android...
[Android Software/Hacking General [Developers Only]] ze7zez:
system.new.dat.br

Hi [MENTION=5132229]xpirt[/MENTION] !

Will there be a new version of the sdat2img.py script to process system.new.dat.br to system.img for the ROM of Android 8.1?
[Android Software/Hacking General [Developers Only]] ze7zez:
Brotli Compression

I managed it.

From the Brotli website, download the tool for your own OS.

If you are using Windows, you can add to your script:

brotli.exe -d system.new.dat.br

[FONT="Courier New"]Usage: brotli.exe [OPTION]... [FILE]...
Options:
-# compression level (0-9)
-c, --stdout write on standard output
-d, --decompress decompress
-f, --force force output file overwrite
-h, --help display this help and exit
-j, --rm remove source file(s)
-k, --keep keep source file(s) (default)
-n, --no-copy-stat do not copy source file(s) attributes
-o FILE, --output=FILE output file (only if 1 input file)
-q NUM, --quality=NUM compression level (0-11)
-t, --test test compressed file integrity
-v, --verbose verbose mode
-w NUM, --lgwin=NUM set LZ77 window size (0, 10-24) (default:22)
window size = 2**NUM - 16
0 lets compressor choose the optimal value
-S SUF, --suffix=SUF output file suffix (default:'.br')
-V, --version display version and exit
-Z, --best use best compression level (11) (default)
Simple options could be coalesced, i.e. '-9kf' is equivalent to '-9 -k -f'.
With no FILE, or when FILE is -, read standard input.
All arguments after '--' are treated as files.[/FONT]
[Android Software/Hacking General [Developers Only]] majinawaz:
How to extract system.new.dat.br file kindly explain thanks
[Android Software/Hacking General [Developers Only]] ze7zez:
For example, like this:

7z.exe e [ROM_8.1].zip system.transfer.list system.new.dat.br
brotli.exe -j -d -o system.new.dat system.new.dat.br
sdat2img.py system.transfer.list system.new.dat system.img
[Android Software/Hacking General [Developers Only]] majinawaz:
copy all three files in ext folder brotli.exe , sadat2.img, system.new.dat.br, system.transfer.list

then type this command

brotli.exe -j -d -o system.new.dat system.new.dat.br

Output is this :

Microsoft Windows [Version 10.0.15063]

(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\maji\Downloads\Compressed\ext>brotli.exe -j -d -o system.new.dat system.new.dat.br

-= USAGE =-

To COMPRESS:

brotli.exe --in filename --out filename [--force] [--quality n] [-w n] [--verbose]

To DECOMPRESS:

brotli.exe --decompress --in filename --out filename [--verbose]

-= INFO =-

Copyright (c) 2009, 2010, 2013-2016 by the Brotli Authors

Provided "AS IS", without warranty of any kind

License: https://opensource.org/licenses/MIT

v0.5.2.1 Compiled November 16th, 2016 from

https://github.com/google/brotli/commit/5db62dcc9d386579609540cdf8869e95ad334bbd

C:\Users\maji\Downloads\Compressed\ext>