yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] xpirt:
fixed.

- sdat2img.py: fixed UnboundLocalError on android 5.x (commit 50eb51e)
[Android Software/Hacking General [Developers Only]] akhilnarang:
Thanks a lot; works fine now :)
[Android Software/Hacking General [Developers Only]] xpirt:
some news:

at this moment the decompression process with sdat2img.py binary works as it should with all android versions until latest 6.0 on a bunch of architectures, including windows x86/x64, linux x86/x64 & arm/arm64 with python3 installed.

unfortunately that is not the same for the compression process with rimg2sdat binaries which has not been updated since 5.0 version and

might

not work on newer versions, this because those binaries were compiled by howellzu and he didn't provide the source code for it.

as a matter of fact I will build a new binary based on latest google sources to do the whole compression process back and will release it probably this week together with proper source.
[Android Software/Hacking General [Developers Only]] bdorr1105:
Unable to get this to work, the exexution on Windows does not work

Edit: Deleted line to from the python code and it worked, the utf8 is holding it up even tho it is commented out. I am using python 3.5 for windows. Heads up for ya
[Android Software/Hacking General [Developers Only]] bdorr1105:
Hey brosephelades, this is not working with some of the marshamallow builds, any possibility of updating it. I used the python script for Windows, but this method was quicker because it required a lot less steps. Thanks
[Android Software/Hacking General [Developers Only]] ikslawlok:
Could you explain how to do that, please ?

Envoyé de mon Nexus 6 en utilisant Tapatalk
[Android Software/Hacking General [Developers Only]] HypoTurtle:
Which part?

Sparse to ext4 use simg2img

Ext4 to sparse use img2simg

For the command on #2 that needs a file_contexts file open the boot.img eith unmkbootimg and get it from there.
[Android Software/Hacking General [Developers Only]] ikslawlok:
Thank you. That was this part. ;)

Envoyé de mon Nexus 6 en utilisant Tapatalk
[Android Software/Hacking General [Developers Only]] suarez1907:
How can i add this source code for me any custom rom which i downloaded? https://github.com/gkraynov/nexus-4-navfix/releases

Nexus 4 cihazımdan Tapatalk kullanılarak gönderildi
[Android Software/Hacking General [Developers Only]] Alexey71:
Hey. android 6.0 - system.img?

I update script, try http://xdaforums.com/android/softwa...t-files-to-t2978952/post57742855#post57742855
[Android Software/Hacking General [Developers Only]] bdorr1105:
Thanks you man, awesome, I love that extractor. Great work....Also the python script for windows worked great, had to remove line 2 so it would run, but it was flawless, especially creating the system.new.dat and system.transfer.list with so I could use extractor version 1.5

Version 1.5 still works with Marshmallow if you rimgs2dat it first. Not every dat file was working, but hey guys, thank you for these tools. They are amazing! You guys rock
[Android Software/Hacking General [Developers Only]] xpirt:
it seems that that program is not working with all .dat files, for me it's not working either, I personally prefer the method I explained.
[Android Software/Hacking General [Developers Only]] bajasur:
Edit: Fixed
[Android Software/Hacking General [Developers Only]] Haidary.T:
Ext2Explore works perfectly but when I extract build.prop, I can't open it. Well.. I can but it's encrypted. At least I think it is since there's only a bunch of letters and numbers and something saying SHA at the end. Any ideas? Here is a pastebin of the file http://pastebin.com/9B1uwctk or the actual file.
[Android Software/Hacking General [Developers Only]] Elluel:
I already got rid of the block-based zip packaging in my own ROM but I downloaded a ROM that uses it, booted into Win8.1 x64 and it extracts perfectly fine for me. What version of ext2explore are you using?

Also, link me to the ROM you're trying to extract and I'll give it a shot.
[Android Software/Hacking General [Developers Only]] Alexey71:
ext2explore and win7-8.1 not work(

Error Opening \\.\PhysicalDrive2. Error Code 2
No of disks 2
Scanning \\.\PhysicalDrive0
index 0 ID 7 size 716800
index 1 ID 7 size 124323840
Scanning \\.\PhysicalDrive1
index 1 ID F size 1953519616
index 4 ID 7 size 1953517568
Partition Table Error on D:/system.new.dat
Invalid End of sector markerBlock size 4096, inp 8144, inodesize 256
Linux Partition found on disk 3 partition 0
Error Reading Inode FFFFFFFF. Invalid Inode Number
Error reading Inode -1 parent inode 2.
Error Reading Inode FFFFFFFF. Invalid Inode Number
Error reading Inode -1 parent inode 2.
[Android Software/Hacking General [Developers Only]] xpirt:
same for me and others, it is more simple to run ubuntu on a virtual machine and you've done.
[Android Software/Hacking General [Developers Only]] Haidary.T:
Myfluxi's CM12 https://www.androidfilehost.com/?fid=95864024717068070 Sorry for delayed reply I went to sleep lol.

---------- Post added at 02:34 PM ---------- Previous post was at 02:32 PM ----------

@Lethargy Something went wrong with quoting :confused:
[Android Software/Hacking General [Developers Only]] Haidary.T:
[MENTION=5666997]Lethargy[/MENTION]

Something definetely wrong with mentionning you o.o
[Android Software/Hacking General [Developers Only]] bunchies:
Nice
[Android Software/Hacking General [Developers Only]] Jason Hyunwoo:
Hi, Do they have a source code for that binary? I am using my MacOS to develop android. I am trying to finding source code
[Android Software/Hacking General [Developers Only]] Elluel:
By the way, for those who are building from source, this really simple line deletion will revert back to the old method. Also make sure you have this commit or it could flash awfully slow in recovery.

---------- Post added at 01:25 PM ---------- Previous post was at 01:21 PM ----------

Okay, I downloaded it and I got the same.

I could look at Github and try to figure out why, but I'm not bothered to and it won't really help since the people who want to do this aren't building from source anyways.
[Android Software/Hacking General [Developers Only]] parrotgeek1:
I found a Python Script!

I was looking through the google translated posts on howellzhu's CSDN post and found this link:

http://blog.csdn.net/luxee/article/details/42060875

Enjoy.
[Android Software/Hacking General [Developers Only]] xpirt:
good! let's see how this can be improved and ported to windows ;)
[Android Software/Hacking General [Developers Only]] Nicklas Van Dam:
is this only work for linux operating system on pc? can this work on windows?:confused:
[Android Software/Hacking General [Developers Only]] Elluel:
Read the thread.
[Android Software/Hacking General [Developers Only]] xpirt:
guys, thanks to luxi78's initial source code, I've managed to edit it and compile a Windows version so anyone can use it :)

DOWNLOAD:

Linux x64: http://d-h.st/LI4

Windows x86/x64: http://d-h.st/bEk

SOURCE:

https://github.com/xpirt/sdat2img/blob/master/sdat2img.py

CHANGELOG:

https://github.com/xpirt/sdat2img/commits/master

enjoy! ;)
[Android Software/Hacking General [Developers Only]] all43:
Thanks xpirt, now how do I run the windows version?
[Android Software/Hacking General [Developers Only]] xpirt:
i've modded it so both windows and linux instructions are the same ;)

sdat2img <transfer_list> <system_new_file> <system_img>

difference is obvious that on windows you have to type this on cmd and not on terminal.

more information in OP.
[Android Software/Hacking General [Developers Only]] all43:
Im an idiot, broke cardinal rule of not reading OP:D thanks again.
[Android Software/Hacking General [Developers Only]] webdroidmt:
Works great, awesome job! I initially got an error that looked to be a pathing error. So, as long as the sdat2img.exe is in the same folder as the dat files to be converted, it's all good.

I setup a specific folder on one of my drives just for converting and put the sdat2img.exe in it. I also put ext2explore in that same folder. Now when I want to convert, I put the dat file in that folder, let sdat2img.exe do its thing and then have ext2explore convert the resulting image to a familiar system folder.

Btw, the conversion is very fast. Thanks again for making this doable on windows, it's appreciated.

Mike T
[Android Software/Hacking General [Developers Only]] twcobra:
I hope the AndImgTool.exe was not important. My antivirus detected it as a Trojan and Quarantined it.
[Android Software/Hacking General [Developers Only]] bdorr1105:
Hey, there was in issue with the script, you used option 3 multiple times. I modified it and changed exit to 5 and its function so it actually exits, it will want to extract recovery.img, but the exit function did not work. just FYSA
[Android Software/Hacking General [Developers Only]] Alexey71:
Clear

AndImgTool.exe - https://www.virustotal.com/en/file/...8d5760a7e2f7a0cb347fe52b/analysis/1444839981/

Imgextractor.exe - https://www.virustotal.com/en/file/...b2ee621de3349234e66be3f7/analysis/1444840487/

---------- Post added at 19:40 ---------- Previous post was at 19:37 ----------

Oh sorry. Updated 1.7...

Work unpacking android 6?

I tested on the original firmware here https://developers.google.com/android/nexus/images#shamu
[Android Software/Hacking General [Developers Only]] flasherman:
[/COLOR]

Oh sorry. Updated 1.7...

Work unpacking android 6?

I tested on the original firmware here https://developers.google.com/android/nexus/images#shamu[/QUOTE]

not for me :(

Extractor for images in EXT2\EXT3\EXT4\YAFFS2\CRAMFS filesystem formats

support SPARSE\SIN\MOTO structure formats

File extract_file_here\system.new.img not found...

Не удается найти C:\ExtractNewRom\extract_file_here\system.new.img

Done. Go to the folder "extract_file_here"
[Android Software/Hacking General [Developers Only]] twcobra:
And of course my work uses Symantic. Hopefully an update to Symantic will stop it from thinking its a virus.
[Android Software/Hacking General [Developers Only]] bdorr1105:
not for me :(

I had the same problem, use the python script to create a system.img from the system.transfer.list and system.new.dat and then place the system.img in the extractor and profit. You can find the python script in post #3 for Windows I believe it is, it is on github, delete line 2 out of the script because it stops it from running
[Android Software/Hacking General [Developers Only]] Alexey71:
Can you give a link to your firmware?:)
[Android Software/Hacking General [Developers Only]] bdorr1105:
Here is the ROM I was unable to use the extractor on

http://xdaforums.com/showthread.php?t=3157756
[Android Software/Hacking General [Developers Only]] androcraze:
That image converted successfully from sdat to img using the python script under linux:

./sdat2img.py system.transfer.list system.new.dat raw.img

Skipping zero command.

Copying 32767 blocks into position 0...

Copying 2 blocks into position 32768...

Copying 2 blocks into position 32897...

Copying 32128 blocks into position 33407...

Copying 2 blocks into position 65536...

Copying 2 blocks into position 98304...

Copying 2 blocks into position 98433...

Copying 32128 blocks into position 98943...

Copying 2 blocks into position 131072...

Copying 2 blocks into position 163840...

Copying 2 blocks into position 163969...

Copying 32084 blocks into position 164479...

Copying 2 blocks into position 196608...

Copying 2 blocks into position 229376...

Copying 2 blocks into position 229505...

Copying 31253 blocks into position 230015...

Copying 2 blocks into position 262144...

Copying 2 blocks into position 294912...

Copying 2 blocks into position 295041...

Copying 2 blocks into position 327680...

Copying 2 blocks into position 360448...

Copying 2 blocks into position 393216...

Copying 2 blocks into position 425984...

Copying 2 blocks into position 458752...

Copying 2 blocks into position 491520...

Copying 28120 blocks into position 492030...

Copying 4105 blocks into position 520151...

Done! Output image: /stuff4/process/Cell Phone Stuff/Nexus 5/Nexus 5 - Install Folder/DAT file scripts/tmp2/raw.img

Is that what you mean?
[Android Software/Hacking General [Developers Only]] bdorr1105:
No, I am able to do this with python, the issue is being able to extract system.img with Alexey extractor tool...its much quicker process because the output is complete in 1 step
[Android Software/Hacking General [Developers Only]] androcraze:
It's a bbbbiiiigggg image file.... 2,147,352,576
[Android Software/Hacking General [Developers Only]] Alexey71:
Thank you. Update script v2.0. Work fine)

Type option:1

Please wait, start conversion in "system.new.img"...

Skipping zero command.
Copying 32767 blocks into position 0...
Copying 2 blocks into position 32768...
Copying 2 blocks into position 32897...
Copying 32128 blocks into position 33407...
Copying 2 blocks into position 65536...
Copying 2 blocks into position 98304...
Copying 2 blocks into position 98433...
Copying 32128 blocks into position 98943...
Copying 2 blocks into position 131072...
Copying 2 blocks into position 163840...
Copying 2 blocks into position 163969...
Copying 32084 blocks into position 164479...
Copying 2 blocks into position 196608...
Copying 2 blocks into position 229376...
Copying 2 blocks into position 229505...
Copying 31253 blocks into position 230015...
Copying 2 blocks into position 262144...
Copying 2 blocks into position 294912...
Copying 2 blocks into position 295041...
Copying 2 blocks into position 327680...
Copying 2 blocks into position 360448...
Copying 2 blocks into position 393216...
Copying 2 blocks into position 425984...
Copying 2 blocks into position 458752...
Copying 2 blocks into position 491520...
Copying 28120 blocks into position 492030...
Copying 4105 blocks into position 520151...

Done! Output image: C:\Users\Alexey\Desktop\Extractor\extract_file_here\system.new.img

Please wait, unpacks "system.new.img"...

ImgExtractor version 1.3.7 <Created by And_PDA (Based on sources ext4_unpacker)>

Extractor for images in EXT2\EXT3\EXT4\YAFFS2\CRAMFS filesystem formats
support SPARSE\SIN\MOTO structure formats

Open image file extract_file_here\system.new.img (size 2147352576 bytes) successfull...
[Android Software/Hacking General [Developers Only]] flasherman:
yes, work now, thx)
[Android Software/Hacking General [Developers Only]] bdorr1105:
Thanks for all the efforts, it is much appreciated
[Android Software/Hacking General [Developers Only]] Ricky Divjakovski:
made a windows command line tool to do basically the same as your script but a little more user friendly, credits are given and shown when sprs2img -help is executed

find tool here - http://xdaforums.com/android/software-hacking/tool-converting-lollipop-marsmallow-dat-t3225945
[Android Software/Hacking General [Developers Only]] xpirt:
did you built it using python?
[Android Software/Hacking General [Developers Only]] Ricky Divjakovski:
yes like i said its mainly your code but with a few bits an pieces added like the output file is inserted automatically, usage, tells you if you enter the wrong syntax etc
[Android Software/Hacking General [Developers Only]] webdroidmt:
Thanks for this update, works great for me on N7 2012 marshmallow rom.

Mike T
[Android Software/Hacking General [Developers Only]] nitesh9:
Thanks, your work is awesome.:)
[Android Software/Hacking General [Developers Only]] PaulPizz:
Thank you!!!!!!!
[Android Software/Hacking General [Developers Only]] SkandaH:
[MENTION=5132229]xpirt[/MENTION]

Thanks for the update :good:

A small request, can it be possible to emulate the block_image_update() function fully so that we can get a new system.img from an older one by applying a diff? Hope my explanation makes sense.
[Android Software/Hacking General [Developers Only]] Uchiha_Dev:
those who are getting error in CMD which says error in line 2, utf8 or something like that, open sda2img.py with Notepad++ only and delete the second line which says "# utf8" and save and run the command again it will work
[Android Software/Hacking General [Developers Only]] ahero123a:
I have tried but terminal say: no conmand found? And i also try on cygwin, bask say: cannot extract binary..

How i have to do?

Please help me!
[Android Software/Hacking General [Developers Only]] keekkenen:
hi, all can you any comment about my situation?..

I tested operation - dat to img, fs to img, img to dat..

1. I made system.img (sdat2img.py) - ok

2. mount system.img in dir - ok (nothing change in mounting dir)

3. made system_new.img from dir (make_ext4fs) - ok

4. made dat from system_new.img (img2sdat.py) - error

Traceback (most recent call last):
File "img2sdat.py", line 40, in <module>
main(sysimg, outdir)
File "img2sdat.py", line 12, in main
tgt = sparse_img.SparseImage(sysimg)
File "/storage/emulated/legacy/dev/sparse_img.py", line 43, in __init__
raise ValueError("Magic should be 0xED26FF3A but is 0x%08X" % (magic,))
ValueError: Magic should be 0xED26FF3A but is 0x00000000

all action I made on android (smart)

do you have any ideas ?
[Android Software/Hacking General [Developers Only]] xpirt:
it is bugged, that's why. use rimg2dat instead, until I'll update it.
[Android Software/Hacking General [Developers Only]] keekkenen:
rimg2sdat (in windows) also output error

Magic number got 0x3a79 (should be 0x53ef), make sure input is an ext4fs image.

I think trouble with make_ext4fs, after creation img, it not mounting..

make_ext4fs -s -l 1536M -a system img/system_new.img out/

what's trouble with it (correct work - no errors) ?
[Android Software/Hacking General [Developers Only]] xpirt:
why don't you use the commands from OP?
[Android Software/Hacking General [Developers Only]] keekkenen:
now I wanted check make_ext4fs but can not open by link (post ubuntu guide)..

I simple convert dat to img - it's ok, and after convert new img to dat and have error like before post (ValueError: Magic should be 0xED26FF3A but is 0x00000000)

and what's problem ?
[Android Software/Hacking General [Developers Only]] mikhaelkh:
I am on Win7 SP1 x64, I installed python 3.5.0 x64 and when I run sdat2img.py I get:

SyntaxError: encoding problem: utf8

How can I fix it?

Edit: I replaced second line to # -*- coding: utf-8 -*- and no more error!
[Android Software/Hacking General [Developers Only]] xpirt:
fixed.
[Android Software/Hacking General [Developers Only]] Q9Nap:
I think the rimg2sdat binary needs to be updated for marshmallow as well.

The current binary works fine for Nexus 6 builds up to 5.1.1 using the stock system image from official fastboot archive, but doesn't work on 6.0 Nexus 6 builds.

The system.transfer.list and system.new.dat files result in booting to a blank screen.

I can, however, make it work by running rim2sdat on a dd system image directly, (without mounting and remaking an ext4 image).

I'd be happy to test in order to help get this working on stock m images...
[Android Software/Hacking General [Developers Only]] PaulPizz:
i am trying to use the make_ext4fs to recompile a marshmallow rom for the nexus and i keep getting this error.

larry@PaulPizz:~/new.dat conversion$ ./make_ext4fs -T 0 -S file_contexts -l 1073741824 -a system system_new.img output/

Creating filesystem with parameters:

Size: 1073741824

Block size: 4096

Blocks per group: 32768

Inodes per group: 8192

Inode size: 256

Journal blocks: 4096

Label:

Blocks: 262144

Block groups: 8

Reserved block group size: 63

error: ext4_allocate_best_fit_partial: failed to allocate 3555 blocks, out of space?

can anyone help?? forgive me if this has been addressed, if so please direct me to the proper post.. thank you!
[Android Software/Hacking General [Developers Only]] Q9Nap:
How did you arrive at that -l flag value?

If you started with a sparse system image and used simg2img to create a raw image, run "wc -c system.raw" on the raw image and use that value for -l.
[Android Software/Hacking General [Developers Only]] PaulPizz:
Worked! Thank you for the knowledge & time! [emoji106]
[Android Software/Hacking General [Developers Only]] Shujath:
Hi, I have downloaded the Python 3.1 and cygwin on my windows pc.

now where Do I need to place the system.transfer.list and system.new.dat file on my pc and run the command?
[Android Software/Hacking General [Developers Only]] androcraze:
http://xdaforums.com/showpost.php?p=60180977&postcount=227

All in the same folder the you're currently in. That makes it easy.
[Android Software/Hacking General [Developers Only]] Shujath:
Thanks, but what if I want to use .py which is metioned in op instead of .exe?
[Android Software/Hacking General [Developers Only]] androcraze:
Is Python system-wide? If you can run it from anywhere on the PC, it doesn't make a difference.

Try creating a folder, put the .py script in that folder, "cd" to that folder via CLI, and try running the script. If it works, you don't need to sweat Python location.
[Android Software/Hacking General [Developers Only]] Shujath:
getting this error if I place the sdat2img.py in a folder ""py"" in desktop

C:\Users\Shujath\Desktop\py>sdat2img.py

File "C:\Users\Shujath\Desktop\py\sdat2img.py", line 4

<!DOCTYPE html>
[Android Software/Hacking General [Developers Only]] androcraze:
Under Linux, I need to set the permissions of the .py script to 755 before I can run it. I'm worried that Phyton isn't system-wide. Did you have to add Python to the system path?

http://www.howtogeek.com/197947/how-to-install-python-on-windows/
[Android Software/Hacking General [Developers Only]] Shujath:
Ive installed Ubuntu now, can you give me some hints on setting file perssions?
[Android Software/Hacking General [Developers Only]] androcraze:
chmod 0755 sdat2img.py
[Android Software/Hacking General [Developers Only]] androcraze:
Under Linux, with everything in the same folder/directory:

./sdat2img system.transfer.list system.new.dat system.fixed.img

I usually open another terminal (or tab - in gnome-terminal), and go to root, by typing "su" then the root password. You can also do the sudo thing if you are able (or willing):

cd <path to system.fixed.img>

mount -t auto -o loop system.fixed.img /mnt

You can then have some fun looking at your ROM under /mnt/
[Android Software/Hacking General [Developers Only]] Shujath:
Do i have to type the commands in python3.4 or i can directly type the command in terminal

i've tried both and it didnt work

also enabled root

note: I decompressed the dat to ext4 using sdddat2img.exe in windows and copied the file in linux, able to mount it perfectly. but again when i have to convert it to ext4 the commands are not working.

any suggestions?
[Android Software/Hacking General [Developers Only]] androcraze:
I'd recommend doing it from the command line. You can cut-and-paste rather than type. What error are you getting?
[Android Software/Hacking General [Developers Only]] Shujath:
Also tried without .py in the command and still same error

btw the ext4 file in ss is copied from windows(using sdat2img.exe)
[Android Software/Hacking General [Developers Only]] androcraze:
OK. For folders, it's "Downloads", not "/Downloads"

A preceding "/" in front of the folder means to look all the way at the System root directory, like /usr, /boot, etc.

I'm also a little troubled that you're doing everything as root (su); you should only be performing system-specific items from there.
[Android Software/Hacking General [Developers Only]] androcraze:
I'm attaching a window shot.
[Android Software/Hacking General [Developers Only]] Shujath:
hmm wierd, i tried with both root and user still same error

do i need to copy sdat2img.py somewhere ?

it is showing sytax error.. unable to understand what wrong i am doing