yashsingh1/xdaforums huggingface dataset:
[Sony Cross-Device General] thrash001:
i think i must repair my phone with pc companion:

--------------------------------------------------------
newflasher.exe by Munjeni @ 2017
--------------------------------------------------------
Device path: \\?\usb#vid_0fce&pid_b00b#5&15c311e1&0&2#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: USB-Controller
Device Instance Id: USB\VID_0FCE&PID_B00B\5&15C311E1&0&2

Product: G8141
Version: 0.4-SONY-0.1
Bootloader version: 1306-5035_X_Boot_MSM8998_LA1.1_N_59-A
Baseband version: 1306-8499_45.0.A.1.229
Serialno: CB512EADT6
Secure: yes
Loader version: XFL-MSM8998-N-35
Phone ID: 0000:35783808059099
Device ID: A6D4C462
Platform ID: 2005E0E1
Max download size: 104857600
Sector size: 4096
Rooting status: ROOTABLE
Ufs info: TOSHIBA,THGAF4G9N4LBAIRB,0100
Emmc info: FAILEmmc-info not supported
Default security: ON
Keystore counter: 1
Security state: BA1894F07A55612C78DAADBA68D953CB4B3D2133
Sake root: D159
S1 root: S1_Root_e69c
Root key hash: C30DEC2471CEA311E6918657367B51068A39583BBF89FD68B379BCD5A709AB1B

Device is put now in flash mode.

Using existing folder flash_session

Processing cache_X-FLASH-ALL-42E5.sin
- setting up infflate...
- infflating, please wait...

- infflate returned: 0
- gzpipe: ok.
- gunziped ok.
- Extracting from cache_X-FLASH-ALL-42E5.sin
- Extracting signature flash_session/cache.cms
- Uploading signature flash_session/cache.cms
signature:00000533
OKAY.
erase:cache
OKAY.
- Extracting sparse chunk flash_session/cache.000
- Uploading chunk flash_session/cache.000
download:0080f0c4
Proccessing sub chunk 1: 0x200000 bytes
Proccessing sub chunk 2: 0x200000 bytes
Proccessing sub chunk 3: 0x200000 bytes
Proccessing sub chunk 4: 0x200000 bytes
Proccessing sub chunk 5: 0xf0c4 bytes
OKAY.
- End of cache_X-FLASH-ALL-42E5.sin
flash:cache
Waiting device to proccess file internaly, it will take minute-two!
SETTING TIMEOUT TO 600000 (ms).
got reaply after ms: 10.000000
Error, didn't got flash OKAY reply! Got reply: DATA0080f0c4

ERROR: TIMEOUT: failed with error code 997 as follows:
▄berlappender E/A-Vorgang wird verarbeitet.
- Error reply: less than 4!
Raw input[0x0]:

- Error, no go_outof_flash_mode DATA reply!
Sent command: Sync
ERROR: TIMEOUT: failed with error code 997 as follows:
▄berlappender E/A-Vorgang wird verarbeitet.
- Error reply: less than 4!
Raw input[0x0]:

Drücken Sie eine beliebige Taste . . .
[Sony Cross-Device General] munjeni:
Yes :( Device restarted download session because of one reason: I have set fixed sub chunks to 0x200000 bytes but seems thats wrong! As per sniff log of the pc companion sub chunks is not with fixed size :( , I thinked it will work if I set it up with fixed lenght of 0x200000 bytes but seems sub chunks is determined by android image sparse chunks size :) See log please!

This is sniff log of the pc companion and steps for cache flash:

Removed.

As we can see we have expected OKAY reaply after command flash but we got in your log seems restart of the download since device replied with DATAsizeofthechunk000, and also in sniff log we can clearly see that sub chunks is not with fixed size. :( I will investigate on this in next day-two, thanks a ton for your time working with me, I am hope when we finish this right all will benefit using our tool. Job is not easy, things is not so easy but we will figure out everything needed :good: I am so sorry, sometimes soft bricks is inevitably :( but you can anytime use pc companion to repair (ofcourse if you folow what I wrote in first post). I will play now with sparse format and try to produce cache ofsets excatly the same like in pc companion sniff log. We solved hash bug, we will solve this too!
[Sony Cross-Device General] thrash001:
yes man.
[Sony Cross-Device General] munjeni:
This will be hard for understand, why first sub chunk is 0x200000 but seccond 0x1ff600, thats non sense. :( Have analysed android sparse format and there things is diferent :( On system sin a lot more diferent size of the sub chunks, some times its below 0xb5567, non sense totaly
[Sony Cross-Device General] munjeni:
I can modify tool and set fixed sizes the same like on sniff log, it would verify what is wrong e.g. OKAY reaply or DATA reaply, just tell me and I will do it!
[Sony Cross-Device General] munjeni:
New version is out, now I have set sub chunks size excatly the same like this https://xdaforums.com/showpost.php?p=72647083&postcount=103 , if this version give DATA reaply than definitelly tool missing something, but if give OKAY than l will continue researching, curently have no idea if tool is wrong or we need to find a way for how sony determine sub chunk size :)
[Sony Cross-Device General] munjeni:
OMG! Found what is wrong! flash:cache string is not 0x11 bytes len, 0x11 in dec is 17 but we need 11, my stupid wrong :( I am not at pc now but tommorow new version! How stupid 0x11 I set omg :(

detail from log:

flash:cache
- Successfully write 0x11 bytes to handle.
[Sony Cross-Device General] munjeni:
Fixed bug, new version out! :fingers-crossed:
[Sony Cross-Device General] thrash001:
looks very good:

--------------------------------------------------------
newflasher.exe by Munjeni @ 2017
--------------------------------------------------------
Device path: \\?\usb#vid_0fce&pid_b00b#5&15c311e1&0&2#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: USB-Controller
Device Instance Id: USB\VID_0FCE&PID_B00B\5&15C311E1&0&2

Product: G8141
Version: 0.4-SONY-0.1
Bootloader version: 1306-5035_X_Boot_MSM8998_LA1.1_N_59-A
Baseband version: 1306-8499_45.0.A.1.229
Serialno: CB512EADT6
Secure: yes
Loader version: XFL-MSM8998-N-35
Phone ID: 0000:35783808059099
Device ID: A6D4C462
Platform ID: 2005E0E1
Max download size: 104857600
Sector size: 4096
Rooting status: ROOTABLE
Ufs info: TOSHIBA,THGAF4G9N4LBAIRB,0100
Emmc info: FAILEmmc-info not supported
Default security: ON
Keystore counter: 1
Security state: BA1894F07A55612C78DAADBA68D953CB4B3D2133
Sake root: D159
S1 root: S1_Root_e69c
Root key hash: C30DEC2471CEA311E6918657367B51068A39583BBF89FD68B379BCD5A709AB1B

Device is put now in flash mode.

Using existing folder flash_session

Processing cache_X-FLASH-ALL-42E5.sin
- setting up infflate...
- infflating, please wait...

- infflate returned: 0
- gzpipe: ok.
- gunziped ok.
- Extracting from cache_X-FLASH-ALL-42E5.sin
- Extracting signature flash_session/cache.cms
- Uploading signature flash_session/cache.cms
signature:00000533
OKAY.
erase:cache
OKAY.
- Extracting sparse chunk flash_session/cache.000
- Uploading chunk flash_session/cache.000
download:0080f0c4
Proccessing sub chunk 1: 0x200000 bytes
Proccessing sub chunk 2: 0x200000 bytes
Proccessing sub chunk 3: 0x200000 bytes
Proccessing sub chunk 4: 0x200000 bytes
Proccessing sub chunk 5: 0xf0c4 bytes
OKAY.
- End of cache_X-FLASH-ALL-42E5.sin
flash:cache
OKAY.

Device is put now out of flash mode.
Sent command: Sync
Sent command: powerdown

End.
Drücken Sie eine beliebige Taste . . .

---------- Post added at 07:27 AM ---------- Previous post was at 07:25 AM ----------

and the phone start normal! yeaha

---------- Post added at 06:39 AM ---------- Previous post was at 06:27 AM ----------

Which file I have flash so my phone is recognized as a free device at pc companion?
[Sony Cross-Device General] munjeni:
Success! Perfectly good! Sorry but didn't understand yout question
[Sony Cross-Device General] jms.flynn:
So now you guys are at this stage, are we close to having a working flash tool for flashing different firmware on our xz premium?
[Sony Cross-Device General] thrash001:
Which file has the information for the specific telecom, vodafone and free version (debranded). I would write this on my telecom telephone and then let it repair via pc companion. Then the complete free firmware would be pulled over pc companion. Formerly one had to change the build.prop it was recognized by pc companion as free device. But without root it is not. Or do you think we can soon the complete firmware flash?
[Sony Cross-Device General] munjeni:
Dont knoe that info but probably system and modem is need. Please wait day or two I will integrate the rest! And don't flash sins from boot folder or from partition zip, need to implement that! I think tool is ready now for flashing system, cache, data... :) Ta files will also add support after boot and partition! We are so close for complete tool :)
[Sony Cross-Device General] jms.flynn:
I think it is the oem_X-FLASH-ALL-42E5.sin file as this has the config.prop and customer.prop file in it.
[Sony Cross-Device General] thrash001:
I can not see the telecoms anymore and wanted to get rid of it quickly. Because I'm more relaxed when testing your tools and does not have to see the "T".
[Sony Cross-Device General] munjeni:
You can try to reflash all sins since I think tool is pretty stable for that task. Please if you going for that try one by one sin if something go wrong... :) Boot delivery, all ta files and partition.zip will not get flashed since need to implement it so do not try to flash it
[Sony Cross-Device General] thrash001:
so i flashed the oem...-sin

--------------------------------------------------------
newflasher.exe by Munjeni @ 2017
--------------------------------------------------------
Device path: \\?\usb#vid_0fce&pid_b00b#5&15c311e1&0&2#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: USB-Controller
Device Instance Id: USB\VID_0FCE&PID_B00B\5&15C311E1&0&2

Product: G8141
Version: 0.4-SONY-0.1
Bootloader version: 1306-5035_X_Boot_MSM8998_LA1.1_N_59-A
Baseband version: 1306-8499_45.0.A.1.229
Serialno: CB512EADT6
Secure: yes
Loader version: XFL-MSM8998-N-35
Phone ID: 0000:35783808059099
Device ID: A6D4C462
Platform ID: 2005E0E1
Max download size: 104857600
Sector size: 4096
Rooting status: ROOTABLE
Ufs info: TOSHIBA,THGAF4G9N4LBAIRB,0100
Emmc info: FAILEmmc-info not supported
Default security: ON
Keystore counter: 1
Security state: BA1894F07A55612C78DAADBA68D953CB4B3D2133
Sake root: D159
S1 root: S1_Root_e69c
Root key hash: C30DEC2471CEA311E6918657367B51068A39583BBF89FD68B379BCD5A709AB1B

Device is put now in flash mode.

Using existing folder flash_session

Processing oem_X-FLASH-ALL-42E5.sin
- setting up infflate...
- infflating, please wait...
..
- infflate returned: 0
- gzpipe: ok.
- gunziped ok.
- Extracting from oem_X-FLASH-ALL-42E5.sin
- Extracting signature flash_session/oem.cms
- Uploading signature flash_session/oem.cms
signature:00000531
OKAY.
erase:oem
OKAY.
- Extracting sparse chunk flash_session/oem.000
- Uploading chunk flash_session/oem.000
download:015a6634
Proccessing sub chunk 1: 0x200000 bytes
Proccessing sub chunk 2: 0x200000 bytes
Proccessing sub chunk 3: 0x200000 bytes
Proccessing sub chunk 4: 0x200000 bytes
Proccessing sub chunk 5: 0x200000 bytes
Proccessing sub chunk 6: 0x200000 bytes
Proccessing sub chunk 7: 0x200000 bytes
Proccessing sub chunk 8: 0x200000 bytes
Proccessing sub chunk 9: 0x200000 bytes
Proccessing sub chunk 10: 0x200000 bytes
Proccessing sub chunk 11: 0x1a6634 bytes
OKAY.
- End of oem_X-FLASH-ALL-42E5.sin
flash:oem
OKAY.

Device is put now out of flash mode.
Sent command: Sync
Sent command: powerdown

End.
Drücken Sie eine beliebige Taste . . .

---------- Post added at 04:03 PM ---------- Previous post was at 03:50 PM ----------

but is not the file for free firmware :(
[Sony Cross-Device General] tanipat:
xperia companion is not work for you?
[Sony Cross-Device General] MeltdownSpectre:
Not sure how useful this is to you, but NewFlasher v22 works on the 1st generation Xperia 1 (J9110).

Log: https://pastebin.com/TLX1wtxP
[Sony Cross-Device General] munjeni:
Perfectly! Many thanks! So it will work for Xperia 10, 10 II, 5, 5 II and the rest. And newflasher is now a bit safer because of battery level checkup, and all the isues from github resolved now, thanks for testing and reporting! Only one untested thing is test from Macos. I'm getting Xperia 10 in day-two so I will start something :)
[Sony Cross-Device General] MeltdownSpectre:
I just noticed the battery percentage in my log says 48%, whereas my phone was above 90% when I flashed it.
[Sony Cross-Device General] munjeni:
Many thanks! Based on this info https://github.com/munjeni/trimarea_tool/blob/master/include/miscta_units.h#L50 TA_BATTERY_CAPACITY is definitelly about battery capacity, but right now after you reported it I just found in one of the trim area dumps from various sony models that those ta unit on some is two bytes len, seems it need some more work :( Can you please try newflasher from attachment, don't flash anything just run newflasher while your device is in flash mode and give me please "raw battery unit" from log and in the same time real battery level on your phone, hope its not problem for you, anyway thanks!
[Sony Cross-Device General] MeltdownSpectre:
Just tested it. My phone was at 65% battery when I ran this build.

Output below:

--------------------------------------------------------
newflasher.exe v22 by Munjeni @ 2017/2019
--------------------------------------------------------

Determining available free space by GetDiskFreeSpaceEx:

Available space to caller = 396958 MB
Total space on current drive = 476316 MB
Free space on drive = 396958 MB

Optional step! Type 'y' and press ENTER if you need GordonGate flash driver, or type 'n' to skip.
This creates GordonGate.7z archive in the same dir with newflasher.exe!
n
Device path: \\?\usb#vid_0fce&pid_b00b#5&c74a7a5&0&3#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: Universal Serial Bus controllers
Device Instance Id: USB\VID_0FCE&PID_B00B\5&C74A7A5&0&3

Optional step! Type 'y' and press ENTER if you want dump trim area, or type 'n' and press ENTER to skip.
Do in mind this doesn't dump drm key since sake authentifiction is need for that!
n
raw battery unit[0x2]:

00000000 30 46 0F

Product: J9110
Version: 0.4
Bootloader version: 1315-2591_X_Boot_SM8150_LA2.0_Q_115
Baseband version: 1318-5872_55.1.A.9.21
Serialno: SANITIZED
Secure: no
Loader version: XFL-SM8150-P-16
Phone ID: SANITIZED
Device ID: SANITIZED
Platform ID: 300A50E1
Max download size: 104857600
Sector size: 4096
Rooting status: ROOTED
Ufs info: TOSHIBA,THGAF8T0T43BAIRB,0300
Emmc info: FAILEmmc-info not supported
Default security: ON
Keystore counter: 2
Security state: SANITIZED
Sake root: SANITIZED
S1 root:
Root key hash: SANITIZED
Slot count: 2
Current slot: a
Battery capacity: 48 percent

Device is put now in flash mode.

No .sin files in partition dir...
You must extract partition.zip into 'partition' folder if you want flash partition image!
On 2018 and UP models you must move partition sin files to 'partition' folder if you need flash partition images!

Processing .sin files...
Using existing folder flash_session
No .sin files in current dir.

Processing .ta files...
No .ta files in current dir.

Processing boot delivery...

boot_delivery.xml not exist in boot folder or no boot folder.

Set slot 'a' active.

Device is put now out of flash mode.
Sent command: powerdown

End. You can disconnect your device when you close newflasher.exe
Press any key to continue . . .
[Sony Cross-Device General] munjeni:
Many thanks! I'm unsure right now, last byte in hex 0x46 in decimal it's 70 percent, close to 65% , this is problematic now without more tests, expecialy since oldest devices have one byte in that unit, newer seems have two byte. I'm getting xperia 10 in day-two, for this I need more test but I will further test this and hope figure thing out, in most case feature going to be removed. Again thanks, it helped!
[Sony Cross-Device General] MeltdownSpectre:
I think 70% is correct. Looks like this build is working just fine.

When I saw your post about running the test, coincidentally my phone was already at 48% battery, so I didn't want to post the output since the bug shows 48% no matter what and I felt that the results would be useless.

I charged the device for 10-15 mins. It was powered off while charging and the bootloader battery charge screen showed 65%. I then performed the test and pasted the output here.

However after I restarted my phone, the battery indicator said 70% which matches up with the number you got.
[Sony Cross-Device General] munjeni:
If you have some more time e.g. tommorow when your battery get e.g. 90% it would help figuring out, anyway I will test further till I get phone. :good:
[Sony Cross-Device General] munjeni:
Got my xperia. Found that those battery bytes is fake, on my xperia its one byte and always 50 percent :( I will remove those feature i next version!
[Sony Cross-Device General] munjeni:
Version 23 is out, enjoy!
[Sony Cross-Device General] Xperia S black:
For me works perfect for my xz3. Thanks
[Sony Cross-Device General] munjeni:
Something is wrong with attachments on xda, I'm reported isue hope it get solved soon.
[Sony Cross-Device General] munjeni:
Version 25 is out!

- version v25 (09.07.2020)

New trim area dump tool, with this change trim area dump is created in 3 secconds. Do in mind this not dump protected units like drm key...etc! Some changes in scripting feature from v24

In general it is reccomended to make a dump in case you hard brick your device since without per device unique units (7D3/* hardware config */ 7DA/* simlock */851/* simlock signature */ ), in case you lost them or delete you can't recover you pnone!!!
[Sony Cross-Device General] munjeni:
Version 26 is out!

- version v26 (10.07.2020)

Added 4 diferent reboot modes, reboot to android, reboot to fastboot, reboot to bootloader, power off
[Sony Cross-Device General] munjeni:
I have some good news for Mac users, got initialy working version for Mac, I'm testing now on 10.14, seems everything is right except seems not working for everybody:

Device is put now out of flash mode.
Sent command: Sync
bulk transfer (in): LIBUSB_ERROR_TIMEOUT
- Error reply: null!
Error, no sync response!

More info here https://github.com/newflasher/newflasher/issues/27 , if anybody have idea why sync command didn't respong to sync let me know please!

I'm tested on my own phone, no sync bug, seems its not working on everybody phone.
[Sony Cross-Device General] munjeni:
Version 28 is out!

- version v27 (11.07.2020) (not yet released)

Workaround in mac libusb

- version v28 (12.07.2020)

Workaround to sync response bug; Fully implemented support for Mac. I'm tested myself on mac 10.14 but confirmed working on mac 10.15 too

I'm included mac 64 bit binary with staticaly linked usb, expat and z, you can test, enjoy!
[Sony Cross-Device General] munjeni:
Version 29 is out!

- version v29 (12.07.2020)

Mac proper libusb deinitialisation

I think all is ready for a long time. Enjoy! :)
[Sony Cross-Device General] John_forsaken:
Hi trying to get this done on Android as my laptop decided to die on me... what I have so far is 1.rooted android 8.1 device.. 2.second device xz2(one I’m going to flash) the 3.firmware folder with everything that is needed inside.. what do I do now?
[Sony Cross-Device General] munjeni:
Just copy newflasher android binary (chose suitable one for your phone) to the that firmware folder, open terminal emulator and invoke su to become root, than simple cd to the firmware directory, chmod 755 newflasher, connect with micro usb cable & reboot to flashmode your seccond phone (one you need to flash) and on terminal emulator just do ./newflasher and folow on screen
[Sony Cross-Device General] samteeee:
@OP

Great tool, much appreciated.

Tried to flash XQ-AT 52 but after the process my device says "Your device is corrupted" and powers off!

Another ting is I can't find the log!

Any ideas? Thank you.
[Sony Cross-Device General] munjeni:
Log is inside main screen, jst copy it. Asuming you wanted to keep userdata and skipped ta files, you need to flash everything except persyst, don't skip any single file except persyst.sin
[Sony Cross-Device General] samteeee:
I flashed everything included persyst!

Will try again?
[Sony Cross-Device General] John_forsaken:
Sorry I’m new to terminal...

1. What terminal emulator what be best to use?

2.Do I keep the firmware folder anywhere specific on the 1st device?

3. cd to the firmware directory, chmod 755 newflasher? How do I do that?
[Sony Cross-Device General] munjeni:
1. Sorry thats question for android general section, I can't teach you that basic android stufs, in short terminal emulator is similar to linux shell (for further info use google), on android you can install terminal emulator from play store https://play.google.com/store/apps/details?id=jackpal.androidterm&hl=en

2. put in the folder where you can see them trought terminal emulator, e.g. copy whole firmwae folder to sdcard than use e.g. https://play.google.com/store/apps/details?id=nextapp.fx&hl=en to move that folder to e.g. /data/local/tmp

3. trought terminal emulator window type e.g.

cd /data/local/tmp/FIRMWARE_FOLDER_NAME

chmod 755 newflasher

about what cd command is doing use google search, about what chmod command doing use google search too... hope this help.
[Sony Cross-Device General] munjeni:
Why you flashed persist? Did you even read newflasher prompt :

Recommended step to skip this! Type 'y' and press ENTER if you want flash persist partition, or type 'n' and press ENTER to skip.
More info https://xdaforums.com/xperia-xz1-compact/help/android-attest-key-lost-bootloader-t3829945

Since you flashed persist you have deleted your device atestation keys.
[Sony Cross-Device General] John_forsaken:
Yes thank you that helped so much thank you, much appreciated ☺️
[Sony Cross-Device General] samteeee:
I read it but misunderstood!!

What can I do??

Thank you.
[Sony Cross-Device General] munjeni:
I don't know, I never done that, but I have mentioned about it inside newflasher, more info on this link -> https://xdaforums.com/xperia-xz1-compact/help/android-attest-key-lost-bootloader-t3829945
[Sony Cross-Device General] munjeni:
Version 30 is out!

- version v30 (13.07.2020)

Preparation for Debian packaging; I'm noticed that hex modified arm64 fake pie binary is not working so its now compiled with ndk and its true pie binary now

After trying something on my xperia 10 I'm noticed that hex modified non pie binary newflasher.arm64 in order for making fake pie binay newflasher.arm64_pie with known hack (17th byte 0x02 to 0x03 change) giving segmentaion fault when running newflasher.arm64_pie (google fixed that hack?) so I have decided to make true pie binary, newflasher.arm64_pie is working now, enjoy!
[Sony Cross-Device General] samteeee:
Thanks.
[Sony Cross-Device General] Winrahr:
What did you mean by skipped TA files? Should I be including all the TA files when I flash or should some be skipped? Apart from simlock.ta which is skipped anyway.
[Sony Cross-Device General] munjeni:
Clean install mean: flash all ta files, flash all sin files (need to exclude only persist.sin), flash partition, flash boot delivery.

Dirty flash mean: skip some ta files and don't expect your device boot, skip flashing userdata.sin and don't except your android boot.... the same for the rest.

I don't know how many times I say that but again I can say that you need to flash everything, only one file to skip is persist.sin, simlock is skipped by default, that's the only right and clean way if you need everything working on your device. If your device not booting it mean that you didn't flashed them correctly.
[Sony Cross-Device General] munjeni:
One good news https://salsa.debian.org/android-tools-team/admin/-/issues/23 , newflasher on Debian distro!
[Sony Cross-Device General] MeltdownSpectre:
[MENTION=4120742]munjeni[/MENTION]

Is there a way to add the prompt for flashing persist at the beginning?

Normally it happens during the flash process, but I feel that more users will read it if they are prompted at the beginning when the flash is about to start.
[Sony Cross-Device General] munjeni:
Neflasher design is to flash everything found inside folder, its not possible, whole design will need to be modified. Those promt happen when newflasher main loop get to the persist.sin file than prompt happen. Its not possible to not see those message, users must see them.
[Sony Cross-Device General] John_forsaken:
Running from Android I get....

Available space to caller = 0 mb

Total space on current drive = 0 mb

Free Space on Drive = 0 mb

Error! You do not have needed 10240 mb available free space on your disk drive! You have only 0 mb free