yashsingh1/xdaforums huggingface dataset:
[Sony Cross-Device General] shoey63:
Looks good. Just put newsflasher.exe into the root of the folder, where all the *.sin files are (not boot or partition) and double click. The simlock.ta file is the only dangerous one, so delete it if you must. However [MENTION=4120742]munjeni[/MENTION] is a top dev, and his tool automatically blocks anything dangerous being flashed.
[Sony Cross-Device General] dcrin3:
I've just found out not 1hr ago that you were the author of this marvelous tool. I took a look at the code on github, my words are not enough to describe my thanks.

You and your crew have done this community a massive favor, thanks [MENTION=4120742]munjeni[/MENTION] for wasting your time for us mere mortals. Wow, just wow!. :good::good::good:
[Sony Cross-Device General] ehsas1:
Do not copy boot folder files in root folder?

Finally, delete the simlock.ta or not?

There is no risk with newflasher? Are I sure???? :o
[Sony Cross-Device General] munjeni:
XA2, XA2 ultra and all 2018 models... see this https://xdaforums.com/showpost.php?p=76097964&postcount=135
[Sony Cross-Device General] Snow_Basinger:
Hi [MENTION=4120742]munjeni[/MENTION]

I tried logging with XZ2 Compact

Sorry if I made a mistake because it's so short

000000: PnP Event: Device Connected (UP), 2018-04-05 00:43:25.6439238 (1. Device: Xperia XZ2 Compact)
The USB device has just been connected to the system.

000001: Bulk or Interrupt Transfer (DOWN), 2018-04-05 00:43:54.6946268 +29.0507024 (1. Device: Xperia XZ2 Compact)
Pipe Handle: 0x3f8e36a0 (Endpoint Address: 0x1)
Send 0x10 bytes to the device
10 00 00 00 01 00 86 91 4B 00 00 00 00 00 00 00 ......・K.......0

000004: Bulk or Interrupt Transfer (UP), 2018-04-05 00:43:54.8544426 +0.1596748. (1. Device: Xperia XZ2 Compact) Status: 0x00000000
Pipe Handle: 0x3ed70d10 (Endpoint Address: 0x81)
Get 0x10 bytes from the device
10 00 00 00 03 00 01 20 4B 00 00 00 00 00 00 00 ....... K.......

000005: Bulk or Interrupt Transfer (UP), 2018-04-05 00:43:56.4507454 +1.5963028. (1. Device: Xperia XZ2 Compact) Status: 0x00000000
Pipe Handle: 0x40049c30 (Endpoint Address: 0x82)
Get 0x10 bytes from the device
10 00 00 00 04 00 05 40 4B 00 00 00 01 00 01 00 .......@K.......

000007: Bulk or Interrupt Transfer (DOWN), 2018-04-05 00:43:56.4509191 +0.0000795 (1. Device: Xperia XZ2 Compact)
Pipe Handle: 0x3f8e36a0 (Endpoint Address: 0x1)
Send 0x10 bytes to the device
10 00 00 00 01 00 05 10 4C 00 00 00 01 00 01 00 ........L.......

000010: Bulk or Interrupt Transfer (UP), 2018-04-05 00:43:56.4510812 +0.0000366. (1. Device: Xperia XZ2 Compact) Status: 0x00000000
Pipe Handle: 0x3ed70d10 (Endpoint Address: 0x81)
Get 0xc bytes from the device
0C 00 00 00 03 00 08 20 4C 00 00 00 ....... L...

000011: Bulk or Interrupt Transfer (UP), 2018-04-05 00:44:01.7249284 +5.2738472. (1. Device: Xperia XZ2 Compact) Status: 0xc0010000
Pipe Handle: 0x40049c30 (Endpoint Address: 0x82)
Get 0x0 bytes from the device

000013: Bulk or Interrupt Transfer (UP), 2018-04-05 00:44:01.7249517 +0.0000047. (1. Device: Xperia XZ2 Compact) Status: 0x80000300
Pipe Handle: 0x40049c30 (Endpoint Address: 0x82)
Get 0x38 bytes from the device
10 00 00 00 04 00 05 40 4B 00 00 00 01 00 01 00 .......@K.......
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 ........

000014: PnP Event: Surprise Removal (UP), 2018-04-05 00:44:01.7384627 +0.0135110 (1. Device: Xperia XZ2 Compact)
The USB device has just been disconnected from the system.

000015: PnP Event: Device Disconnected (UP), 2018-04-05 00:44:01.7431931 +0.0047304 (1. Device: Xperia XZ2 Compact)
The USB device has just been removed from the system, all drivers unloaded.
[Sony Cross-Device General] jimzhong:
I think you should run the repair tool in Xperia Companion together with the USB logger.
[Sony Cross-Device General] Snow_Basinger:
thank you for the advice

I will challenge again when I have time.
[Sony Cross-Device General] munjeni:
Yes thats right, and log is not soo short like 17kb, its not lower than 2-3 gb! It would be of help too if somebody try flash only cache.sin with newflasher and give me newflasher log maybe I can investigate with it without needs for looking into usb log! In that case I can make test version by guesing (e.g. sin files which have two hashes is probably one hash file joined into two, just guesing! It can be seen in newflasher log). If you want to try newflasher make sure delete all files but keep only cache.sin and cache hash files in newflasher folder, and please don't ask me questions like how to use newflasher :)
[Sony Cross-Device General] jimzhong:
Hi, where is cache.sin? I cannot find any sin files starting with "cache" in the firmware folder of H3123 created by XperiFirm.
[Sony Cross-Device General] munjeni:
I realy don't know since I don't own xa2, but you can use something else, keep in mind use something which will not hard brick your phone e.g. try with userdata, e.g. don't try with bootloader or something similar! If you try with userdata keep your private files on safe destination in case something goes wrong and You lost your userdata!
[Sony Cross-Device General] munjeni:
You are one of those who get things right, and many thanks for that!
[Sony Cross-Device General] Snow_Basinger:
XZ2 Compact dmslog7 files

is this ok?

DMS Export Files
[Sony Cross-Device General] munjeni:
Yes sure, many thanks!
[Sony Cross-Device General] munjeni:
It will take some days before I make new newflasher, need to analyse things first :good:
[Sony Cross-Device General] munjeni:
On first look newflasher no need big update, all .hash files is of no use. But found some minor problems which need to investigate.

1. command at the end of process (set_active:a) sets partition group? There is for example two system partition, system_a and system_b, also there is many more doubled partitions e.g. dsp_a dsp_b, so problem by now is how can we determine which of the "group _a or _b" is active, a or b? Which of the partitions we need to flash, _a or _b? Questionable is ow can we determine that.

2. e.g. system.sin contain two files system.000 and system.cms. Newflasher use basename of that .000 without file extension .000 and .cms as a "filename" to send command e.g. erase:system or flash:system but it fail since instead of erase:system_a or flash:system_a we send erase:system or flash:system, it fail since problem is again that _a or _b which we need to determine.

3. system_X-FLASH-ALL-18AE.sin, but what is system_other_X-FLASH-ALL-18AE.sin? Does exist partition "system_other"? Somebody need to give me partition list from device!

Original flash protocol which is found in sniff log sends some commands (hope to determine which of the partitions is active): Get-ufs-info, Get-gpt-info:0, Get-gpt-info:1, Get-gpt-info:2, these 4 commands gets information from phone to determines that _a or _b. Generaly we need to determine which of the partitions is active _a or _b. At the end of flash proces flasher sends command : set_active:a or set_active:b (which set active partition?). Can we determine _a or _b from gpt? Or from ufs? If you have idea let me know!

It would be good if somebody share partiton info with fdisk of whatewer, I don't own 2018 device so can't get that infos!

Edit:

One great example:

One guy tried newflasher on 2018 device and result was error in dsp since dsp partition not exist (wrongly determined by basename from newflasher)

and that is again _a or _b problem which need to be investigated :) The rest of newflasher is ok, need small right update. Ideas is welcome!
[Sony Cross-Device General] munjeni:
Update about _a and _b... seems _b is _other_ e.g. system_other.sin :) Anybody mount manualy /dev/block/bootdevice/blabla.../system_b to an folder and give me list of files from it?
[Sony Cross-Device General] munjeni:
Found what mean _a and _b, its partition slots, e.g. system partition have slot a and slot b. :) So I have made test version which will not flash anything, just to get some info from device like get-partition-list, slot-count, current-slot, this is test version for xz2, xz2c, xa2, xa2c, please try newflasher from attachment and post me its log, device model, ignore any error, thanks :)
[Sony Cross-Device General] munjeni:
Sin files contain _other_ is in slot B, found that, but hope we figure the rest of puzle. On your mounts it seems that device have activated B slot as a defautl, hope we figure things, thanks mate!
[Sony Cross-Device General] serajr:
fastboot has some interesting vars (see attached pic). Are these vars accessible while on flash mode?
[Sony Cross-Device General] li0nh3art3d:
I think the slot B is for the updates, fastboot, and recovery partition, and that the devices generally boots on the slot A. I say this because I had to switch to slot A after installing TWRP, because upon rebooting with slot B it just kept going to TWRP.
[Sony Cross-Device General] serajr:
Thanks for the all the info you provided! :good:

Edit ^^
[Sony Cross-Device General] jimzhong:
I tested it on a XA2. It got stuck at the get-partition-list and eventually timed out.
[Sony Cross-Device General] munjeni:
Can you try again and post complete log?
[Sony Cross-Device General] munjeni:
Interesting realy! I found in sniff log on xz2c, slob A is flashed, slot B is used for sin files contain string _other_ , probably on that device slot A is active. I realy by now have no idea what about if some devices have B slot as default, than probaly slot A is used for sin files contain string _other_... all that things we need to figure out :)

I can't start working on newflasher since it depend on how you guys are fast for providing me needed info! First thing which I need is output from newflasher test version which I have posted 3 posts back :) Can't start working on newflasher without some urgent infos.
[Sony Cross-Device General] Snow_Basinger:
I tested it with XZ2 Compact

--------------------------------------------------------
newflasher.exe v10 by Munjeni @ 2017
--------------------------------------------------------

Determining available free space by GetDiskFreeSpaceEx:

Available space to caller = 150423 MB
Total space on current drive = 1907725 MB
Free space on drive = 150423 MB

Optional step! Type 'y' and press enter if you need GordonGate flash driver, or type 'n' to skip.
This creates GordonGate.7z archive in the same dir with newflasher.exe!
n
Device path: \\?\usb#vid_0fce&pid_b00b#5&2aa56d36&0&1#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: SOMC Flash Device
Device Instance Id: USB\VID_0FCE&PID_B00B\5&2AA56D36&0&1

Optional step! Type 'y' and press ENTER if you want dump trim area, or type 'n' and press ENTER to skip.
Do in mind this doesn dump drm key since sake authentifiction is need for that!
n
Get-partition-list[0xC]:

00000000 44 41 54 41 30 30 30 30 30 32 36 33 DATA00000263

ERROR: TIMEOUT: failed with error code 997 as follows:
重複した I/O 処理を実行しています。
- Error write! Need nBytes: 0x16 but done: 0x0
nBytes[0x0]:

- Error writing command Get-partition-list:all!

End. You can disconnect your device when you close newflasher.exe
続行するには何かキーを押してください . . .
[Sony Cross-Device General] munjeni:
Hey good info! Thats what I wrote in test newflasher version, found that commands in disasembly of the LILO (bootloader), curently investigating how to send these commands :)
[Sony Cross-Device General] munjeni:
Thanks guys, you are fast thanks! Removed previous test version, hope new one give some info :fingers-crossed:

Edit:

removed attachment
[Sony Cross-Device General] munjeni:
Many thanks man! Its good progress, at least we found right command to get current slot and partition list! :)
[Sony Cross-Device General] munjeni:
This one will give me needed info (test partition slots by choice), in my hope no more needs from newflasher will need :) Other than that we need to figure last thing if this one success: to get idea about a case if device have slot A active by default - what is a path for flashing _other_.sin files and vice versa in case slot B is active by default, it would be a last thing we need :) I'm believing:

if slot A is default than slot B is used for _other_.sin

if slot B is default than slot A is used for _other_.sin
[Sony Cross-Device General] munjeni:
Thanks! Still not understand why get-partitions-info giving timeouts, we no need that info so I have removed it and added some more things, let me show log please! If this one give no error I can easily start making update to newflasher :) Only thing which I can't test is CASE explained in my previous post but hope we figure that out. I have found diferencie in sin files between xz2 and xa2, xz2 don'tt use gzip for sin but xa2 use, allready updated newflasher for that thing. Two steps is remaining for final update, firstt one to implement thinng from test version and seccond to implement CASE which we need to figure and thats all :)

1. For testing device 2018 or older we can use command getvar:current-slot, old devices don't have that command so in case we have reaply thats indication for 2018 devices.

2. _a and _b sufix for command e.g. flash:system we can determine by sending command getvar:has-slot:PARTITION e.g. getvar:has-slot:system, in case YES we add sufix _a or _b to commad erase:PARTITION_a/_b and flash:PARTITION:_a/_b, thats easy

3. CASE will remain
[Sony Cross-Device General] Snow_Basinger:
tested it with XZ2 Compact

--------------------------------------------------------
newflasher.exe v10 by Munjeni @ 2017
--------------------------------------------------------

Determining available free space by GetDiskFreeSpaceEx:

Available space to caller = 150404 MB
Total space on current drive = 1907725 MB
Free space on drive = 150404 MB

Optional step! Type 'y' and press enter if you need GordonGate flash driver, or type 'n' to skip.
This creates GordonGate.7z archive in the same dir with newflasher.exe!
n
Device path: \\?\usb#vid_0fce&pid_b00b#5&2aa56d36&0&1#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Class Description: SOMC Flash Device
Device Instance Id: USB\VID_0FCE&PID_B00B\5&2AA56D36&0&1

getvar:current-slot[0x1]:

00000000 61 a

getvar:slot-count[0x1]:

00000000 32 2

getvar:has-slot:cmnlib[0x3]:

00000000 79 65 73 yes

getvar:has-slot:fsg[0x2]:

00000000 6E 6F no

getvar:has-slot:userdata[0x2]:

00000000 6E 6F no

getvar:has-slot:limits[0x2]:

00000000 6E 6F no

getvar:has-slot:qupfw[0x3]:

00000000 79 65 73 yes

Product: H8324
Version: 0.4
Bootloader version: 1310-7079_X_Boot_SDM845_LA1.0_O_54
Baseband version: 1311-5320_51.1.A.2.213
Secure: yes
Loader version: XFL-SDM845-O-19
Phone ID: 0000:35500309059551,0000:35500309541351
Device ID: 55E5176B
Platform ID: 2008B0E1
Max download size: 104857600
Sector size: 4096
Rooting status: ROOTABLE
Ufs info: SAMSUNG,KLUCG4J1ED-B0C1,0200
Emmc info: FAILEmmc-info not supported
Default security: ON
Keystore counter: 2
Security state: U0vSz8eM1gk1FZup2A6hQlhcZnS9ZR3Up1pg2aPJewY=
Sake root: B7DF
S1 root: S1_Root_e090
Root key hash: C8D92A2DBA1482588EFF712963329500AC7155CEA8494FDD4B7B180F2F871801

End. You can disconnect your device when you close newflasher.exe
続行するには何かキーを押してください . . .
[Sony Cross-Device General] munjeni:
Thats it as expected, perfect!!!
[Sony Cross-Device General] munjeni:
This is last thing soebody need to figure-> https://xdaforums.com/showpost.php?p=76156109&postcount=746 (AB case)! I will start making newflasher update at least for things which we found today. Thanks! Waiting somebody figure CASE. :)
[Sony Cross-Device General] Snow_Basinger:
Good Luck!

I will cooperate if I can understand;)
[Sony Cross-Device General] :
XA2 H4113
[Sony Cross-Device General] munjeni:
Done complletee integration, but please read disclaimer carefully from first page! Test version from attachment have completelly implemented support for 2018 devices but I can't guarant if it will work or if it will hard brick your device, try on your own risk! What have implemented:

- support for non gziped sin files like one from xz2 and xz2c device

- implemented check for every each partition slot, in case partition have slot command like erase:PARTITION and command like flash:PARTITION have appended _a or _b in next cases (curently guesed! Read this first https://xdaforums.com/showpost.php?p=76156109&postcount=746 for idea):

1.) if .sin file contain string _other_ in its filename and your current_slot is set to A by default, that sin files will be flashed to slot B! But .sin files without _other_ will be flash to slot A! At the end of flashing your current_slot will be set to the same default slot A!

2.) if .sin file contain string _other_ in its filename and your current_slot is set to B by default, that sin files will be flashed to slot A! But .sin files without _other_ will be flash to slot B! At the end of flashing your current_slot will be set to the same default slot B!

Note this, this is uconfirmed! I don't know how slots is used in 2018 devices so you flashing on your own risk! I can suggest you only try flash system.sin so in case something goes wrong probably you will not get hard brick, simple repair phone with Sony PC Companion. Again, do on your own risk! Post your newflasher log, good luck! I must go, see you tommorow.
[Sony Cross-Device General] munjeni:
Thanks! Forgot that detail, this one will work https://xdaforums.com/showpost.php?p=76157916&postcount=754 , make sure you use excatly the same system.sin like your installed on your device or device might not boot because the rest of firmware is missing!
[Sony Cross-Device General] munjeni:
Result was ok? Device boots? Need to change "system have slot" and make them one time only, it will speedup proccess :)

Edit:

Uh I see, booth system and system_other_ is flashed to system_a, not a good, must modify tool a bit, thanks!
[Sony Cross-Device General] Snow_Basinger:
No problem:good:, I repaired with Xperia companion
[Sony Cross-Device General] :
system_other and system on XA2 looks good, system booting. sorry but no log, after i closed windows in vmware there was nothing to paste on macos.

flashing it again i see system_other is being flashed to A
[Sony Cross-Device General] munjeni:
I'm fixed some bugs related to _other_ (flashed to wrong partition) and moved slot check routine to run only one time per partition, its faster now, and fixed missing set_slot active at the eng of flashing, let me know :)

Is something wrong with xda? For me page load almost half minute
[Sony Cross-Device General] :
looks good for now, flashing system_other to system_b

EDIT: system_other finished, started flashing system to system_a
[Sony Cross-Device General] munjeni:
Okey, thats great! Made v11 and uploaded to the first post, hope somebody flash whole firmware and tell us about a result, if possible post newflasher log! All changes uploaed to git https://github.com/munjeni/newflasher/commits/master , thank you guys for your time!
[Sony Cross-Device General] munjeni:
Omg, must check that now. Set_active:a replied with DATA?
[Sony Cross-Device General] :
there was only system

EDIT: nevermind, you ment xz2
[Sony Cross-Device General] munjeni:
Found it, missing get_reply, fixing now, will let you know when it is ready! 5 minutes!
[Sony Cross-Device General] munjeni:
Wait please new version, I will tell when it is ready, 2 minute please! Partition image sin file you must move to partition folder! When we get tool working I will do some more sanity checks!
[Sony Cross-Device General] munjeni:
New version is ready, please remember to move partition sin file to partition folder! :)
[Sony Cross-Device General] :
ok flashing
[Sony Cross-Device General] munjeni:
New version is out, this one will work :good:
[Sony Cross-Device General] munjeni:
Finally!!! :good: Now time for porting flashing mechanism :)

Edit:

I see in your log "Error, device replied with:" , seems your mistake when writing post? Or still error?
[Sony Cross-Device General] thrash001:
i try it again..

---------- Post added at 03:35 PM ---------- Previous post was at 03:33 PM ----------

Can you put the penultimate version again online?

---------- Post added at 03:37 PM ---------- Previous post was at 03:35 PM ----------

looks like a error..
[Sony Cross-Device General] tanipat:
wow means we are going to try flashing the real things now?
[Sony Cross-Device General] munjeni:
Not now, need to implement flashing mechanism first! :) I have done only simulation, but I think the rest will be not so hard :)

New version is out, fine tuned things, disabled debug log, done some small modifications and optimisations, lets see now. :) It would be great having full full log with sin, please post full log
[Sony Cross-Device General] munjeni:
Seems perfect! But didn't saw first part of the device info :( It was ok?
[Sony Cross-Device General] munjeni:
One good thing for new Sony flashing protocol, its more safer than s1 protocol! How it works:

1. flasher sends signature, device replies with OKAY

2. In case OKAY reaply, flasher sends command format:blabla, OKAY again, sends data chunks to phone one by one, but good thing is phone is not flashed until we send command flash:blabla!

3. OKAY again, ...etc...etc until end of file

4. at the end of file if everything was OKAY, flasher sends final command flash:blabla, thats a moment when device flashes all data which was uploaded before that command

5. device replies with OKAY again, done

Thats good realy, I like! :) I will try implement flashing functions and it will sends data to phone, but will comment lines "erase:blabla" and "flash:blabla" to prevent flashing, we just need to make things fully working before we uncomment lines "erase:blabla" and "flash:blabla" since we no want brick, right? :)
[Sony Cross-Device General] thrash001:
yea the cmd window erase the first lines. can the output write in a extra log file?
[Sony Cross-Device General] munjeni:
Yes of course! Just make an batch file and put this:

newflasher > newflasher.log

run .bat file and you will get newflasher.log file, or solution 2, open command prompt and type:

newflasher > newflasher.log

you must go to folder first e.g.

cd "C:\users\blabla\home\Desktop\newflasherfolder"
newflasher > newflasher.log
[Sony Cross-Device General] munjeni:
Looks good! Tanipat got inflate error (seems not enought pc hard disk space for system extraction!!?), and also root key hash wrong and you too have Root key hash wrong, the rest seems ok! :) OhhMG

Edit:

Tanipat please use CODE tag when you post log

Edit:

get root key hash I will solve later, it will need for boot delivery, working now on flashing, it will take a long, maybe will not finish today, signature function finished, now porting data chunks proccession function :)
[Sony Cross-Device General] munjeni:
New version is out! PLEASE DOUBLE READ RED WARNING in first post before you double click newflasher.exe!!! I didn't enabled command "erase:blabla" and didn't enabled command "flash:blabla", but all other functions is implemented, signature is sent to the device, data is sent to the device, I am pretty sure nothing will happen since these two final command flashes device, I think if these two commands is disable device will not be flashed, only flashing simulation, BUT nobody can guarant!!! So please flash only cache.sin so if something goes wrong you have chance to repair using sony pc companion! My hope that tool is ok and working, let see. DOUBLE READ RED WARNING FROM FIRST POST! :)
[Sony Cross-Device General] thrash001:
i copy only the cache.sin in the dir from newflasher?
[Sony Cross-Device General] munjeni:
Put cache.sin and neflasher.exe into empty dir, do not put any other sin file in the same dir! :)