[OnePlus 3 & 3T Cross Device Development] trachyte:
Ok, results from my continued testing. The news isn't great but at least it's damn interesting.

9.0.5 firmware with LOS 16.0-2019042(8? 9?): Does not produce the same keymaster_blob

5.0.8 firmware with LOS 14.1-20170727: Does not produce the same keymaster_blob

In both cases I did a (wipe, clean flash of LOS, go through setup process and enable encryption) x 2, and compared the resulting keymaster_blob's in the crypto footers. They were different every time.

The exact setup for the 5.0.8 test was: Stable5.0.8 Firmware+Modem_OnePlus3T.zip + lineage-14.1-20170727-nightly-oneplus3-signed.zip + twrp-3.3.0-1-oneplus3.img. I chose these because they are from or near 06/2017 when my phone was initially encrypted (not 2016 as I had previously thought). For good measure I copied over my latest userdata (the one it would be nice to recover), and tried to decrypt that. No luck. It wouldn't even recognize that the password was correct: both recovery and system always failed with 'Invalid password'.

Ok, so I'm not able to generate the same signing key. What's left? Let's do a 5.0.8 -> 9.0.5 upgrade again, this time keeping meticulous track of what changes so we can nail down what exactly goes wrong during the process. This is where it gets interesting.

Before: Stable5.0.8 Firmware+Modem_OnePlus3T.zip + lineage-16.0-20190501-nightly-oneplus3-signed.zip + twrp-3.3.0-1-oneplus3.img

After: Stable9.0.5 Firmware+Modem_OnePlus 3T.zip + same OS and TWRP as before. Only the firmware was flashed, using TWRP.

After each step below I saved the crypto footer and the first 10MB of userdata to see if there were any changes. The steps taken immediately after flashing the firmware and their results:

1. Reboot to system, don't enter password, reboot to recovery: Results:

CRYPTO_DATA_CORRUPT not set

Crypto footer has not changed

First 10MB did change (~4K out of 10MB, mostly in a few large chunks but also small changes)

2. Enter correct password in recovery:

CRYPTO_DATA_CORRUPT not set

Crypto footer has not changed

First 10MB has not changed

Recovery returns -33 aka INVALID_KEY_BLOB

3. Boot into system, enter correct password, get 'Password is correct but data is corrupt' error message, reboot into recovery:

CRYPTO_DATA_CORRUPT is set

First 10MB has not changed

Crypto footer has changed:

CRYPTO_DATA_CORRUPT is set

0x90e - 0x92b has changed completely (This is just the SHA256 sum of the crypto footer. A single byte changed when CRYPTO_DATA_CORRUPT was set, therefore the SHA256 sum changed too.)

Conclusions:

1. My theories regarding the crypto footer / keymaster_blob being modified during the firmware upgrade were completely wrong. The crypto footer is not modified except for CRYPTO_DATA_CORRUPT being set, which doesn't effect the underlying crypto at all.

2. Also the HW module signing key is probably not changed. Or at least there's no evidence to suggest it is. If it was changed then keymaster_blob should have been changed too, and it wasn't.

3. It was the userdata that was changed.

4. The process outlined in my last post for (post 5.0.x -> 9.0.x, after reflashing 5.0.8 firmware) is mostly correct, just with the userdata being changed instead of the keymaster_blob.

IK2 is generated correctly

IK2 is used to decrypt the encrypted master key correctly

The correct master key is passed to dm-crypt to set up /dev/block/dm-0. This works even though the userdata is incorrect. dm-0 can be read from, but it returns nonsense data because dm-crypt is combining the correct master key with incorrect data.

Mounting /data fails because dm-0 is not ext4

Let me reiterate that point number 3 since it's the big discovery here:

The userdata was modified.

Not only was the userdata modified, it was modified even before a password was entered. userdata had not been mounted and there was zero chance it could have been modified correctly. Whatever software modified it had no business writing to userdata. It was doing so completely blindly with no idea of what userdata contained. This was a massive f*ck up and shows complete disregard for users' data. In any situation when you're going to blindly overwrite userdata, the user should be prompted for confirmation.

What's next?

1. I'm going to do more 5.0.8 -> 9.0.5 firmware upgrades. For these I'll:

Check more precisely exactly when the userdata is modified in order to narrow down which software actually makes the changes, and when

Save more than 10MB in order to see how extensive the modifications are

Restore the 5.0.8 userdata after it is modified, to see if 9.0.5 keeps modifying it every boot

2. I'll be comparing the changes from the first upgrade to the new upgrades to see if the changes to the userdata are the same each time.

3. Based on the above, try to figure out what the userdata modifications actually are. It's possible it's something like a quick format that might only affect the initial data, or otherwise be limited in such a way that most of the userdata is still intact and recoverable.