[OnePlus 5 Guides, News, & Discussion] optimumpro:
Oneplus 5/5T bootloader included with 5.1.5 firmware allows booting self-signed recoveries and kernels. In short, you generate signing keys; sign recovery and kernel from your current custom rom (kernel could be signed on the phone); transfer recovery on your phone; apply boot signer for kernel; and relock bootloader. This guide borrows from Chainfire's guide and customizes it for our device.

PROS:

1. Virtually total protection of your data, especially if encrypted

2. Inability to flash another recovery, even stock recovery (if OEM unlock allowed is unchecked)

3. Inability to flash another kernel, including stock kernel, (again if OEM unlock is unchecked)

4. Inability to unlock bootloader in fastboot, see above

5. Total inability to flash anything in fastboot. The only access to the phone is through TWRP

6. You can still change/update roms, backup/restore data to your liking

7. You get a different boot warning screen: 'your phone has loaded a different operating system' with a fingerprint (four rows of numbers). Write them down and compare once in a while: if the numbers are different, someone (and I am talking a sophisticated adversary) tempered with your phone

CONS:

1. You would have to set up things once

2. When changing or updating roms, one extra step is required - flashing Chainfire's modified Verified boot signer zip to resign kernel (right after Magisk and before reboot).

The key generation and signing is based on Android source directions and Chainfire's thread about relocking bootloaders with custom roms. So, credit for that goes to him

THESE ARE INSTRUCTIONS FOR LINUX. I am sure there is a way to do the same on Windows

Preliminary steps:

Remember, if you are not on 5.1.5, you may have problems. For example, my own rom, Jaguar Oreo, requires 5.1.4 firmware. I did all the steps and everything worked, except that TWRP couldn't de-crypt. However, I went ahead and flashed 5.1.5 firmware and the rom is working fine. So, I re-did all the steps and now de-cryption works too. This may or may not be the case with your favorite rom, if it is not on 5.1.5.

1. Create a directory on your PC named, let's say, Bootkeys.

2. Get Chainfire's Bootsignature.jar from here: https://xdaforums.com/attachment.php?attachmentid=4136392&d=1493804209 and VerifiedBootsigner.zip from here: https://xdaforums.com/attachment.php?attachmentid=4164411&d=1496000476 and put both files in that newly created directory

3. Get your favorite TWRP (I use Blue_Spark) and put it also in that directory

4. Key Generation:

Run the following code one line at a time from PC terminal opened in your newly created directory. Skip the lines with "#" sign, these are for comments only.

# private key
openssl genrsa -f4 -out custom.pem 2048
openssl pkcs8 -in custom.pem -topk8 -outform DER -out custom.pk8 -nocrypt

# public key
openssl req -new -x509 -sha256 -key custom.pem -out custom.x509.pem
openssl x509 -outform DER -in custom.x509.pem -out custom.x509.der

You don't need to use pem files and can delete them after key generation.

5. Signing:

Rename your TWRP into recovery.img and run the following code one line at a time from the same terminal

java -jar BootSignature.jar /recovery recovery.img custom.pk8 custom.x509.der recovery_signed.img
java -jar BootSignature.jar -verify recovery_signed.img

Your recovery is signed (first command) and verified (second command - the output should be 'signature valid').

6. Open Verifiedbootsigner-v8.zip you downloaded from Chainfire's thread with your PC's archive manager (don't have to unzip it). Grab your newly generated keys custom.pk8 and custom.x509.der and put them into the opened zip. Make sure the files are there and close archive manager

7. Now back to the phone. Flash your newly signed 'recovery_signed.img' (not original 'recovery.img') to the phone via fastboot or in your existing TWRP. Reboot in your new recovery.

8. Now, format the phone - you have to type 'yes'; next, format separately system/cache/dalvik/data/SD. Reboot the phone into TWRP again.

9. Transfer your favorite Rom, No verity (only if your rom is force-encrypt) and Verifiedbootsigner to your SD card. Remember. You must be decrypted to relock. Locking bootloader on encrypted device will destroy encryption key. Once bootloader is locked and everything is working, you can encrypt.

10. Flash the rom, No verity (only if your rom is force-encrypt) and Verifiedbootsigner. Reboot and make sure you are NOT encrypted (in Settings/Security). (If encrypted, stop and return to step 8: you either haven't formatted to factory reset or your no verity didn't work).

Now, back to TWRP: most likely your data is gone, so, re-transfer the rom and and Verifiedbootsigner to internal SD

Now, you are ready for the FUN PART: re-locking:

11. Boot in fastboot and execute fastboot oem lock

12. Reboot. You will get a yellow warning: 'Your phone loaded a different operating system". The first boot may throw you into TWRP. Just reboot normally again

13. Now, you can do whatever you want, including Gapps and Magisk. Everything should operate normally. Just remember, every time after flashing Magisk/update/change rom, you MUST reflash Verifiedbootsigner, as the last step and before reboot, even if during flashing, the script tells you kernel is signed. Follow the script and press volume down to sign again