yashsingh1/xdaforums huggingface dataset:
[Android Q&A, Help & Troubleshooting] Lanchon:
im assuming you rebooted the bootloader after oem unlock and before erase.

well, you stumbled onto a bug. your device is 4 generations old and discussing a bug of your bootloader in detail here is off-topic. many, many devices have security broken, by design or by bugs, but that doesnt mean TWRP should break the security of those -many or few- that get it right.

let's do this anyway but please let's keep it brief.

recovery is only invoked on device wipe (oem unlock), never on fastboot erase. the emulated partition was deprecated 4 years ago, before encryption was added to android. current devices are of the /data/media type and encryption on these devices encrypts everything. on the other hand, fastboot oem unlock wipes everything, including emulated sdcard on old emulated partition devices.

fastboot (device side) understands the GPT but does not know about file systems. for sure fastboot erase did not "just wrote a new filesystem signature", it probably issued a (nonsecure) DISCARD emmc command spanning the partition. (or else maybe it did a no-operation?) nonsecure DISCARD is actually very efficient: it only unmaps blocks. the data is still in the flash though, the NSA could get it back. the emmc will be erasing the affected erase blocks at its leisure, when it needs to, or when it is idle, or the main cpu informs the emmc it's time to do housekeeping (device unused and charging, for instance).

after the DISCARD is issued, you would be unable to read anything out of the emmc. the emmc synthesizes blocks of zeros when reading out unmapped blocks. data is really gone, unless you can hack the emmc firmware (entirely possible for a worthy adversary), or you decap the module and access the flash directly. SECURE DISCARD on the other hand actually wipes the flash, including the many stale copies that might exist of the wiped blocks.

in any case, fastboot erase should be inaccessible in a locked bootloader and whatever it did in your device is nonstandard and not our concern here. write the nexus S off as an insecure device if you will.

moving on, the next point is moot: if you get root you are over the bootloader protection. the whole point of this thread is preventing root access by an unauthorized party.

i never white-listed any phones! for sure all nexus make a desgin effort to defend your data, but implementations can be flawed. in particular, the boot-into-recovery-to-wipe-and-then-unlock bootloader behavior, i can only vouch for it on the OPO where i tested it. and ive stated this in the OP. but according to messages ive read, it seems to hold true for the nexus 4 and 5 at least.

i cannot confirm or deny the fastboot erase vuln on other phones at this time. (this would wipe all useful data on newer phones though.)
[Android Q&A, Help & Troubleshooting] Lanchon:
hi and thanks for answering! and for TWRP too :)

i absolutely understand the cost vs. return argument, i only wrote this thread because i didn't agree with the "we shouldn't do it because it's useless" stance as a way to trigger constructive discussion, such as this here.

as an option to where to store the password:

last sector of the recovery partition: a header plus integrity checking at the end of the sector (that could be the same hash function used to store the password). on init, the recovery reads the last sector of the recovery partition. (that shouldnt cause a problem in fastboot boot scenarios.) if the sector doesnt pass header/integrity checks, continue booting. otherwise ask for a password if the payload section of the sector has a password hash in it.

downside: susceptible to induced glitching attacks during the sector read. workaround (far from perfect): read the sector twice (bypassing caches) and fail to boot if the reads differ. (real workaround: die hard enthusiasts can build their own TWRP that fails to boot if the sector fails the checks.)

the fastboot erase vuln: this should be fixed by google if it wasnt ages ago. they can easily fix bootloaders in the field too. i will try to investigate this and post results here.

usb: well, because i always used patterns to unlock, i never thought i could encrypt the phone with a password then use a usb keyboard to decrypt. this is so simple it's brilliant! :) do you know if this would work while booting standard encrypted android? would the kernel connect usb keyboards before mounting data? (for sure BT keyboards wouldnt work.) if this works i would encrypt my phone.

yes, i stated in post #2 that adbd and mtpd have to remain disabled before auth of course. or some new adbd mode could be used to get the password. or maybe a completely independent, hacked adbd binary could do it; that could make maintenance of standard adbd easier.

lol, i AM serious about security! im paranoid about my printers getting worms! :) really, i am serious, so much so that my phone doesnt keep anything of value outside of my keepass vault. it is protected by a 30 or so character passphrase that cannot be dictionary attacked. you should see me typing that phrase to unlock my laptop harddrive in public places such as airports: im so wary of cameras filming my figer movements, you'd think im scheming to blow something up, its ridiculous! :)

on the realistic side, with the opaque baseband owning the phone, connected to everything, and presumably full of vulns, no phone can really be trusted. phones will never be a trusted device for me.

yes, the first /data/media nexus was the galaxy nexus. the cutoff was probably at android 4.1, not 4.0 as i previously stated, i dont really remember.
[Android Q&A, Help & Troubleshooting] Lanchon:
nexus 4: not true on the opo. the recovery wipes, then the recovery unlocks. if the recovery doesn't boot, the oem unlock doesnt unlock. verified on opo. i heard this is the case on the nexus 4 too. any other design would be silly, i dont deem google engineers silly.

opo: i will not touch my device, but i might experiment on a different one. you have to understand this: modern phones do not have an "internal sdcard" partition, its all stored in /data. if /data goes, everything goes. if /data is encrypted, everything is.

the nexus S: is an obsolete device (yes, it has two flash chips, fast and big). we've established it's not secure. discussing it beyond that point is not relevant to this thread. same thing applies to you xperia.

fastboot erase is not secure. nor it is meant to be, at all. in android phone wiping is done by the recovery, never fastboot. and btw, secure erase is not done writing zeros, it's done with the right secure erase command of the emmc.

opo: i stated eralier: "i cannot confirm or deny the fastboot erase vuln on other phones at this time. (this would wipe all useful data on newer phones though.)" this applies to the opo too.

/system is a possible store. but is very intrusive, is asking for trouble in the future, and interacts badly with flashing and backups. the password can be unexpectedly reset under many circumstances. the last sector of recovery is way better IMHO.
[Android Q&A, Help & Troubleshooting] Lanchon:
confirmed: the fastboot erase vuln is NOT present in the OPO.

(thus, probably also not present in recent nexuses, i suppose.)

rod@rod-latitude ~ $ sudo fastboot devices
[sudo] password for rod:
e615fff	fastboot
rod@rod-latitude ~ $ sudo fastboot oem device-info
...
(bootloader)	Device tampered: true
(bootloader)	Device unlocked: false
(bootloader)	Charger screen enabled: false
OKAY [ 0.004s]
finished. total time: 0.006s
rod@rod-latitude ~ $ sudo fastboot erase userdata
******** Did you mean to fastboot format this partition?
erasing 'userdata'...
FAILED (remote:	Device not unlocked cannot flash or erase)
finished. total time: 0.003s
rod@rod-latitude ~ $

things of note:

1) this is a "new" straight out of the box OPO delivered to spain by oneplus. it says "tampered: true". (mine didn't.) oneplus is probably selling refurbs as new units. some people reported receiving unwiped phones, with pictures taken by the previous owner inside and all. oneplus is also making it next to impossible to return the device on warranty. oneplus, i have to say, is shady IMO. this is the SECOND "new" OPO that i personally found to be sold in the "tampered" state.

2) fastboot (device side) refuses to erase the partition, as it should.

3) "fastboot format" (host command) creates the formatted partition on the PC side, then forwards and flashes the partition image like any other image. fastboot (device side) does not understand file systems.
[Android Q&A, Help & Troubleshooting] Lanchon:
the vuln is not present in the OPO, please see the previous post.
[Android Q&A, Help & Troubleshooting] Defier525:
I asked my friend to send me the partition layout of his Nexus4 and looked at it. Indeed - userdata and sdcard are on the same partition in eMMC. I always thought Android 4.0+ devices all had the ability to encrypt stuff, seems like only 4.1+ support this by default. I will try to find out if the newer stock ROM of the xperia does repartitioning, otherwise maybe I could try it myself. But I think I now get the point about the whole encryptable= option which should allow me to encrypt userdata and both sds on the Xperia using just one key without too much hacking. But this is really OT here, sorry.

Some quick brainstorming on this:

I agree regarding /system being intrusive. Regarding the last sector of recovery I see one issue: As far as I understand, recovery usually is not a filesystem like ext4 but a rootfs image? In the nexus4 it is mounted as "emmc" according to fstab. What does this option do? If it's a compressed rootfs, it would be hard to use it as a file inside it. And if it's a regular filesystem, it might get corrupted if we write some unexpected stuff into its last sector?

Another idea: As we've discussed, most people will wanna use encryption anyway if password-protecting their TWRP. Some of these newer 4.1+ devices you mentioned also seem to store the crypto-block of encrypted devices not in the last bytes of the partition (like 4.0 devices do), but on a special metadata partition if I read the Nexus4's fstab correclty:

encryptable=/dev/block/platform/msm_sdcc.1/by-name/metadata

As you can read in Nikolay Elenkov's article I linked earlier, the cryptofooter contains tons of spare space which is reserved for future use. Since TWRP already attempts to extract the cryptofooter of encryptable partitions during launch, it already reads the cryptofooter anyways we could read a bit more of it and look for the TWRP pin in there as well.

Or even better: Why not use the same PIN for TWRP as the encryption passphrase? You will just have to compare the SHA256 sum of the user-entered password and you gain several enhancements: The user only has to remember and enter only ONE password when opening TWRP and TWRP access could even could be protected by default on encrypted devices (well, would be dangerous in cases fastboot erase does not work and you have a "secure" device like the OPO). But it would be pretty close to stock AOSP and reuse all its functionality.

A major issue with this approach could be the attempt to restrict access to the key on hardware level as it is probably already implemented in some qcom devices on android 5. The question here would be which parts of the cryptofooter are accessible how via API - but this is certainly relevant to TWRP offering key unlock support of data for Android 5+ ROMs anyway.
[Android Q&A, Help & Troubleshooting] Lanchon:
the recovery image is always smaller than the partition. the last sector is completely empty, it's not part of anything. if TWRP adopted this password storage, the TWRP images should be padded with an extra sector of zeroes. that way, you wouldnt be able flash the recovery if the partition doesnt have the extra storage space. the worse that can happen then is, you reset the password while flashing. but setting a password would never overwrite and corrupt TWRP.

using the same password for encryption and TWRP is an option i proposed my original posts. you do loose all access "forever" if crypt gets somehow corrupted, or if you change your password then forget it. it's definitely an option though.

im looking at alternatives given that it's clear that TWRP is not getting this functionality i think.
[Android Q&A, Help & Troubleshooting] Defier525:
Sorry, it's been a while since I read your OP. :)

And if you say that the last bytes in the recovery partition should be fine - okay. But it could be a serious problem for devices like my Xperia which run (custom) recovery from rootfs in boot.img and not from a separate partition. No this is not because imy device is "legacy" - newer Xperias are like that as well. They simply don't have a recovery partition at all, although I don't know what boot2 is about...

I wouldn't say so. I am sure If I understood Dees_Troy correctly (correct me if I am mistaken, Dess), such a patch would need to fulfill some requirements:

- it has to be a patch and not just a feature request (as it is now), they just don't want to do it themselves because they consider its priority too low

- it must make satisfy a purpose. I think even when running encrypted devices which suffer from the "fastboot erase" vuln it makes sense; and if your OPO is not affected that makes it even more justified (at least to me). The biggest drawback here is that it will only offer protection for devices with locked bootloaders, and probably most "TWRP customers" have unlocked bootloaders anyways. But at least it could be valuable for some of the 500.000 OPO owners (if you believe in the numbers...) who run custom roms but value security at least to some degree.

- it has to be low intrusive and easy to maintain in the future

[MENTION=4141005]Lanchon[/MENTION] Please PM me if you are interested in developing such a patch. Most stuff needed for it is already in TWRP (like the lockscreen), so it sounds really doable to me. If it turns out to be too poor for official inclusion in TWRP, you can still manually patch the sources of your TWRP and rebuild it. But I doubt the situation will get resolved if people keep demanding the functionality instead of coding a patch.
[Android Q&A, Help & Troubleshooting] Lanchon:
confirmed: in an OPO, i tried to interrupt "fastboot oem unlock" recovery bootup in a zillion ways and the bootloader never gets unlocked.

rebooted to QHSUSB__BULK, to fastboot, turned off while booting recovery, nothing worked (as expected).

but later, the next time i booted into recovery, stock recovery found "some state" and wiped the phone then unlocked.

DANGEROUS: doing a fast oem unlock with a custom recovery. command will be ignored but may plant a bomb. if u ever flash stock recovery again ages later, booting into it might unexpectedly wipe your phone. (this is all supposition for now.)

it might be worth to investigate where this "command" is stored. it could be stored the same way the tamper and unlock flags are.
[Android Q&A, Help & Troubleshooting] Defier525:
Well, on the OPO it's still not clear to me how the recovery (and not fastboot) can unlock the bootloader? I assume they know which relevant bit is responsible for the unlocked state and directly change/flash this bit on the bootloader, partition. If this is the case, it's a very untypical way of unlocking and very much OPO specific.

Possible cause for your observation: If you had data encrypted, the cryptoheader contains a counter of attempts to unlock the device. I could imagine that this counter is increased each time you are asked for a password even if you do not enter a (wrong) one. Pure and wild speculation though, I assume you still don't have your OPO encrypted, right?
[Android Q&A, Help & Troubleshooting] Lanchon:
youd use the last sector of wherever the recovery is stored. in your case the last sector of the boot partition.
[Android Q&A, Help & Troubleshooting] Lanchon:
no, the behaviour is standard on nexuses it seems, the recovery unlocks the bootloader after wiping, i mentioned a zillion times in this thread!

no, i didnt have encryption and i didnt enter any bad password. the recovery wiped because there was a pending instruction to do so from the bootloader.

finally, no, i wont implement the functionality for twrp and its not simple: if i cant unlock via usb it is useless for me. if i implement something it will be something else.
[Android Q&A, Help & Troubleshooting] Defier525:
Well the question is: Does fastboot check for a flag recovery sets after wiping (so unlocking is done by fastboot) or does recovery unlock itself? If it's the latter, how does recovery actually accomplish this?

And as I said, on both my friend's Nexus 4 and my "legacy" Nexus S wiping can be easily prevented. If the OPO (and maybe the Nexus 5?) is different in this respect they are probably better choices regarding security. But in fact, on devices where wiping can be prevented, implementing TWRP password security makes no sense which further lowers the number of devices which would profit from such a feature.
[Android Q&A, Help & Troubleshooting] Lanchon:
1) non issue.

2) for the record, i won't believe your friend is right about the nexus 4 without further independent confirmation.
[Android Q&A, Help & Troubleshooting] webvan:
After I realized I could remove the password after restoring a (smart)locked backup I started researching this, has there been any update ? ;)

I did find some themes for older versions ot TWRP that added a password but not for 3.x versions ? Thanks !
[Android Q&A, Help & Troubleshooting] Lanchon:
unfortunately Dees_Troy doesn't want to do it.
[Android Q&A, Help & Troubleshooting] webvan:
ok, and themes can't do it anymore for 3.x ?
[Android Q&A, Help & Troubleshooting] quarvin:
Hi, has something changed by now? is it possible to add some kind of lock / password for TWRP?
[Android Q&A, Help & Troubleshooting] letsflyak:
give the end user, the person who BOUGHT and PAID for the device the OPTION to add a password if they want. your normal thief is not knowledgeable enough to hack into these levels you speak of just to resell the phone. Thats their main objective in most cases anyway. Take out that equation and he stole a brick, or at least a phone that has a better chance then 0% in being returned to its owner.
[Android Q&A, Help & Troubleshooting] helblazer:
It's not just that. TWRP and other custom recoveries would do better with a password. Forget burglers or other such elements, even if a kid who just happens to know how to boot into recovery without any idea of what he is doing, he can wipe the data, system and everything else by mistake. Anyone who wants to mess with you can find that phone lying idle and simply wipe or brick it just to mess with you. So a password in the recovery is kinda helpful.
[Android Q&A, Help & Troubleshooting] Lanchon:
> give the end user...

begs the question of to whom you are giving orders. in non-alternative-fact planet earth, the devs of TWRP are not interested. so unless you are willing to pay them...
[Android Q&A, Help & Troubleshooting] Vasilyevaa:
There are topics for the latest versions of TWRP with the option to put a password.

But for the time being it's raw. those. there are no password settings, you can change it only by editing ui.xml and when changing theme there is a possibility that the password will be deleted.

https://xdaforums.com/android/themes/themes-mat-recovery-theme-t3653644
[Android Q&A, Help & Troubleshooting] go vegan:
I think this is an argument for adding password protection for TWRP: Today, my phone freaked out and wouldn't let me log in. No matter what I did, the password to unlock the phone would not be accepted. ADB is turned off, but TWRP is installed. Booting into TWRP, it took me about a minute to REMOVE PASSWORD PROTECTION from the phone and GAIN FULL ACCESS TO THE PHONE AND ALL OF ITS DATA.

I get it that physical security is a top priority, but **** happens. There's no reason why gaining access to 100% of a phone's data should be made trivially easy because TWRP doesn't support password protection. Applying the principal of "layered security", what I did today reveals a very big hole.
[Android Q&A, Help & Troubleshooting] JaggedPill:
Is it possible that we can do something about this ourselves..like adding a solid plugin or integrate security that will enable us to lock twrp.
[Android Q&A, Help & Troubleshooting] JaggedPill:
Is redwolf twrp password protected?
[Android Q&A, Help & Troubleshooting] antonio.galea:
It is indeed possible if you don't mind a little effort - see this gist and confront it with the original TWRP theme file.

In order to activate password protection:

- unpack your recovery image with AIK

- put the new portrait.xml in twres/portrait.xml and change line 742 from

<action function="set">tw_unlock_pass=0</action>

to something like

<action function="set">tw_unlock_pass=MyUnguessableNewPassword</action>

- repack recovery and flash it back to your phone.

Note that to this password alone just protects you from kids abusing your TWRP to wipe your phone - so don't think you are "secure" in any way.

But you can use this lock to achieve a decent level of security in case of phone theft if you lock your bootloader, always have a screenlock with a secure password and deactivate android debugging while the lock is active.

HTH,

Antonio
[Android Q&A, Help & Troubleshooting] Lanchon:
-how will this prevent an attacker to adb shell into your recovery?

-and why do you say "deactivate android debugging"? what do you think is the risk of keeping ADB enabled?
[Android Q&A, Help & Troubleshooting] antonio.galea:
Because adb is deactivated untill you unlock TWRP with your password:

<action>
<conditions>
<condition var1="tw_unlock_pass" op="!=" var2="0"/>
<condition var1="tw_unlock_pass" op="!=" var2="tw_pass" />
</conditions>
<actions>
<action function="cmd">echo 0 > /sys/class/android_usb/android0/enable</action>
<action function="page">unlock_pass</action>
</actions>
</action>

To be precise, there is still room for a race condition - since before the theme starts, adb is already active. You can close that hole too, commenting a single line in init.rc: change the ending stanza from

# Restart adbd so it can run as root
on property:service.adb.root=1
write /sys/class/android_usb/android0/enable 0
restart adbd
write /sys/class/android_usb/android0/enable 1

to

# Restart adbd so it can run as root
on property:service.adb.root=1
write /sys/class/android_usb/android0/enable 0
restart adbd
#write /sys/class/android_usb/android0/enable 1

and there you are - no adb root shell until you unlock TWRP.

Because (depending on Android version, device and screenlock app) there might be methods for bypassing or disabling the screenlock via adb shell. If your adb shell only accepts authenticated connections, you should be fine leaving it on.

Antonio
[Android Q&A, Help & Troubleshooting] Lanchon:
interesting...

but what happens if you also remove these lines?

write /sys/class/android_usb/android0/enable 0

restart adbd

it looks like usb and adbd might be on before the switch to root. running with everything commented out should show us. you know, maybe it isn't that hard to implement this after all.

regarding adb, that's my point: with authenticated adb i don't see any problem with leaving it on.
[Android Q&A, Help & Troubleshooting] antonio.galea:
The first line disables the usb connection, independently of adb - it's kernel related; the other one restarts adb in case it was already running without root privileges. I don't know if it is indeed running or not, but as long as USB is not available to attackers we do not really care.

Don't know if every adbd has autentication (an old tablet I have does not, for instance). If you have it, then actually leaving adb on could be a good idea: suppose you break the monitor, with adb you can access your device anyway.

Antonio