yashsingh1/xdaforums huggingface dataset:
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
I feel silly that I did not see the "fix recovery boot loop" option... That worked. ?

Edit2: I was able to backup and restore successfully from the SD card using with TWRP installed. I could not restore persist from internal or sdcard.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] big_edd:
https://androidfilehost.com/?w=files&flid=304372
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] xeropressence:
Looks like the issue was on my end.

The sdcard reader i use (https://smile.amazon.com/gp/product/B07KGR39PL) appears to support mounting both the sdcard and microsd cards at the same time.

Using a adapter for micro->standard it gets detected correctly on syberhexens build.

I guess its probably using sda2 for the microsd card slot which is why it fails to mount. Is this something worth making main twrp issue for?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Forgot to mention that as of test6, I am still unable to restore persist partition and have to revert to official TWRP to do this. I am unencrypted. Let me know if I can provide any other details.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
Heyyo, a new test build of TWRP for payton is going up! twrp-3.3.1-test9 I fixed fstab stuff. Turns out there's a bunch of random extra fstabs I didn't notice that were messing things up, plus some really odd things like binding persist via fstab in TWRP which isn't required. I also cherry-picked syberhexen's 9.0 kernel for TWRP.

https://osdn.net/projects/twrp-for-payton/releases/73018
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] dhinesh_cool:
What are things need to test?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
I assume you mean test7? I have attempted to backup and restore persist, but still get the same failure.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
[MENTION=2091002]jhedfors[/MENTION] can you please provide recovery logs after trying the restore of persist?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Here you go!
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Anybody experiencing dates and times being WAY off? Not sure when this started. In the screenshot of the backup, I appended the current date and time.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] flieswithhawks:
I always have had incorrect date info when using TWRP on Moto X4 whenever I append date to file save name. Currently using 3.3.1 stock and it's done the same things with other versions as well...
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Interesting... I have not experienced this. In this scenario however, I was not using the append date function.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] wkn000:
New TWRP version twrp-3.4.0-0-payton.img (https://dl.twrp.me/payton) is working fine for me so far.

- Able to decrypt data partition with my PIN code when entering TWRP.

- Backup to my SDcard also worked flawless.

But i will still use it only temporary with "fastboot boot TWRP.img" if necessary.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
I received errors where restoring persist partition, as well as the recovery boot loop fix option. Are these two functions working for anyone else?

Also, it asks for my credentials even though I am not encrypted.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] flieswithhawks:
Why is that the case? Are you running a stock ROM?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] wkn000:
OTA update replaces TWRP with Lineage recovery (as usual on A/B devices) and i don't want it to reinstall all that time, that's why.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Anyone else experiencing this date issue in TWRP 3.4.0-0?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Update :Found this post by [MENTION=1162986]Captain_Throwback[/MENTION] that fixed this issue.

https://github.com/TeamWin/Team-Win-Recovery-Project/issues/1549#issuecomment-571764766

I still, however, am unable to restore Persist with TWRP 3.4.0-0.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] SyberHexen:
Is this older persist? I heard someone say not to use backups from older twrp
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
With a suggestion from [MENTION=1162986]Captain_Throwback[/MENTION] to delete the cache folder from persist, it is working now. :)
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] SyberHexen:
Yea we fixed issue of persist being filled up by twrp logs

For as long as I used Payton as dd I'm supprised more users didn't fill this up as fast. On g7 we noticed it out of space which caused many broken radios.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
[MENTION=1162986]Captain_Throwback[/MENTION], [MENTION=7887589]SyberHexen[/MENTION]

With TWRP_3.4.0-0 Is it normal to get a log-in prompt even if you are not encrypted? I enter the PIN that I use on my lock screen, and it lets me in, but does show an error (in screenshot).

"SET_POLICY_FAILED"

EDIT: I tried again, and the above error no longer appears. Interestingly, I can put ANY password in, and I can access everything just fine.

Separately (likely unrelated), I have been having an issue with after every OTA (Pixel Experience), I have to restore the Vendor partition to boot. I have posted in that thread for months now, with no resolution.

See attached logs.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
[MENTION=1162986]Captain_Throwback[/MENTION], [MENTION=7887589]SyberHexen[/MENTION]

I am still having the same issue. Updated log attached. Reading the log, it looks like it detects the existence of an encrypted file. If this is what is causing the prompt, any suggestion of how too find the file and fix issue?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
Heyyo, new test build is going up! twrp-3.4.0-test8 switches to persist images for backup/restore as TWRP seems to have broken persist data backup/restore (same issue on my x2, Z01R and also reported on max_plus, zl0 and zl1 so no idea if it even works on any device :silly: ).

As always, please test ALL functionality.

https://osdn.net/projects/twrp-for-payton/releases/73649

Commits are here

https://github.com/ThEMarD/android_device_motorola_payton-twrp/commits/6af43d64a7fde0577d25c0ff0418f40f0a7742ee
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
Heyyo, so some great news! [MENTION=7887589]SyberHexen[/MENTION] donated me a payton! So I'll be testing my build of TWRP and if it's good? I'll be submitting it for TWRP Official updates. :good:

I got it formatted and I fastboot flashed stock ROM (oof! Didn't know that Payton never got official Android 10 stock ROM that sucks...) so meow I'll test all the TWRP functions and see how it does! ;)
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] dhinesh_cool:
I told u it ****ed my persist:(
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] SyberHexen:
If the files are intact likely the permissions are off.

You can read about rebuilding broken persist here.

https://xdaforums.com/g5-plus/how-to/fix-persist-resolve-imei0-explanation-t3825147/page44
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] dhinesh_cool:
I tried this ,actually I had g5plus before I know this

But on payton persist file is missing

Here u go https://xdaforums.com/showpost.php?p=77249212&postcount=3

Somehow when restoring backup it deleted bak file?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
For the persist issue? It's definitely TWRP who broke it as it is broken on x2, max_plus, Nash and Payton. That's why on my latest test build I switched to persist images. A partition image backup is superior anyways as it is a perfect one-to-one backup and persist partition is super small so it doesn't take much storage or time to backup / restore.

I still haven't had any commuty feedback on the build btw. :Silly:
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] jhedfors:
Hi [MENTION=6235294]ThE_MarD[/MENTION],

I have tried test8 version of your recovery, but it does not boot for me - I have attached the recovery log.

Also, I noticed that the recovery is missing a couple of the advanced options that are present in the official version (ex. fix recovery boot loop).

I have couple other issues not related specifically to your recovery, but have been bothering me for some time:

1. I have always had to recover my vendor partition to boot after ROM update (different ROMs)

2. My device is not encrypted, yet I am asked to enter password in TWRP. I select cancel, and everything proceeds normally.

I appreciate your help.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] ThE_MarD:
Heyyo [MENTION=2091002]jhedfors[/MENTION], Thanks for the feedback! :)

As for those missing options? It must be something that official builds or the TWRP installer adds I guess as my trees are completely based on the official TWRP trees.

I can try and add a check to not enable sbinqseecomd if TWRP thinks the device is decrypted but tbh I haven't tested it on a device with FBE in fstab so it will be up to you to try it out.

Test build twrp-3.4.0-test9-payton with the encryption check is here:

https://osdn.net/projects/twrp-for-payton/releases/73882

Commit history is here:

https://github.com/ThEMarD/android_device_motorola_payton-twrp/commits/59aac183f4b5b40ac4bf0da630634c9115fc6b0d

As for that vendor partition on different ROMs? Those other ROMs are supposed to include their own vendor partition. There's also a chance maybe they're missing some commits from LineageOS? I'm not sure tbh as I haven't checked their trees and I only work on TWRP and LineageOS for any device I have. That would be something I'd bring up with the maintainer of the ROM you're trying to switch to.

[update]

Nevermind, I won't be merging this as it breaks FBE decryption. Users unencrypted will just have to live with tapping cancel.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] JimDandy68:
Hello.

I am having the backup failure issue described here, and the only way I seem to be able to get past it is to remove any security (e.g., PIN) in the OS. Note that I don't have multiple users and have tried all remedies suggested in the linked thread. I am not using any filesystem encryption (at my own peril, I guess).

Like others, when I enter TWRP, it prompts me for a password, and I've just been hitting "Cancel." Everything else works fine, but I get the sense that no one else is having the issue I am with backup. Note that I am using Magisk, if that makes any difference.

My best workaround, for now, is to remove all security before going to recovery to do a backup, and then I get no errors. But is anyone else experiencing this issue, and does anyone have a better workaround?

Thanks

EDIT:

Hmm, well, as I wrote this, I thought, "maybe I should try using the PIN as a password." This seems to "work," yet I don't understand what it means. Am I actually using FBE without knowing it, or is this the way to do things with payton TWRP?

EDIT 2:

Well, I didn't do an adequate search here on XDA: ROOT METHODS for MOTO X4 [7.1.1/8.0]. I have not yet figured out how to follow up, but at least that might get me on the right path.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
My changes to init files in /recovery/root are not being picked up in builds... hmmmm.

May need to relink libs manually or copy individually from within BoardConfig.

This thread from the OnePlus2 seems remarkably​ similar to our situation:

https://github.com/TeamWin/android_device_oneplus_oneplus2/issues/1
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
If I manually try to decrypt from the TWRP command line, this is the error.

Seems that qseecomd is not being brought up at all in recovery. Also digging through cryptfs.c, which may require some device-specific changes.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] gee one:
Yes, the basic UI is working. Decrypt and USB are not. I also have a different fstab, which is breaking some of the mount points. I'll sync them and confirm that it works. Edit: yup, the A/B stuff was tripping me up. I think my build is at the same place as yours. The recovery.img does pack successfully- I think because of some of the changes in BoardConfig.mk

I'm not sure it it will help, but chainfire seems to have the stock boot.img on firmware.mobi.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
I uploaded the boot to there back in October so people could build their own root packages.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] johnjingle:
I bought an extra sdcard to help you guys test when you're ready. I appreciate your work!
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Maybe we should focus on USB for a little while... abd would be a real nice tool to have :)
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
I thought this would do it, but it isn't there yet... maybe need to work on init.usb.rc ?? (NOTE: I realized later that lun path is missing on recovery boot!)

NOTE: this flag ^^ will definitely be needed as the .lun file is not in a standard location.

And yes, init.usb.rc is written with hard-coded addressing to the wrong location. So do we re-write it, or do we declare this (below) in BoardConfig and just inlude proper addressing in init.recovery.qcom.rc ?

---

Tested with the custom .lun and excluding default USB. Didn't bring up USB, but causes the adb sideload feature to crash in TWRP so it did *something* at least :)

Have confirmed that there is no other .lun than the one listed above, so it seems that this must be the correct location.. NOPE! This is a symlink that is not established until a full boot... so it doesn't exist in recovery. Hmmmm.... the reason TWRP can't find the file is that it does not seem to exist :)

---

Another tidbit... init.rc is calling init.recovery.usb.rc which does not seem to exist. I have added it to my tree and just duplicated the USB bringup in the qcom script for now.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
On the decrypt front, take a look at

init.qcom.factory.rc

It tells us how the decryption process flows on a regular boot... needs to flow similarly in TWRP, so certain things will need to be brought up before others.

---

Pushed a major clean-up to github, and some new changes to test a different direction altogether. Hoping to tackle both USB and decrypt with these changes.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
OK, getting somewhere now.

Latest changes (pushed to github) have caused the lun.0 folder to be built where we expect it (based on the custom path in BoardConfig), and logs show that it is being properly brought up by TWRP.

MTP thinks it is working, the service starts correctly and sets the path to the SDCard...but no connection via USB. The "Mount USB Storage" button has appeared and seems to at least attempt mounting (I have no USB C OTG cable handy to check with). Unfortunately, ADB is still non-functional. But, obviously this is the right direction to be moving. Probably just need to correct some of the symlinks or hard addressess used in the generic qCom scripts I based this on.

The remaining USB error seems to be that one or more of the scripts is still looking for

and a few other files that are likely located elsewhere. [CONFIRMED: These are all located in /config/usb_gadget/g1]

And finally, these changes also exposed this error in the log:

Which makes me believe our libs are not being properly included, therefore breaking the decrypt process.

Out of time for today -- feel like if I had another couple hours, I could get this finished! Maybe a little time tomorrow.

---
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] gee one:
I blink for a few hours... nice work! Regarding the vendor files. There doesn't seem to be a vendor partition on this device. There is an oem partition that has higher level apk stuff. It looks like the vendor libs are in /system/vendor on the system partition. There are also libs in the /sytem/lib folders- I haven't tried to see if they are different yet, although I would assume that they are. Maybe this is a stop gap measure to prepare for project treble?
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
I think we need a symlink to direct /vendor calls to the right place.

This is implied in some of the scripts... maybe just need to make it more explicit.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
USB still not working...

...but the "failed to open driver control" error is new.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
---

And this is in the default.prop log:

That can't be good.

---

Made some changes to scripts and a few source files that eliminated all of errors about "idVendor"
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] kraatus90:
Can we use this recovery to flash kernel zips ? or its not possible atm ? asking this because i have made an EAS-kernel for this, and current flashing it via fastboot method, but i can't install WiFi module that way.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Should work. I have been flashing my own tests with from TWRP for a few weeks.

I will post an updated build with the fixes for SD card later this morning. Then you can make a nandroid of everything except /data first.

New build should also support USG OTG storage, but still no adb/mtp.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Incidentally, I brought up a LineageOS environment yesterday and am going to start trying to finish the device tree.

If decrypt can't be permanently fixed for stock/FBE (looking very difficult) at least we should be able to flash custom ROM and kernel, then format data and choose a TWRP-compatible encryption.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
For anyone following along, LOS source now seems to be better equipped for TWRP than Omnirom... I am still, obviously, using the Omni bootable recovery repo, but have shifted over to LOS, and will be updating the device tree later today.

LOS 14.1 source compiled TWRP without any additional repos or modifications... Omni was much more finicky. I am hoping this allows me to work in parallel on an eventual test build for LOS.

As none of my USB/decrypt fixes have worked thus far, I will be stripping BoardConfig.mk right back down to the bare essentials and starting over in my LOS environment.

NOTE: BoardConfig flag "RECOVERY_VARIANT := twrp" breaks the build process... strange, but it compiles for TWRP fine without it.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
- OP updated with current build and updated device tree links (still considered experimental/advanced use only)

NOTE: Recovery logging seems to be broken again...they just show "Error: File not found". Will need to fix this and update. (seems to only work when rooted... will work on it).
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
OK. This is where we are at:

Decrypt: I have removed all but the most basic flags, and TWRP still says it is being compiled without crypto support. This is pretty complicated, and without adb it is a real pain.

ADB/MTP: from kernel log, it appears that the adbd service is getting stuck in a start/stop loop. But it ends by saying it is starting...and certainly works on full boot. But TWRP log shows that adbd service is stopped. [NOTE: This error was fixed by adding some init changes. Still no usb in recovery].

kernel log:

Recovery Log:

[This is fixed. Now says "running".]

---

As noted in previous posts, there is a widespread discrepancy in the source files relating to the location and content of /config/usb_gadget (our actual location) versus /sys/class (the expected location).

This seems to be the source of some our usb troubles in recovery.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Speaking of kernels...

Moving forward, for working TWRP I am pretty sure we are going to need a custom kernel to disable /data encryption on first boot and permanently provide fde as the default encryption type in fstab.qcom
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] kraatus90:
I can make one with no huge changes. btw i tried alpha 3 twrp, it doesn't mount internal storage and i don't have any external sdcard. so can't install kernel atm. :p

been working on EAS kernel for a week now, having some really good results with it for both performance and battery life. just wifi is broken atm.

but it can be fixed, just not that easy for now. if everything goes well, this SD630 will be the first soc to have EAS working apart from sd835. :D
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] johnjingle:
I can try if you want, I have a couple external sdcards I can use.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] kraatus90:
Okay thn, i'll make a thread tomorrow so that more people can help test new kernel. we need more tests because i'm using a custom energy cost model for sd630 since i can't find or make proper one without all those testing equipments. although it doesn't really need 100% accurate data for it.

but different types of use cases can help fine tuning the energy model. :o
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Yes, it is a catch 22. Internal storage is on /data/media, which is encrypted with FBE that TWRP doesn't handle and we can't format and switch to FDE without a custom kernel :)
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] kraatus90:
Any idea what needs to be changed in kernel for that ? i thought it's just via fstab file.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
It is /system/fstab.qcom, which seems to overwritten with the original on boot no matter how it is modified... may also need to disable verity. I patched a stock boot.img to test with, but was never able to format /data fully anyway.

CORRECTION: That should be root /fstab.qcom !
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] kraatus90:
That's weird, i have removed "verify" from it and it is actually removed when i check it, that's how i'm able to boot even after many changes in system.

i'm looking at some custom kernel sources for op5t which has FBE, can't see anything for disabling it so far. :p
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
Not disabling FBE but enabling the option to change it (with the flag "forcefdeorfbe="), and preventing it from being overwritten. (the "format data" button remains mysteriously absent). Right now I can mount rw and use a flashable script to append the forcefdeorfbe tag into fstab.qcom. I can then use TWRP file manager to copy that file back to the sdcard... and when I reboot I see that /fstab.qcom is back to the original version despite the copy being the modified one. It is getting rewritten during boot somehow.

The problem, then, is how to *fully* format /data. Regular methods seem to leave the encryption footer intact and/or FBE is force enabled on boot.

I will be delving deeper into this as I see it as a more viable long-term solution. FBE is not friendly to TWRP development
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] johnjingle:
I have been working looking at the code and not the best with this. I understand the difference between FBE and FDE, but what is posing the biggest problem? I wish I could help more.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] mightysween:
The problem remains that TWRP has only rudimentary support for FBE, and then only for devices using Google's implementation of FBE.

The easy answer is to just stop using FBE...

But TWRP's "format data" button has gone missing, fastboot format fails, and changes to fstab.qcom that would allow us to select the more TWRP friendly full disk encryption do not stick. Frustrating.
[Moto X4 ROMs, Kernels, Recoveries, & Other Develop] johnjingle:
And without ADB working in TWRP, which I have tried yours, we can't format data via PC.