yashsingh1/xdaforums huggingface dataset:
[Magisk] shouk_1987:
[MENTION=5228676]VR25[/MENTION] , i have tested it today switching from nos to xtended rom

It is super and almost perfect and it is really a game changer

The main flaw is missing restore apps permission and it is actually critical as you need to open each app to let it ask you the needed permission and you allow

Moreover some apps donot ask for permissions and they misbehave or malfunction and you have to assign permissions manually

I have made a workaround on apps by restoring migrate which overwrite apks and assigned permissions

The lovely thing that all my google account stuff , all system ui settings , wallpaperss were restored from the start at first boot even before magisk

Keep up the good work , you are really brilliant :)
[Magisk] VR25:
**2019.2.8 (201902080)**

- All CPU architectures are now supported. I only provide ARM and x86 rsync binaries, though. Only the more advanced backups need rsync.

- Auto-backup config before upgrades. Also, try patching it instead of overwriting.

- Backups and migrated data have the same format. The wizard is able to restore both.

- After migration, mount Magisk image to /M, so that users can remove/disable platform dependent Magisk modules (e.g., systemless Xposed) with ease. These cause bootloop and/or others annoyances. Alternatively, one can use the the config construct `disable <module ID>` or `remove $MOUNTPATH/<module ID>`.

- Major fixes & optimizations

- Migrate multiuser data as well. Apps are not included.

- Module ID changed to `migrator`. Legacy version (adk) is removed automatically.

- Preserve all traces of standard content providers' data (e.g., contacts, call logs, SMS/MMS messages).

- Super fast snapshots with virtually no extra storage space needed (no sdcard, no problem). Automatic backups are enabled by default.

- Updated documentation (much more detailed and comprehensive) and default config (user config will be reset this time).

- Wizard can be launched either by running `migrator` or `M`. `M -v (or --verbose)` enables verbose for the next running session. The 5th option from the wizard menu shows the project's documentation.

Note: Magisk Manager may not show an update today, since the module ID changed from "adk" to "migrator". So, get the update from here.

Everyone,

Please, read the updated documentation from top to bottom!

The legacy version will be automatically removed. /data/media/adk/ is left intact, but the config is moved to /data/media/migrator/. Old (adk) backups are obsolete. So, that folder is useless if you're not going back.

I'll focus on runtime permissions next. The upcoming version may preserve these. :)
[Magisk] slimshouki:
I did succeeded in migrating but I had to uninstall the module.from mm twrp before os starting up to avoid restoring apps because as soon as apps are restored and rebooting device, os crash and systemui and cannot log in again.

Taking into consideration that this behaviour doesnt happen if I didnt restore user apps.

This happened to me in all.your version except I didnt try ur latest one yet.
[Magisk] VR25:
**2019.2.12 (201902120)**

- Better compatibility (more data preserved)

- Code and config cleanup

- Fix issues (such as bootloop or apps force-closing) caused by incompatible system data - by running `sh /data/M` in recovery. Refer to README.md for details.

- Fixed <some apps not being backed up or migrated>

- General fixes & optimizations

- Include Magisk Manager in backups and migration (`inc topjohnwu`).

- Updated documentation -- refreshed migration and backup instructions, and more.

- Verbose is always enabled for automatic restores.

EDIT

Latest zip uploaded.
[Magisk] shouk_1987:
Ooh seems very very promising

Dude you are really changing the game here

I.cannot see today's version in OP

Where is it ? As I wantto test
[Magisk] Walden0:
Kn magisk repo
[Magisk] VR25:
**2019.2.12-r2 (201902121)**

- Fixed "unset parameter" error during automatic restore. Those who tried to migrate recently and the automatic restore failed, install this update, rename /data/media/migrator/`failed_restores.old` to `failed_restores` and reboot. The automatic restore should resume. Alternatively, apps+data can be restored manually and directly from `failed_restores.old` - using the wizard - even the previous version.

**2019.2.12-r3 (201902122)**

- Fixed "refresh_backups not found".
[Magisk] bkkzak:
Hey I love the concept of the app!

Just FYI, I used the version linked in the quote a couple days ago and had some issues. I had it installed, flashed again in recovery, wiped system,dalvik,data, flashed Feb image (with "-W" removed so it didn't wipe SD card), then ROM install instructions (wipe system,dalvik,data, flash ROM and TWRP, boot into system, reboot to recovery to flash Magisk).

What I encountered was that the data partition (including internal SD card) was encrypted so I couldn't read or write anything to that partition, neither while booted nor while in TWRP.

I had a backup from a few weeks ago on my computer I used, plus combined with Google's backup/restore feature I didn't lose much.

It looks like you changed your instructions after the newer versions were released noting to not wipe data. That was probably the issue I had. Just wanted to let you know in case that helps any. I'd provide logs but lost everything and had to reformat data.

Looking forward to using the module in the future!
[Magisk] VR25:
Even though you wiped (hopefully you didn't format) data, you should have had migrated apps+data intact and restored shortly after boot. It seems to me that the great offender was the "flashed Feb image" part.

Anyway, be sure to always skim through the documentation, preferably on GitHub - so you can notice the changes right away. Also, keep the module updated. If you use Telegram, subscribe to the updates channel https://t.me/vr25_xda/ and/or join the group https://t.me/migrator_magisk/.
[Magisk] chickentuna:
I really wanted this module to work. I have tried a couple of times but it really does not restore data on boot. Tried about 3 times and always ended up restoring data manually thru terminal. At first boot, my google account, wifi and bluetooth pairings are all intact plus it restored all Magisk modules from the previous ROM. What's strange was only the magisk modules gets restored (I know because I see Youtube vanced in my app drawer). No other data was restore even Magisk app and root itself despite flashing them in recovery prior to the first boot.

Anyway here's what I did:

1. Flash latest Migrator module via Magisk app

2. Reboot

3. Confirmed that Migrator module is active in Magisk app.

4. Rebooted to recovery

5. Flashed Migrator through TWRP

6. Installed new ROM

7. Magisk was installed automatically but still flashed Magisk 18.1 just to be sure (tried it without manually flashing Magisk but it did not made any difference)

8. Flashed GAPPS and rebooted

9. 30 mins has passed and no apps were restored.

Although manual restore works, I would love to have this module 100% working as intended so any help troubleshooting is greatly appreciated.
[Magisk] VR25:
Hi,

Sometimes and in some systems, service.sh is not executed. That's a Magisk bug.

Did you grab /data/media/logs/migrator/boot.log?

Do you see "migrator: exc [service.sh]" in Magisk log?

Is Migrator taking apps+data snapshots? These are stored at /data/media/migrator/backups/ and backups.old/.
[Magisk] chickentuna:
Did you grab /data/media/logs/migrator/boot.log? - Yes. Please see the below code.

Do you see "migrator: exc [service.sh]" in Magisk log? - Sorry I don't know where to view this. Edit: Yes. I can see this in Magisk log

Is Migrator taking apps+data snapshots? These are stored at /data/media/migrator/backups/ and backups.old/. - Yes. I can see backups folder but no backups.old folder. I can also see migrated and migrated.old folders if that makes any difference.

Boot.log

+ . /sbin/.magisk/img/migrator/core.sh
+ modId=migrator
+ tmpDir=/dev/migrator
+ rsync=/sbin/.magisk/img/migrator/bin/rsync
+ modData=/data/media/migrator
+ config=/data/media/migrator/config.txt
+ backupsDir=/data/media/migrator/backups
+ backupsDirOrig=/data/media/migrator/backups
+ pkgList=/data/system/packages.list
+ failedRes=/data/media/migrator/failed_restores
+ migratedData=/data/media/migrator/migrated_data
+ defaultConfig=/sbin/.magisk/img/migrator/default_config.txt
+ mkdir -p /dev/migrator
+ '[' -f /data/media/migrator/.nomedia ']'
+ '[' -f /data/media/migrator/config.txt ']'
+ sed -n 's/^bkpFreq=//p' /data/media/migrator/config.txt
+ bkpFreq=8
+ onboot
+ typeset 'pmReady=false'
+ set +u
+ '[' x '!=' xondemand ']'
+ set -u
+ restore_on_boot
+ typeset 'pkg=' 'owner='
+ grep -q /
+ 2>/dev/null
+ ls -p /data/media/migrator/migrated_data
+ grep -iq ^noapps /data/media/migrator/config.txt
+ set +eo pipefail
+ wait_booted
+ grep -Eq '1|true'
+ getprop sys.boot_completed
+ echo x
+ sleep 10
+ grep -Eq '1|true'
+ getprop sys.boot_completed
+ echo x
+ sleep 10
+ grep -Eq '1|true'
+ getprop sys.boot_completed
+ echo x1
+ grep -q /storage/emulated /proc/mounts
+ false
+ >/dev/null
+ 2>&1
+ pm get-install-location
+ pmReady=true
+ tr -d /
+ grep /
+ ls -1p /data/media/migrator/migrated_data
+ '[' -f /data/media/migrator/migrated_data/com.android.chrome.apk ']'
+ grep -q '^com.android.chrome ' /data/system/packages.list
+ >/dev/null
+ pm install -r /data/media/migrator/migrated_data/com.android.chrome.apk
+ '[' 0 -eq 0 ']'
+ rm /data/media/migrator/migrated_data/com.android.chrome.apk
+ refresh_apk_backups com.android.chrome
+ grep -q '^com.android.chrome ' /data/system/packages.list
+ set +eo pipefail
+ 2>/dev/null
+ rm /data/media/migrator/backups/com.android.chrome.apk
+ sed 's/codePath="//;s/"//'
+ awk '{print $3}'
+ grep 'name="com.android.chrome"' /data/system/packages.xml
+ cp -al '/data/app/com.android.chrome-GQMOsimYPLRusZyCp8h3rw==/base.apk' /data/media/migrator/backups/com.android.chrome.apk
+ 2>/dev/null
+ rm /data/media/migrator/backups.old/com.android.chrome.apk
+ set -eo pipefail
+ grep -q '^com.android.chrome ' /data/system/packages.list
+ >/dev/null
+ pm disable com.android.chrome
+ rm -rf /data/data/com.android.chrome
+ mv /data/media/migrator/migrated_data/com.android.chrome /data/data/
+ 2>/dev/null
+ rm /data/data/com.android.chrome/shared_prefs/com.google.android.gms.appid.xml
[Magisk] VR25:
Thanks for the information.

I identified and reproduced the issue.

An update will be up within a few hours. I'll throw in there a workaround for that Magisk bug too, just in case.
[Magisk] VR25:
**2019.2.14 (201902140)**

- General fixes & optimizations

- Workaround for Magisk service.sh bug (script not executed)

Can't upload latest zip right now.

Download the update from Magisk Manager.
[Magisk] Cutieboy:
del
[Magisk] chickentuna:
Thanks for the fix. Works fine now and it's amazing.
[Magisk] Cutieboy:
It renew backups automatically or only with flash zip in TWRP?
[Magisk] VR25:
Backups are refreshed automatically, every 8 hours, by default.
[Magisk] J4jks:
If somehow magisk got issues, with upgrade or any other, restored by boot.img, then all the migrated data moved back to its original place??

We don't lose any data which is from, before to the default backup schedule?
[Magisk] VR25:
Magisk is only needed for initializing the module on boot. Even if it (Magisk) crashes or you loose root for some reason, migrator daemon will still be running fine - until you reboot.

Now, backups and migration are two different and independent features.

No data is ever moved from a running (booted) system. Backups are copies (hard links) of data. Therefore, a Magisk/root failure won't cause data loss.
[Magisk] Cutieboy:
40% apps crashes and stop work. "System interface" also crash
[Magisk] VR25:
Did you follow migration instructions exactly?

...Particularly step 5...

Note: [MENTION=2575765]Cutieboy[/MENTION], since some apps were already restored, a factory reset will wipe these! Make sure the automatic backup has finished, before proceeding - or backup manually with wizard option 4.
[Magisk] Cutieboy:
Migrate on same firmware PixelExperience_beryllium-9.0-20190215-1946-OFFICIAL.zip

It happens not after first boot. But only after ~12:00
[Magisk] VR25:
Hmmm... ?

Have you installed anything else or upgraded Magisk and such?
[Magisk] Cutieboy:
Nope. Now I'm on LineageOS16. Everything restored and works fine. Waiting for next automatic backup at 12:00
[Magisk] Cutieboy:
Done. Then reboot and system hangs on first screen "System starting..." (don't know how in english)
[MENTION=5228676]VR25[/MENTION] tried in my second phone. Same problem

https://youtu.be/qMHMu0zg--k
[Magisk] coombech:
The automatic restore failed, no apps were restored upon boot after flashing ROM. I followed the steps exactly in the readme. So I tried the manual restore using terminal but this fails too on every apk. I receive in the terminal window, Error: Can't open file: /data/media/migrator/migrated_data/cn.wps.moffice_eng.apk as an example. Can anybody please help?
[Magisk] emam damon:
Can i use it on Oxygen os which is encrypted?
[Magisk] emam damon:
Thank you for your answer !
[Magisk] BigBrosMo:
Apparently yes, in discussion on this thread with the dev, it SHOULD work. But I haven't had a chance to try it yet myself. You can search this thread for my posts.
[Magisk] emam damon:
Thank you for your help.

I read whole thread.somewhere he said it is not working on encrypted device s and another time he said it is working.

BTW I installed it and because of formatting data b4 install another rom ,twrp can't read the storage and i had to flash a fastboot rom!!
[Magisk] sachin karinki:
Please provide a Video tutorial, very hard to understand this module
[Magisk] PokemonTotalWar:
That's pretty much an impossibility, since its main point is how much customization you can do. But it's not that hard, if you don't have anything you want to do that's not the default settings, you literally just flash the ZIP twice (one to install and one to run it, if you have it installed already DON'T flash it twice, just once), and it'll automatically do a factory reset, and then flash your new ROM as normal. When you boot up, it'll restore your apps, and whatever data is compatible will still be there. Sometimes there are issues, though, so keep a backup. Anything beyond that you probably shouldn't mess with if you don't know what you're doing, but you can get a full overview of everything you can possibly do with the module in the Documentation
[Magisk] coombech:
I have a problem with the internal sdcard on my Nexus 6P. Since I ran the restore all of my folders in /sdcard have been wiped and replaced by folders with unintelligible letters and numbers! I therefore can no longer download files via a browser as the Downloads folder doesn't exist. I tried to recreate the folder using ES File Explorer but the operation failed. I managed to delete the folders with unintelligible letters and numbers but can't create new ones. I'm rooted with Magisk and have made sure that / and /system are mounted R/W in ES File Explorer. I think it may be a permissions issue but not sure? I have also tried resetting contexts via TWRP to no evail. Any help would be greatly appreciated.
[Magisk] jhedfors:
I am having the same trouble with this tool. I ended up formatting data, and restoring backup. The folders were then recreated successfully, but am back to square 1.

I am on a stock 9.0 rooted Moto X4, trying to migrate to another 9.0 ROM.

Edit: I suggest replacing ES File Explorer with something like Solid Explorer.
[Magisk] PokemonTotalWar:
Are your phones encrypted? The module isn't entirely tested or guaranteed for encrypted phones, as far as I know. And yeah, definitely replace ES with Solid Explorer, MiXplorer, or Amaze, off the top of my head.
[Magisk] jhedfors:
Encryption is on by default in recent versions of Android, however I have removed password/PIN so that it is accessible in TWRP.

[MENTION=5228676]VR25[/MENTION] if it is the case the encrypted devices are not supported yet, perhaps it should be stated in the OP as most devices today are encrypted. However, maybe this does not affect all encrypted devices. :)
[Magisk] coombech:
Yes I'm using Pixel experience ROM by PixelBoot which uses FBE (File Based Encryption). I will try Solid Explorer and see if that allows me to create folders? Thanks
[Magisk] KaMyKaSii:
Forced encryption is disabled when installing Magisk, if you are running encrypted it is because you started manual encryption or did not format data before installing Magisk. So far as I know FDE (full disk encryption) would not cause problems with Migrator but FBE (file-based encryption) I think so. I do not have any device that uses FBE but I have seen several reports on threads from project treble roms that a complete format data (while running encrypted with FBE) is required to do the installation of another rom, a wipe data is not enough. Maybe it's the same problem here
[Magisk] PokemonTotalWar:
Yeah, but you can decrypt by flashing a no-verity ZIP, which is what the author, myself, and a lot of other people do. There have been plenty of people asking about encryption and no answers, likely because not many people have tested it. It sounds like your issue is just that -- the script for some reason broke your encryption, by trying to add or remove files before the device was decrypted.
[Magisk] emam damon:
But in OnePlus 6t with decryption all security method like finger print on lockscreen won't be able to work.
[Magisk] VR25:
Full Disk Encryption vs File Based Encryption - What a Pain

Full disk encryption should be supported, since soon after you enter your password, all data is decrypted at once.

I haven't put much thought into file based encryption. Indeed, there's a potential issue with that.

Migrator waits until /data/media/?ndroid exists (i.e., the folder is decrypted). The problem with this approach is that by the time that folder is decrypted, other data may still be scrambled (encrypted). In other words (my understanding), not all FBE data is decrypted at once, unlike in FDE decryption.

The idea now is waiting for a few minutes after /data/media/0/?ndroid is decrypted. Does anyone have different thoughts/suggestions?

P.S., I don't use encryption.
[Magisk] jhedfors:
Thanks for looking into this!

Perhaps you could add a wait option to the configuration so we could try different amounts of time?
[Magisk] PokemonTotalWar:
This is probably the easiest/best option until something more complex could be implemented, if at all. This way people who aren't using encryption or are using FDE can just leave no wait, and people with FBE can add the line in config. Probably have a commented out wait line with a standard time (2 minutes maybe? Not sure how long it takes to decrypt generally) so they have an example/easy way to enable it if they want to just uncomment the line.
[Magisk] VR25:
Yes, that's what I thought. However, someone who's facing issues told me they don't use encryption. The puzzle just got tougher. There must be something else causing issues, besides FBE.

I'll implement the "customizable timeout" and we'll see how things go.
[Magisk] Estebanium:
Can someone please explain this to me? I don't understand that, when do I need that?
[Magisk] sachin karinki:
Useless to ask this question, i tried asking this question, they never answered it properly.
[Magisk] PokemonTotalWar:
When you flash the same Migrator version as you have installed in TWRP, you get a list of apps it's migrating, and then it will perform a factory reset. Factory resetting yourself afterwards wouldn't hurt anything, but that's all that that means. Flash the Migrator ZIP, it'll either install or run. If it installed, then run it again to Migrate. Migrating will preserve your apps and their data, as well as accounts and a few other things, and automatically factory resets in the process.
[Magisk] coombech:
Please bear in mind though that if you're running an FBE based ROM, it won't work and you will lose data, at least that's what happened for me! [MENTION=5228676]VR25[/MENTION] is aware of the issue, hopefully there will be a fix in a future release.
[Magisk] PokemonTotalWar:
Yes, Migrator doesn't support FBE at the moment. Hopefully it'll be fixed soon. In the meantime, there are other solutions, albeit a bit slower, for people who use FBE based encryption, such as Migrate, the app, TiBu, and Swift Backup. All of them should allow you to reinstall your apps pretty well.
[Magisk] VR25:
Greetings,

Could an *FBE user (with adb setup) screenshot the contents of /data/app/ and /data/data/ (root paths only) before and after inputting the password?

adb shell

su

ls /data/app /data/data

PM me (Facebook|Telegram|XDA) if you have privacy concerns.

*File Based Encryption
[Magisk] coombech:
I have PM'd you with the contents of the directories requested.

Thanks
[Magisk] :
Successful

After many attempts, I have finally been able to successfully use this system (even though I lost all my data).

At the end of the day, we must follow the steps EXACTLY as mentioned.
[Magisk] nogaff:
Question: Can this module help out with a particular annoyance I've been experiencing?

Specifically, I have the the "YouTube Vanced" Magisk module installed which replaces the com.google.android.youtube package. This package has a lot of settings to configure. However, after sideloading the monthly Android updates, Magisk is no longer installed which means the YouTube Vanced package is no longer loaded and my configuration gets erased. Then when I reinstall Magisk I have to configure YouTube Vanced all over again.

So, can Migrator backup the configuration for just that one package and then automatically restore it after the Magisk reinstall?

I've read the documentation, so I think I know how to generate the backups, but I'm not entirely clear on how and when auto-restore kicks in. Does it take place after every reboot or only after reflashing the Migrator module (which I don't want to do since I'm not switching ROMs)?
[Magisk] VR25:
Got your PM and been working on a major update for a while.

Thanks

Can your TWRP decrypt /data?

Simply add inc youtube to config.

The auto-restore kicks in shortly after boot if /data/media/migrator/migrated_data/ or ...failed_restores/ exist.

You can copy (with cp -al source dest) some data from /data/media/migrator/backups/ to /data/media/migrator/migrated_data/ to test that.
[Magisk] coombech:
Thank you for your efforts in fixing the issue with FBE. It will be invaluable to have Migrator working on my ROM as I'm clean flashing ROM's on a regular basis. Yes the TWRP version I'm using can decrypt /data.

Thanks
[Magisk] VR25:
[MENTION=9679871]coombech[/MENTION], do you also have to do that even though your TWRP can decrypt /data?

Hmmm... ?️

I finished implementing that. The default timeout is 5 minutes AFTER the package manager and media (internal) storage are ready.

Reasoning: I'm targeting two issues - FBE and incompatible data.

Regarding the second, users will have more time to determine whether conditions are appropriate for automatic apps+data restore. If issues (e.g., system package FCs) are identified within that timeframe, one has time to reboot to recovery before restore begins. Once in recovery mode, the data wipe wizard (work in progress) can be used to remove just the incompatible data.

Since migrated apps+data can also be restored manually, it's best not to decrease/nullify that timeout value. Manual restore overrides automatic restore.
[Magisk] coombech:
No I only factory reset to clean flash, no need to format data as I'm flashing FBE based ROM's everytime.
[Magisk] nogaff:
Ah, so just to confirm, if I follow this procedure:

Sideload OTA and reboot (at which point YouTube Vanced's config is discarded)

Copy backed up data to /data/media/migrator/migrated_data/

Boot to TWRP and reinstall Magisk

Reboot into Android again

Then I should end up back where I was before the sideload, with minimal hassle? :D

Cool, I'll give it a try next time I update!