yashsingh1/xdaforums huggingface dataset:
[Magisk] VR25:
Once upon a time, I couldn't access the internet, but decided to switch ROMs anyway. The pain of setting everything up from scratch hit me harder than usual. To make matters worse, the most recent nandroid backup was corrupted. I couldn't go back. At that point, a thought crossed my mind - "Why not automate this mess?"

To be continued...
[Magisk] SayantanRC:
[MENTION=5228676]VR25[/MENTION] your module name is very similar to my app name "Migrate - custom ROM migration tool"

https://xdaforums.com/android/apps-...ation-tool-t3862763/post78060823#post78060823

It would be great if you would like to rename it or add a message that your module has no link with Migrate. It will be good for both of us. Thanks.
[Magisk] VR25:
Indeed, the description was too similar. I removed it and added a note to post #1.
[Magisk] slimshouki:
Gr8 app

I.hope you fix all.bugs concerning notifications and sms time stamps after restore

Dear [MENTION=5228676]VR25[/MENTION]

I like.all your modules a lot except I cannot try this one since it is not compatible with magisk 18 yet
[Magisk] VR25:
Magisk 18 is supported since December 25, 2018.
[Magisk] slimshouki:
Its saying backupd not found in terminal in wizard!

Also I tried to read a lot in order to figure out how to backup extra things but seems complicated

I need to backup sms, Apps permissions, battery optimized apps.and restriced apps.
[Magisk] VR25:
I forgot to rename "backupd". That's not a big deal, though. An update will be up soon to fix this.

Since this has a command line interface over a GUI, that's expected. However, for this same reason, most things are automatic and straightforward. For instance, what you just tried (wizard option 4) is done automatically and incrementally (every 8 hours by default). In essence, the command line part is really for advanced users only.

Regarding backups, these are just a secondary feature - the other side of the coin. You don't need to backup anything before using the data migration (and automatic factory reset) feature. In fact, even if you don't have the module installed, flashing it twice from recovery triggers data migration and factory reset. This is because apps are not copied (this takes time). Instead, they are moved to a location that is not wiped out by a regular factory reset. As you can imagine, this process is completed in seconds.

Next, during the automatic factory reset, select data is left intact. A few examples are generic system settings, Magisk modules and settings, SMS/MMS, contacts, Bluetooth and WiFi data.

Finally, upon boot, apps are reinstalled (because recompilation is necessary) and respective data - instantly moved back to /data/data/. Symbolic links and filesystem permissions are handled afterwards.
[Magisk] slimshouki:
Ok

Seems promising and very good

But I need a simple straight forward instructions in order to migrate to another different rom

Do I have to manually backup?

And if its automatic, what shall I do now to migrate?

Do you mean I just flash the module twice through twrp.and thats it.and then when do I flash the new rom and kernel and magisk and dfe

If its done through twrp then why magisk.module?

Is it for schedule.backups.
[Magisk] dyingbread:
I got update process ended with error 7 in twrp while flashing new rom.

I did it like this.

1. Install migrator, reboot.

2. Setting config.txt, just exclude some apps.

3. Reboot to twrp and flash migrator.

4. Flash rom, ended with error 7.

Did I miss something?:confused:
[Magisk] bnwg:
Thanks for sharing your experience trying this. I'm tempted to try and think I will after I back home from business trip.

I think one logic point is depending how big is the total size of all apps being "move" around it will vary the time taken. Nowadays people used to install many apps and its data should be huge.

The other factor may be affecting is whether it "move" to internal memory or external like sdcard. I use magic binding and found out the time taken with and without using magic binding have huge different. Usually it's min v.s. 20mins for 1G size file. Reading/writing to sdcard is significantly longer than to internal memory.
[Magisk] handers1234:
Yeah same thought if this module main function is alternative for backup app like titanium or even twrp,then why backup is mandatory before actually using this module,I think that kill its purpose.
[Magisk] The Wrath of Kahn:
Glad I found this before attempting it. Perhaps an option to copy data BEFORE committing to a move would be wise? Regardless, it would also be good to update the url in the description to point to this thread. I'll hold off until we get more feedback and hopefully a safer method of migration.
[Magisk] 93ae8074:
May I suggest you try a manual "mount -o bind" to some directory in your external_sd first to make sure it works? I reviewed the code before and made sure that a "mount -o bind" works in my case. Module work as promised, moved twice between ROMs last arvo.
[Magisk] slim_pickins:
I picked a bad day to scroll through Magisk Manager to see if there were any interesting new modules to try. Luckily I didn't lose ALL my data, but every time I reboot it erases the data of all Google System apps & resets the settings in Magisk Manager. I didn't do a nandroid backup beforehand bc doing a factory reset isn't that big of a deal for me as I keep all my important data saved on a usb stick.

My question is, what happens when I uninstall this module? I just need to know if I need to set aside some time to restore everything manually afterwards? Btw, I tried running the command "adk" in recovery (twrp) to uninstall it & set everything back like it says in the description, but it didn't recognize the command.
[Magisk] henningsieh:
I think our feedback is ok and correct because the script results in an anoying mess - you are right but this is independent from having a backup or not.

--- Post added at 10:16 PM --- Previous post was at 10:13 PM ---

I suggest the script must perform that mandatory testing... according to the "fire and forget" manual. :eek:
[Magisk] djcoolguyno1:
I tried it on my redmi note 5 pro

My screen is totally white after boot dirty flashed rom but same result

Had to clean flash everything
[Magisk] J4jks:
I tried uninstall it as said in OP , from recovery but didn't work until I flash again from recovery then adk worked in terminal an asked to delete "adk" completely and move data I did yes then rebooted

"mv: can't rename /data/datarestore no such file or directory

Good bye"

This is what shown after reboot I lost all user data and few user apps started crashing

In data/media/adk/ I have not found any apks before unistall this module, found only data of the apps but no apks have backedup , apk backup folder not created in said location
[Magisk] Mr. Y:
There some real potential here if this work because I have a app with 3.+GB and I have having to set it up again after every clean install.
[Magisk] VR25:
Everyone who claims their data has been lost, please check whether it is in /data/media/adk/.appData.

I'm not here to blame you, but I even wrote "app data backup is RECOMMENDED" and clearly enough, the last word in that line is capitalized. Besides, why wouldn't you just backup your data before trying something that moves it around?

Ok, nuff yelling. I assure you this works (for me at least) and I've tested it multiple times before publishing. As a rule, I don't post anything which hasn't worked flawlessly at least on my device. When posting, warnings are always included and clearly visible.

Now, back to the main subject. If your data is in the aforementioned folder, then it is not lost. All you have to do is reboot into recovery, go to terminal and run "mv -f /data/media/adk/.appData/* /data/data" whithout quotes. And of course, uninstall or disable adk with Magisk Manager for Recovery Mode.

If it is not clear by now (even though that's in the README), no app data is moved to the sdcard. Besides being dangerous, that would take ages and be extremely inefficient. Did I say dangerous yes? Good. App data goes to /data/media/adk/.appData. Since that's still in internal storage, the moving process is executed in a breeze (could be well less than 10 seconds). What goes to the sdcard (if it's available within the 90 seconds timeout) are APK copies. If no sdcard is found, APK backups are stored at /data/media/adk/apksBkp.

Last, I haven't tried this module on Magisk v16.7. I'm currently on 16.6 and everything is going fine. Don't look at anything here as insensitive and/or poorly tested, as that couldn't be farther from the truth. I'm here to provide useful tools and support. However, I cannot guarantee those work for everyone, considering all which is involved. Thus, disclaimers/warnings are never unseen.
[Magisk] neekless:
People who lost data, was it because you wiped internal storage on twrp?
[Magisk] SuperSandro2000:
[MENTION=5228676]VR25[/MENTION] what device are you using? Can imagine that this could not work with Samsung devices cause reasons.
[Magisk] hackiee:
Successfully installed adk, but at the time of restoring back it gives following

Please help
[Magisk] HatRiGt:
Actually for being in safer side , he told us to do that. If it works fine for first time, den we need not to do that for 2nd time.
[Magisk] J4jks:
I seen the app data is linked from data/media/adk/.appdata but apks have not backed up in said location and

No config.txt file is there

Redmi note 3, AEX N with magisk 16.0 + system less xposed
[Magisk] J4jks:
Screenshots were not uploaded in previous posts, XDA labs error
[Magisk] Urnyx05:
Thanks for this module.
[Magisk] ovizii:
Thanks for getting back to us and clarifying things. The only part I am still confused about is the one I quoted above.

You reference the README:

I and probably many other read that as:

user and updated system apps' data = app data

Even the name: App Data Keeper suggest this tool keeps app data...

I think this was part of the confusion when people started using this extension then freaking out.

And again, in the same post of yours you talk about app data being moved again:

I hope this explains why some people expected their app DATA to be moved to the SD card.

Now back on topic, I experienced this problem with a Samsung S7 and I do have indeed "stuff" inside /data/media/adk/.appData - but why would I lose all my app settings after your extension apparently only moved my apks around?

Do you think you could supply a "test" script we can run on our devices to check for compatibility? Apart from all these issues, this is an ingenious solution, especially the rsync approach to back up apks.

###edit###

I had also opened a github issue but I'm going to quote it here for completeness: https://github.com/Magisk-Modules-Repo/App-Data-Keeper/issues/1
[Magisk] VR25:
A new build is up. See the changelog -- first post.

Clearing up confusion...

This module redirects user and updated system apps data (yes, data only) from /data/data to /data/media/adk/.appData, which is not affected by TWRP's standard factory reset. In addition, it copies (doesn't move) the respective APK's to $sdcard/adk/apksBkp or to /data/media/adk/apksBkp if no sdcard is found.

IMPORTANT NOTE: from today till further notice, I'm no longer supporting non-stable Magisk versions. The reason being, Magisk beta/unstable builds are actually behind most modules issues (yes, I said "modules"). I tested that further just now and had to downgrade to v16.0, the latest stable. Smiling ever since.

P.S., I use a Moto G4 Play and a Moto Z Play, by the way.
[Magisk] J4jks:
This time backed up data so I'm on safeside as after reboot from flashing in twrp app data is lost,

It is failed with this time too

Couldn't access location error in all apps

You can see the error in logs

Redmi note 3 AEX N Magisk 16 +xposed system less
[Magisk] VR25:
Backups are automatic and incremental. If you have an SDcard, backups are saved to <largest partition>/adk/backups/, else, /data/media/adk/backups/.

Backups are enabled by default as a secondary data protection measure. You can disable that by replacing the backup folders (listed above) with empty files.

Again, backups are a secondary feature. If you don't need that, you can use this module only for migrating ROMs.

How to migrate

1) Flash the same version again from recovery (or flash the module twice if it's not already installed). This will move apps and respective data to /data/media/adk/migrated_data/. Next, /data and /cache are automatically wiped, leaving select data intact (e.g., Magisk modules, Bluetooth setting, encryption keys).

2) Flash ROM, Kernel, Magisk (not modules), GApps, etc..

3) Reboot

Once system has successfully booted, apps will be automatically restored, one by one, according to your config. No prompts, no notifications - you'll simply see apps popping on the app drawer.

P.S., it's possible to backup any root-accessible data from your device to a remote location (over WI-FI, internet, Ethernet... you name it).

It seems your ROM doesn't want to install because of the existing data. That data is not supposed to cause this, though. Without a recovery log, I can't determine what went wrong.

Try the following

1) Backup /data (migrator's leftovers).

2) Flash the ROM

3) Restore /data

If that fails, one more thing you can try is factory resetting (using TWRP). The downside is that after this, only apps and respective data will be left. Other goodies, such as system settings and Magisk modules will be wiped.
[Magisk] slimshouki:
He just needs to reboot to recovery and continue flashing new rom after running ur script

I beleieve its twrp bug
[Magisk] slimshouki:
Unfortunately, I tried ur zip many times without success.

Everything working smoothely until.I boot into new OS (cRdroid from NOS) and then phone starts to restore apps (about 6 apps) then it stops without any msg or notification and I knew that from app.drawer so when I tried to reboot, os becomes corrupted or system crash since black screen comes after boot image and no option except.reboot or power off only.

Everytime.same symptoms.

Here is what I do exactly

Flash ur zip

Reboot.into recovery

Flash vendor 9.1.24

Flash crdroid

Flash gapps

Flash magisk

Flash dfe

Reboot.into os

Coming from.NOS
[Magisk] VR25:
Which apps were these?

Did you include any system app?
[Magisk] dyingbread:
Attached a recovery log with eror 7.

I tried with manual wipe (nowipe) but my app+data is not restored.
[Magisk] VR25:
You didn't forget to install the module, did you?

Recall that when you use "nowipe" and perform a manual factory reset, all Magisk modules are removed.
[Magisk] dyingbread:
You mean after I flash rom, gapps and magisk I need to flash migrator aswell?

Or I need to flash migrator after the first boot?
[Magisk] VR25:
Any time is fine. The important thing is having it installed. Even if you rebooted a thousand times, you can install it and the automatic restore will still kick in.
[Magisk] slimshouki:
Normal user apps

And not necessarily it restore.same.apps
[Magisk] VR25:
Incompatible system data perhaps... ?

Try resetting system settings (rm -rf /data/system*) from recovery terminal.

The new build doesn't keep these by default.

**2019.2.2 (201902020)**

- Added `delete` command for removing select data after a factory reset.

- Automatic backups are disabled by default (`nobkp`).

- Experimental fix for push notifications.

- Fixed wizard option 4: backupd not found.

- System-specific settings are not preserved by default. Refer to `README.md` for details.

- Updated usage instructions, module description and defaut config (user config will be reset).
[Magisk] shouk_1987:
I am going to move rom same as user [MENTION=4991063]slimshouki[/MENTION]

So the new build doesnot need the above line ?

And if it is required it should be done after flashing the module twice in recovery and then flashing rom with all its extras and BEFORE boot to.write this command ?

Also crdroid dev mentioned that magisk should be flashed after system boot in clean flash while you are saying to flash magisk before boot which contradicts to your steps

Kindly reply to me with the correct process to be on track

I will be using your module for rom migration only and not normal frequent user apps backup
[Magisk] VR25:
The new build doesn't require that command and by default, it doesn't automatically backup apps.

You only flash twice if the module is not installed. If it is already installed, flash only once.

My ROM fashing steps are not strict. If the dev says to flash Magisk after the first boot, then do so - no bid deal. Once Magisk is up and running, Migrator will do its job.

Btw, latest documentation is much clearer. Take a look at the updated README.md.
[Magisk] shouk_1987:
Thanks for your answer I will.read readme for sure

However I want to backup all apps and it.is the main module usage for me

I was tslking.flashing twice as I am not installing it at all

My quest is to go to twrp flash module twice then flash all my new rom files for clean flash then boot up with all my apps and some system backups like bluetooth wifi etc

How can I accomplish such quest with the easiest steps ?

Thanks again for your huge efforts and assistance
[Magisk] dyingbread:
Jus tried again and it works. But yeah there is a little problem like some apps are keep getting force close when I open it, 5 out of 77 apps. So, it's fine. I could just reinstall those 5 apps. I'd rather do that than to install 77 apps whenever I flash a new rom.

I have a question. Is there any differences between migration on first boot and restore manually with the wizard from terminal?
[Magisk] VR25:
This is still under investigation.

Yes, these are two distinct and independent features.

The automatic restore data comes from /data/media/adk/migrated_data/ and the process is faster than manual restore. The end result is the same, though.

The manual restore is only possible if there are backups available. The automatic restore starts on boot whenever /data/media/adk/migrated_data/ is not empty. You can't manually restore data from /data/media/adk/migrated_data/.
[Magisk] BigBrosMo:
A little confusion on the encryption stuff... I have OOS and I guess it's encrypted... I have to enter a pin to get into twrp etc... I can flash no_verity and get rid of encryption but I think I would have to reinstall/wipe my device to do that and start from scratch. Am I right that Migrator requires this? Ie the device to be unencrypted?
[Magisk] shouk_1987:
I am keeping default settings and using migrate rom feature

Cleaned flash new rom and after flashing magisk saw around 4 apps in the app drawer then it stopped and no more apps are loading I have around 80 user app while the system ia responsive normally and not frozen

Also noticed that apps permissions are not backed up as I opened a music app from the 4 restored apps and it asked for permission

while all modules , wifi , bluetooth were restored while contacts didnot restore

Also worth mentioning that I didn't sign to my google account but I think it is irrelevant as whole gapps is an optional thing

I really want to use migrator module but no luck till now

I have reverted to my twrp backup waiting for the module to be solved
[Magisk] VR25:
Yes, /data must be decrypted before flashing migrator (TWRP).

Once system has booted, it waits until /data is decrypted to do its job.

This seems to happen randomly for me.

I didn't implement runtime permissions preservation yet. This will come soon. I'm focusing heavily on migrator this month.
[Magisk] BigBrosMo:
Ok... Sorry still a little unclear. If I boot into twrp, it asks me for my pin and then decrypts and takes me to the TWRP home where I can flash etc. So... Does that mean I just need to do that for Migrator to work, or there needs to be no encryption whatsoever?
[Magisk] VR25:
Well, since /data is always decrypted when you start flashing stuff, you don't have to worry about this. In other words, Migrator will work whether or not you use encryption.
[Magisk] andrizmitnick:
does the new update had already support for preservation of runtime permissions?

thanks before
[Magisk] VR25:
No, this is work in progress. I'm prioritizing optimizations/multithreading and bug fixes, though.

[MENTION=4717631]shouk_1987[/MENTION], regarding contacts not being restored, did you migrate to a different Android version or just to another ROM (same Android version)?
[Magisk] shouk_1987:
Actually I am not sure about contacts as I cannot recall correctly

I am now back to previous rom with the twrp backup I took

I will wait your module.to be working fine before another trial

I changed only.rom with same android version

Both are pie aosp
[Magisk] Walden0:
I have a question, if one wants to switch rom and use this tool, should he wipe data, system etc partitions and flash rom as usual, or it wiping will wipe backup also?
[Magisk] VR25:
**2019.2.3 (201902030)**

- Added [Telegram group link](https://t.me/migrator_magisk/).

- Customizable multithreading for apps+data migration.

- Do not retry app restore after the 3rd failed attempt. This can be overridden. Refer to README.md for details.

- General fixes & optimizations

- Reverted "do not preserve system settings by default" (false alarm).

- Show package names during apps+data migration.

- Updated documentation and config.

- Wait until pm (package manager) is ready before initiating app restore.

I just added this to the README.md (first post):

I'm looking for alternatives. Meanwhile, you can export your contacts to contacts.vcf.

The "apps not being restored (random failure)" issue should be fixed. I tested that multiples times under the new build and it worked consistently. Let me know how it goes for you.

/data and /cache are wiped automatically. You only have to wipe /system -- but that's usually unnecessary, since most ROMs do it already.
[Magisk] Walden0:
Oh, thank you, i didn't know that.?
[Magisk] Gusti Putu Suarmayasa:
This is just like the app data keeper that he make before,too good to to be true,and i guarantee it will break the system if you dont fully understand how this module work,it will break your system after installing this module,when you install this module and you reboot your phone, "BOOM" your phone break ,so don't instal if you dont have backup

Sorry for my bad English and if i saying something stupid

"Like i said if you dont understand like me and who ever that have been use this module and got trouble with their device,just dont install,im not saying all phone will break,but for people who don't understand this module,this module is very complex maybe other senior could tutoring the juniors about how to use this module " when i said it will guarantee it will break your phone it's just my stupidity and my anger because this type of module have fail me twice first app data keeper (adk) and this one,and it made by the same person, to VR25 sorry bro,i dont say your module is bad for everyone, it's just fail me twice sorry
[Magisk] goldencore:
Yes, you are saying something stupid!

RTFM!

This module is *kinda* complex and if you don't understand its use, then don't use it.

Dev answers all questions here with great patience!
[Magisk] PokemonTotalWar:
While RTFM is some of the better advice out there, he does have a point in that you shouldn't install ANY modification, especially one like this, without a backup first, even if you think you know how to reverse whatever you did.
[Magisk] goldencore:
Yeah, sure! But he said "I guarantee this will break your system." and that is just nonsense.
[Magisk] godofsalt:
No this will not break your system.

Great Work VR25!!! .... Very nice update too!!

As we last spoke.... this might change the whole game, (when doing or having Back ups)

AND... true it is faster than TB, so far for now...

Tanx