[Android Software/Hacking General [Developers Only]] scandiun:
What is an android adb backup?

An adb backup is a file with ab extension, generated by android's backup manager when we request it via adb shell. This allows you to backup some data of the phone, but is not a replacement of a clockworkmod backup:

- Java 7 or higher is required because of SYNC_FLUSH mode for the Deflater.

- If the backup contains apk+data for an app, restore will work. If contains only the data, you must install the app first on the device and then restore. Installing the app later won't work.

- Some apps include a policy where the apk is never backed up even if specified.

- Inside an ab file is a tar file, which contains files and folders in a certain order. You have you respect that order, which is not necessarily alphabetical like tar does by default. Such order is listed in "Full local backup infrastructure".

- Inside the tar file, directories must not have trailing slash, for that reason pax, star or equivalent has to be used.

- There are some bugs present in the android source code.

- 'adb backup' or 'adb restore' are pretty slow, between 1 and 2.6 MBps.

- The adb backups (usually with .ab extension) can be password protected or not. If the device is encrypted this is a must and has to be the same used for the device.

- Star is recommended instead of pax, which supports path length only up to 100 characters, so will fail in some cases. This usually happens with browsers history.

- ADB backups are not a replacement for a Nandroid backup. The whole /data partition is not backed up, only a part of it. Also, other partitions like /system, /preload, /cache, modem, RIL, efs, kernel or recovery are never backed up because are not user data. This is to prevent issue when restoring on a different device. It has also the biggest

- If you have the device encrypted with a password, you must use that particular one for backup creation and restore. You can't create a backup without password or a different one in that particular case, or if you try to restore will fail.

- star for cygwin for windows is attached (move it to C:\cygwin\bin\star.exe) since there is no package available. It can be compiled from the schily (ftp://ftp.berlios.de/pub/schily/).

- The best way to test if an adb backup has errors, is to convert it to tar and then check.advantage that doesn't require root to operate, so is totally compatible with stock roms, locked bootloaders and device encryption.

- There's also a bash script called adb-split.sh that creates individual adb backups for each recognized app from the full one, so you can restore apps individually. Same encryption is preserved to them, if any.

You can use java or perl, although with perl can be more complicated because requires downloading some modules from cpan and some ssl headers.

To know more information about types of android backups:

[Guide] Types of Android backups

Software needed

The entire process can be done from Windows, but is better to use a unix-like operating system, like Linux, OS X or BSD, since we should extract the files on a filesystem that preservers file permissions and owners and repack the tar archive. Furthermore some tools like star are easier to get for linux. In such cases, using Virtualbox or VMWare Workstation is highly recommended. Using Windows should work in any case whatsoever.

Cygwin

Preferably 32 bits: has more packages and is less prone to errors. Furthermore, the star for cygwin attached is 32 bits.

Java 7 or higher

Java 7 or OpenJDK 7 (more builds here)

Oracle Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 7 if you are going to work with password encrypted backups.

You need to install the files local_policy.jar and US_export_policy.jar under jre's lib/security folder, for example:

- For Windows:

C:\Program Files\Java\jdk1.7.0_09\jre\lib\security\

C:\Program Files\Java\jre7\lib\security\

C:\Program Files (x86)\Java\jdk1.7.0_07\jre\lib\security\

C:\Program Files (x86)\Java\jre7\lib\security\

- For Linux or BSD:

/usr/local/jdk1.7/jre/lib/security/

/usr/lib/jvm/java-7-openjdk-*/jre/lib/security/

/usr/local/openjdk7/jre/lib/security/

- For OS X:

/Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home/jre/lib/security/

Perl

Perl is available for several operating systems

libssl or openssl headers. If you are using Linux or Cygwin is much more easier

cpan modules required by use functions

pax or star

pax is an archiving utility that compresses in ustar by default. In this case is extremely useful because stores directories without trailing slash. Is available for all operating systems, and in windows can be installed via cygwin's setup.exe or download a native version like gnuwin32 (sometimes available via bsdtar with pax option). pax doesn't work when paths are more than 100 characters lenght, so I recommend star instead.

star allows storing directories without trailing slash. You can get the ubuntu version 1.5 here. For Windows you can use Cygwin's version included.

Android Backup Extractor

Android Backup Extractor is the java application that does all the job. It includes the perl scripts.

How it works, better with an example

I will use an example to demonstrate how it works, with java version. If you like to use perl, just grab the perl scripts and is nearly the same.

This is extracted from the readme file, and will extract a whole adb backup and repack only the data for the game Grand Theft Auto III for android.

1) Convert the original adb backup to tar format:

java -jar abe.jar unpack nexus7.ab nexus7.tar <password>

2) Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:

tar -xvf nexus7.tar

3) Make a list of all the contents of the original archive in the order they are archived:

tar -tf nexus7.tar > nexus7.list

4) Create a new list only with the files and folders you want, in proper order. For example for the GTA 3 (you can try savegames instead of all data):

cat nexus7.list | grep com.rockstar.gta3 > gta3.list

5) Create the new tar archive. The directories stored on tar shouldn't contain trailing slashes, so I use pax or star instead of tar. Pax works also if paths are up to 100 of lenght:

cat gta.list | pax -wd > gta3.tar

OR

star -c -v -f gta3.tar -no-dirslash list=gta3.list

6) Create the adb backup from the tar archive. Password is optional:

java -jar abe.jar pack gta3.tar gta3.ab <password>

Note: if the backup is not encrypted zlib can be used instead for both unpack and pack the ab archive:

- Quick unpacking:

dd if=nexus7.ab bs=24 skip=1 | openssl zlib -d > nexus7.tar

- Quick packing:

dd if=nexus7.ab bs=24 count=1 of=gta3.ab ; openssl zlib -in gta3.tar >> gta3.ab

Split an Android backup

Use Android Backup Splitter (is a shell script). Works on Cygwin too:

sh adb-split.sh backup.ab [password if needed]

Resulting files go in app-ab folder

# ADB Backup Splitter:

#

# This bash script utility generates an adb backup for each item

# on apps/ folder. That equals to generating an adb backup for

# each app.

#

# shared/0 and shared/1 (sdcards) are ignored, so that means that

# external data and OBB files, although may be backed up, may not

# be restored. For doing so backup the folder Android of the sdcards.

# Creating a backup with -shared flag has know issues (corruption)

# so is not recommended. Future releases of this script may add

# the option to automatically detect things from sdcards inside

# the adb backup file.

#

# This script works on Cygwin also, and in general where star does.

#

# There may be apps in the backup that don't include the apk. In that

# particular case, the data won't be restored, so the individual backup

# is generated for testing purposes only. Installing the apk afterwards won't work.

# There is generated an html file called apk-missing.html so you can open it and

# install all applications from Play Store on any of your devices.

Patch apps to allow adb backup

Some apps specify in the AndroidManifest.xml android:allowBackup="false" so it won't be backed up by adb backup. If you have root and use Titanium Backup or CWM or TWRP they will do the job, but if you don't want root the only solution is to modify the apk to edit that string. Read here how to do that and the pros and cons:

[GUIDE] How to enable adb backup for any app changing android:allowBackup

Links

Android Backup Splitter (@ Sourceforge)

Android Backup Extractor (@ Sourceforge)

Android Backup Extractor (@ GitHub)

Documentation at Blogspot

Full local backup infrastructure

[Guide] Types of Android backups

Backing Up Android Games

Titanium Backup Decrypter

Perl scripts to encrypt/decrypt adb backup files

[GUIDE] Full Phone Backup without Unlock or Root

[GUIDE] Internal Memory Data Recovery - Yes We Can!

[Q] [adb] backup not working for Temple Run

Related adb backup and restore bugs:

Issue 55860: adb backup skips the apk for certain apps and doesn't prompt for download when restoring

Issue 48126: device adb connecting to localhost adbd cannot execute commands anymore since 4.2.2 (error device offline)

Issue 3254: adb shell doesn't return exit code of program

Issue 53065: Debugging via ADB stopping the onClick action on ImageView widget

Issue 53668: ADB : An existing connection was forcibly closed by the remote host

Issue 54781: adb install: single quotes in filenames cause quoting issues

Issue 55173: (Game) Offroad Legends does not detect obb when restored via adb restore

Issue 55178: (Game) Carmageddon Promo does not detect obb when restored via adb restore

Issue 28303: adb backup doesn't respect -noshared flag

Issue 32830: adb restore errors not displayed on device

Issue 34311: Galaxy Nexus gets stuck when restoring adb backup

Issue 25780:	BackupManager causes reboot when BackupAgent missing

Issue 40691: ab backup and restore Ski Safari does not restore saved data

Issue 16286: Restoration of phone not working properly

Issue 39014: Nexus 7 adb restore freezes when restoring udk.android.reader

Warning about Helium (Carbon): helium backups are just android adb backups that are not compressed with Java Deflater. It means they were created with the option static final boolean COMPRESS_FULL_BACKUPS = false.

A normal adb restore should work.