yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] logge22:
database disk image is malformed

Hello,

i've installed all programms(Whatsapp_extract_v2.2 ,...) and it works fine with a new msgstore.db.crypt - File.

But when I'm trying to "use" an older and much larger msgstore.db.crypt - File (7MB) I always get the error: "database disk image is malformed" :(

When i use the plain.db - Version with SQLite Database Browser 2.0 i can see the messages.

Can someone help me pls?
[Android Apps and Games] advance512:
How do you expect anyone to help you without all of the information?

What is this "old" file? What version of WhatsApp is it from? Is there any chance it got corrupted? Do you have a backup? Better yet - is the "plain.db" file a decrypted version of this encrypted file? If so, why do you need to decrypt said file?

Please read the actual instruction to see you can open a decrypted file with the script, just as you can open an encrypted file.
[Android Apps and Games] hape70:
hi advance512

nice enhancement :good:

I have a problem when opening the html file on another machine. The full original path is written in the html file (which dows not exists on the second machine)
[Android Apps and Games] advance512:
Thanks. I actually added other enhancements - a Media file to see all photos quickly, but I am too lazy to release it as I did it in a sort of ugly way (code-wise).

As far as your problem, I would have to see a specific file to understand. The code generates HTML files which refer to relative files names, all based on the input filename, the command line argument. I assume you input a complete file path as a command line argument? If so, try and just give the filename.

Otherwise, I'd have to understand more about your process.
[Android Apps and Games] logge22:
The msgstore.plain.db-File was generated during the process using Whatsapp_Extract like you can see in the picture. Also you can see that the file is from December 2012, i don't know exactly the Whatsapp-version I used at this time.

The plain.db-File is decrypted but I only can use it with sqlite-manager but not with Whatsapp_Extract. And no I don't have a backup this is the only file I have for this chat-history i want to get back. And yes it is possible that the file got courrupted because i have had several problems tranferring my data from the sd-card to my computer.

Hope this is enough information and you can help me? Otherwise i can send you the file if you have time to have a short look at it.
[Android Apps and Games] advance512:
Well, the error you showed me comes from the SQLite library.

You probably have a bit of corruption in the decrypted file - even 1 bit off in the encrypted file could affect large parts of the decrypted file. Anyways, the fact that you can see your info (or some of it) in SQLite Manager is great. This is your solution to save what remains.

Open the database, and export the data into an SQL file.

Then, create a NEW database, and import the data from that SQL file. Save the new database.

Now, feed the new database into WhatsApp_Xtract using the "decrypted database" option found in the OP.

It should work.

Let me know how it goes. :)
[Android Apps and Games] logge22:
I created a new database like u told me (the imported sql-file had about 6 mb, the new created file had about 4 mb) and called it msgstore.db and pasted it into the whatsapp_xtract folder like you can see on the image. then i ran whatsapp_xtract_android but it didn't work, it also didn't work when i pasted the file into whatsapp_xtract_drag_and_drop_databse(s), where the following error message occured.

Sorry but I really tried a lot of things (and wasted a lot of time) but it doesn't work... :( Do you think you could manage it when i would send you the file?
[Android Apps and Games] advance512:
Okay, post it somewhere so I could download it, I'll see what I can do.. though I am not promising anything.
[Android Apps and Games] logge22:
Thanks a lot! I've sent you a private message with a link.
[Android Apps and Games] hape70:
your're right, when entering the command manually all is fine.

When I drag and drop the db-file to "whatsapp_xtract_drag'n'drop_database(s)_here.bat" the full path will be submitted.

Thanks for your help!

Update:

I changed the lines in whatsapp_xtract_drag'n'drop_database(s)_here.bat

from

python "%~dp0whatsapp_xtract.py" %1

to

python "%~dp0whatsapp_xtract.py" %~nx1

now it's ok
[Android Apps and Games] pippo81:
Thank You Fabio :D !

It works perfectly for restoring my backups (cambiero' 3-4 rom al mese, ma il fatto che non mi ripristinasse la chat non era mai capitato..)
[Android Apps and Games] Snapy1991:
Whatsapp Backup gefunden,aber nicht wiederhergestellt

Hallo ,

Ich habe folgendes Problem, und zwar kann ich meine Backup Datei nicht mehr auf dem Handy wiederherstellen.Ich habe mein Handy formatiert und habe aber vorher den Whatsapp Ordner von der SD Karte auf den PC kopiert.Darin befindet sich im Ordner Databases die Datei msgstore.db.crypt.

Ich habe dann die Anleitung befolgt und es klappte alles bis zum Fehler :

Error sqlite3.Error while reading Chat #: database disk image is malformed

So habe ich dann deinen Rat befolgt :

- start whatsapp_xtract_console.bat and then type:

- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql

- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db

- python whatsapp_xtract.py msgstore_new.db

und konnte am Ende die Datei auf dem PC anzeigen und hatte die Verläufe zumindest auf dem PC.

Nun meine Frage: Kann ich nun irgendwie die msgstore.db.crypt Datei so reparieren und auf dem Handy wieder kopieren, damit Whatsapp auch die Nachrichten tatsächlich wiederherstellt oder kann ich sie nur maximal auf dem PC anzeigen lassen?Kannst du mir bitte sagen,welche Datei ich wohin genau verschieben soll?Denn es heißt ja im Normalfall,wenn es klappt die Datei msgstore.plain.db in den Whatsapp Ordner auf dem Handy kopieren und in msgstore.db umbenennen,jedoch wenn ich

whatsapp_xtract_android_crypted.bat

ausführe, entsteht ja der oben in rot angezeigte Fehler und die msgstore.plain.db Datei ensteht auch mit genau der gleichen Dateigröße wie die msgstore.db.crypt Datei.Nun als ich Whatsapp wieder installiert habe, fragte es mich,ob ich den Verlauf wiederherstellen will,ich bestätigte und es zeigte an,dass es Nachrichten wiederherstellt,jedoch waren keine Verläufe da und alles war weg.

Ich hoffe ihr habt verstanden was ich meine und hoffe, dass ihr mir helfen könnt, das wäre echt lieb.

Noch ein kleiner Nachtrag,wenn ich diese Befehle ausführe

- start whatsapp_xtract_console.bat and then type:

- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql

- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db

ensteht ja die msgstore_new.db Datei und die ist nur minimal kleiner als die msgstore.db oder msgstore.plain.db Datei und die ist ja dann die reparierete Datei oder? Und durch sie kann ich den Inhalt auf dem PC sehen und so erhalte ich ja auch die msgstore.db als HTML Datei. Müsste ich diese Datei umbennen und irgendwohin kopieren oder was muss ich jetzt genau machen,damit Whatsapp die Backup Datei wirklich richtig erkennt und die Nachrichten tatsächlich wiederherstellt?

Zur Zeit ist Whatsapp installiert,müsste ich es deinstallieren und vorher noch irgendwas sichern oder kopieren?

Vielen lieben Dank im Vorraus
[Android Apps and Games] Labersack:
This is an international forum.

I understand german, but me (and maybe some others) will not answer you, if you don't even try to write english.

If everyone posts in his native language, this thread will soon be a mess.

Get help by some friend (good) or try the google translator (worse)

Dieses Forum ist international

Ich verstehe deutsch, aber ich (und evtl. einige Andere) werden nicht antworten, wenn du nichmal versuchst, englisch zu schreiben.

Wenn jeder hier in seiner eigenen Sprache schreibt, wird dieser Thread ein Chaos.

Lass dir von einem Freund helfen (gut) oder versuche den Google Translator (schlechter).
[Android Apps and Games] abhi.v:
hi all i face the same problem and here is my screenshot for the same :
[Android Apps and Games] LizTheCat:
I am a nativ german speaker and have exactly the same problem that Snappy has, so I will explain it:

I have 9 older backup files with dates (msgstore-2013-02-22.1.db.crypt) (Snappy has just msgstore.db.crypt). My phone (HTC Sensation) was in repair for a couple of weeks and I now tried to restore the WhatsApp Backup. It all worked fine till we got the following error: Error sqlite3.Error while reading Chat #13: database disk image is malformed

I tried it with all of my backup files and got the same error with the newest 5 of them - the others where not readable by the program at all. We both got msgstore.plain.db files but they were just about the same size as the crypt ones and although WhatsApp was able to find the databases it was not able to restore the chats. In my case I had all the groups and contacts but not a single message - and the plain file was about 11 MB, so this can't really be. Please help - I spent half a day now trying to solve the problem and I have no clue anymore... :p

Snappy tried all your advices and got the readable HTML file but this still didn't solve the problem - obviously - and he or she is mainly asking how to repair the plain file and where to save it so the backup would be restored and working.

Thank you very, very much for your help and this thread - I really, really hope you can help me/us!

Liz
[Android Apps and Games] Snapy1991:
if you don't speak or understand german i will try to text you in englisch so that you can help me because i need my chat.

Liz has translated my text and i want to give you a detailed description of my problem and the way how i tried to solve it so you can understand and follow my problem.

I have tried to restore my chat but Whatsapp didn't restore anything allthough the question appeared during the installation if i want to restore the chat and i accepted.

I have just the Whatsapp folder from sd card where the file msgstore.db.crypt is in the databases. I copied this file and overwrote the file in the xtract whatsapp folder.Than i started the !install pyCrypto file and executed the whatsapp_xtract_android_crypted file but the following error appeared:

Error sqlite3.Error while reading Chat #: database disk image is malformed

and i got the msgstore.plain db file which has exactly the same size like the msgstore.db.crypt file (5,439 KB)

Than i copied the msgstore.plain db file again to the folder sdcard/Whatsapp/databases and renamed it to msgstore.db but Whatsapp didn't restore anything allthough it showed me that it seems to restore something during the installation.

So i did this to repair the database:

- start whatsapp_xtract_console.bat and then type:

- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql

- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db

- python whatsapp_xtract.py msgstore_new.db

Now i got the msgstore_new db file which has just a little bit smaller size than the msgstore.db.crypt and msgstore.plain.db file (5,291 KB) and it seemed to be correct because i got also the HTML file msgstore.db file which showed me all my chat in firefox.

My question now: Which file do i have to copy to my sd card so i get my chat back and so Whatsapp can restore them correctly?

I would be very grateful for your help

ps. Liz thank you for your help and the translation :)
[Android Apps and Games] LizTheCat:
That is where my problem differs from Snappy's: When I work with whatsapp_xtract_console.bat, first everything looks normal (tryubg to repair android database... trying to decrypt android database... decrypted database written to msgstorenew.db.plain.db trying to repair decrypted andorid database...) but then I get an error telling me: "Could not open database file. Guess it's not a valid Android or Iphone database file." The file msgstore_new.db is created but with a size of 0 KB. I really have no clue what to do there...
[Android Apps and Games] Sethide:
Is it the other way around?

I mean it is Android -> Iphone where there is no solution yet, am I right?

Thank you.
[Android Apps and Games] advance512:
Hi there,

Just wanted to post my "unofficial" (as in - I needed extra features for my own usage of this tool, so I coded them, but I am not a part of the official dev team) version 2.5 of this tool.

This version can now automatically handle malformed databases (that received the "database disk image is malformed" error).

--------------

CHANGELOG:

v2.5 (updated by Alon Diamant - Mar 14, 2013)

- Improved encrypted Android database detection and decryption code

- Can now repair malformed Android databases (depends on availability of sqlite3 executable)

v2.4 (updated by Alon Diamant - Mar 06, 2013)

- Generates media index file - but crappily, we should set this up better..

v2.3 (updated by Alon Diamant - Mar 05, 2013)

- now generates separate file for each contact

- fixed file search to search for image files in days prior to date given (to fix a bug where because of timezone differences the image file exists but is not found)

- fixed message counts for contacts

- does not list contacts with 0 messages

- now writes version number in generated files

- (Android Version) displays WhatsApp name (server based) if no display name is found for a contact

- (Android Version) Supports Python 2.6

--------------

How to use:

After doing step 2 of the original post, and before doing step 3 - extract attached archive (Whatsapp_Xtract_Py_2_5.zip) to the same folder selected in step 2. Continue to step 3 normally.

Good luck :)

Make sure you click the "Thanks" button beneath this post if you like this!

--------------

Original poster: feel free to merge this into your code base, if you feel like it, or contact me to take it down, if you don't like it. :)
[Android Apps and Games] LizTheCat:
Hello advance512

I tried your tool exactly how you wrote to, hoping it would solve my problem. But unfortunately it didn't work at all and I get the error attached when trying to use whatsapp_xtract_android_crypted.bat after having unpacked your tool.

Could you please help?

Thank you!

Liz
[Android Apps and Games] advance512:
Okay, so a quick fix to 2.5.

I forgot test it on Python 3 (silly me programming late at night), but now I did.

--------------

CHANGELOG:

v2.51 (updated by Alon Diamant - Mar 15, 2013)

- v2.5 was now tested with Python 3 as well. :S

v2.5 (updated by Alon Diamant - Mar 14, 2013)

- Improved encrypted Android database detection and decryption code

- Can now repair malformed Android databases (depends on availability of sqlite3 executable)

v2.4 (updated by Alon Diamant - Mar 06, 2013)

- Generates media index file - but crappily, we should set this up better..

v2.3 (updated by Alon Diamant - Mar 05, 2013)

- now generates separate file for each contact

- fixed file search to search for image files in days prior to date given (to fix a bug where because of timezone differences the image file exists but is not found)

- fixed message counts for contacts

- does not list contacts with 0 messages

- now writes version number in generated files

- (Android Version) displays WhatsApp name (server based) if no display name is found for a contact

- (Android Version) Supports Python 2.6

--------------

How to use:

After doing step 2 of the original post, and before doing step 3 - extract attached archive (Whatsapp_Xtract_Py_2_51.zip) to the same folder selected in step 2. Continue to step 3 normally.

Good luck :)

Make sure you click the "Thanks" button beneath this post if you like this!

--------------

Original poster: feel free to merge this into your code base, if you feel like it, or contact me to take it down, if you don't like it. :)
[Android Apps and Games] Blinkin89:
First I thought that I wouldn't like the unofficial future development because of the separated contacts in html files...

But I've tried it and I must admit: well done. It looks pretty good now. Especially now the loading time in my browser is way lower.

I'm using your unofficial version now and I it's a pretty good addition. Great work!
[Android Apps and Games] ggclanlord:
Downloading unofficial v2.5 now.

@OP, if at all you check this thread now - Are you going to update the tool now? Or there's going to be no more support for it?
[Android Apps and Games] allforfriend:
HI,,

i have two problem in that.

can u solve my problem ??

I attach my screen capture.

Thank you very much=D:fingers-crossed:
[Android Apps and Games] advance512:
Try using the latest version (v2.51 posted here). The previous versions' had problematic malformed database repair code, and did not detect bad/corrupted WhatsApp database files.
[Android Apps and Games] allforfriend:
Where i can find the v2.51 link ???

can you show me or post it to me

I just find the new version is Whatsapp_Xtract_V2.2_2012-11-17.zip

Or Activepython-3.2.2.3?? 2.7.2.5?
[Android Apps and Games] coolhof:
You didn't look richt.......

If you scroll the page up you see the link.

Verzonden door mijn GT-I9300 met xda premium
[Android Apps and Games] allforfriend:
Can you tell me which tools i need to update?

1.WhatsApp Xtract

2.ActivePython

Can you provide the updated tools link in there?

I Already try 3.2.2.3 and 2.7.2.5 activestate

But still can't produce the html file >.<...

and the can't using sql browser read the file.

the Whatsapp_Xtract_V2.2_2012-11-17.zip already is updated version.

What can u do? what is my mistake ???

Thank you very ,much
[Android Apps and Games] Blinkin89:
He means that you should download Whatsapp Extractor 2.51.

If you did look one page back and scrolled down you would have seen it. ;)

http://xdaforums.com/showpost.php?p=39151141&postcount=518
[Android Apps and Games] allforfriend:
already installed...

but i still does not work-,-...
[Android Apps and Games] advance512:
You probably did not install it right. Please read pages 51, 52 and 53 of this thread and see my latest post and follow the steps there.

If this doesn't help, I'll soon post a "full" ZIP file with all of the WhatsApp Xtract solution, that will also write smarter instructions to users in cases of typical problems (no PyCrypto, for example) as well as (arghhhhhh) write the version number when you start it.

Cheers
[Android Apps and Games] allforfriend:
I already using v.2.51.... still cant using.....

Can you take a video show all step to me ??

All Put all steps clarify!!!!!

step by step...

And zip all file we needed.
[Android Apps and Games] CRXed:
Hi, could someone help me with this?

I installed everything correctly, including the 2.51 version, but I get this

Detected an SQLite database
Android mode!
Checking Android database integrity...
Android database has integrity issues. Attempting to fix it...
Traceback (most recent call last):
File "whatsapp_xtract.py", line 2678, in <module>
main(sys.argv[1:])
File "whatsapp_xtract.py", line 1549, in main
for line in f:
File "C:\Program Files (x86)\Python27\lib\codecs.py", line 296, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 1439: invali
d start byte

How can I fix this?

EDIT: It does work with version 2.2, 2.4 and 2.5, not 2.51
[Android Apps and Games] advance512:
I am learning how to properly code this with support for both Python 2.x and 3.x - it is hell, so it might not work everywhere.

Would you mind sending me the file that causes this crash?
[Android Apps and Games] advance512:
I have big plans for v3.0 of this tool.

To continue developing and to keep support for iPhone databases, I need a few sample databases.

Could anyone be so kind as to send me such a sample database?

Please let me know in a private message.
[Android Apps and Games] fendymn:
How to use: :cool:

(instrucciones en español / deutsche Anleitung)

You need to copy the whatsapp database.

On Android, either get this file:

/sdcard/WhatsApp/Databases/msgstore.db.crypt

(crypted database on SD card, can be created by starting backup from whatsapp advanced settings: settings - more - Backup Chats)

or these files:

/data/data/com.whatsapp/databases/msgstore.db and wa.db

(for this you need root access. detailed instructions in the bottom of this file. the advantage is that the corresponding contact names of phone numbers will be displayed.)

On IPhone, get this file:

net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite

(You can use an Iphone Backup Tool to get the file, e.g. I-Twin or Iphone Backup Extractor. Make sure to create an unencrypted backup with Itunes, as these tools can't handle encrypted backups. Another possibility are forensic tools like UFED Physical Analyzer.)

Extract this archive (Whatsapp_Xtract....zip) to a certain folder on your computer, e.g. C:\WhatsApp.

Copy the database(s) to e.g. C:\WhatsApp (on Android, you simply copy the whole folder WhatsApp on SD card to your computer e.g. to C:\WhatsApp and then copy the database file from C:\WhatsApp\Databases to C:\WhatsApp)

You need Python and (for Android msgstore.db.crypt decryption) the PyCrypto library

The easiest way is to install ActivePython

!! on Windows choose 32bit/x86 version even if you have 64bit windows (if not you will get an "requires Business Edition subscription" error!):

http://www.activestate.com/activepython/downloads

It's available for Windows, Linux and Mac. However in my HowTo, I focus on Windows.

and then run !install pyCrypto.bat (contained in this archive. The best is to rightclick on it and choose "run as administrator".)

If you have troubles installing pyCrypto this way, then please try to install it using an installation package from this site (Thanks, andcha!): http://www.voidspace.org.uk/python/modules.shtml#pycrypto

Now run whatsapp_xtract_android.bat or whatsapp_xtract_android_crypted.bat or whatsapp_xtract_iphone.bat

OR simply drag and drop the database file(s) to whatsapp_xtract_drag'n'drop_database(s)_here.bat

OR whatsapp_xtract_console.bat and then manually specify the input file with one of these commands:

COMMAND LINE OPTIONS:

Once finished, your browser will open and show the chats.

The resulting file size of the .html file will be slightly bigger than the size of the .db database.

i succeed until #5. then what to do? just copy back all file in my pc into my whatsapp file in my phone?
[Android Apps and Games] Tr1gun:
hello

i tried this tool but in the screenshot of the first post shows the name of contacts.. in mine show only numbers....

another thing...

can u add a counter of msg sent inside of a group conversation?

like the "index" but for each conversation add the number of msg for each user...
[Android Apps and Games] coolhof:
From Op:

On Android, either get this file:

/sdcard/WhatsApp/Databases/msgstore.db.crypt

(crypted database on SD card, can be created by starting backup from whatsapp advanced settings: settings - more - Backup Chats)

or these files:

/data/data/com.whatsapp/databases/msgstore.db and wa.db

(for this you need root access. detailed instructions in the bottom of this file. the advantage is that the corresponding contact names of phone numbers will be displayed.)

Verzonden door mijn GT-I9300 met xda premium
[Android Apps and Games] Tr1gun:
and?

in that file u don't have the name for the contacts... u have the tel number...

on the screen under FROM colomn is writed

Giovanni

me

in mine is

345678345678

me

35569667487
[Android Apps and Games] Snapy1991:
Which file do i have to copy

First of all thank you for your work :

I have done this like you described and i got now a lot of msgstore.plain.db files with the numbers of the chat members and also the media

and i also got the msgstore.plain.db.repaired file which has now a smaller size than the msgstore.db and msgstore.db.crypt files.Do i have to copy this file (msgstore.plain.db.repaired) instead of the msgstore.plain.db file like described in the original post?

Thanks a lot
[Android Apps and Games] allforfriend:
Can u Zip the all file we needed and then provide to us.:good: