[Upgrading, Modifying and Unlocking] Vivalis:
Part 3: The solution and tools

Solution:

Based on my investigations and the findings above, I tried the following steps which finally worked and solved all my problems regarding the migration of messaging data. However, especially the MMS part is quite some work, so maybe someone has ideas on how to speed this up.

If you can't access your old device anymore, goto step 2. If you still can use your old device, you're luckier than I was and have a much easier process: Install PIM Backup on both your old and new device and create a text-mode based (deselect the 'Binary backup' option) backup of all your messages on the old device. Copy the following files to your storage card and goto step 5 after:

PIM Backup file

Folder "\My Documents\UAContents\" (dat-files with MMS content)

Folder "\Windows\Messaging\" (text and attachments of emails)

If you can't use your old device anymore, but you have a text-mode based PIM Backup file as well as the 2 folders mentioned in step 1 on your storage card, then you can skip the next steps and goto step 5.

If you have a backup of a backup software (like Sprite Backup), including the cemail.vol as well as the 2 folders mentioned in step 1, then do restore this backup on your new device now.

(Note: If you don't have access to your old device and no backup file, there's nothing to restore and you can start your life with your new device free of history :-). If you have a binary PIM Backup file, you can restore your SMS, but you won't be able to restore your MMS as described in this approach.)

Only if you have executed step 3: Install/start PIM Backup on your new device and create a text-mode based (deselect the 'Binary backup' option) backup of all your messages on your new device.

In WM 6.1 MMS are stored in the same account as SMS, so we need to alter the PIM Backup file a little to make sure that PIM Backup does not restore the MMS into a hidden account again:

Open the PIM Backup file like a ZIP file and extract the msgs_<date>.csm.

Open the csm file with any text editor.

To redirect your MMS in the MMS-Inbox to the Inbox of the SMS, replace all occurrences of the string "\\%MDF1";"MMS";"IPM.MMS1" by "\\MMS\\in";"SMS";"IPM.MMS".

To redirect your MMS in the MMS-Sent Items to the Sent Items of the SMS, replace all occurrences of the string "\\%MDF3";"MMS";"IPM.MMS1" by "\\MMS\\out";"SMS";"IPM.MMS".

(Note: If there are no occurrences of the above search strings, you can delete the "1" from "IPM.MMS1" and try again.)

Save the csm-file, place it back into the PIM Backup-ZIP-file and copy the backup-file to your storage card.

If your new device is not "fresh" and contains messages, then you need to delete the existing cemail.vol file in order to start with a fresh messaging-file that has no "legacy pollution" on your new device. As the file is locked during operation of the PPC, MortScript helps you to perform the deletion on startup:

Install MortScript on your new device.

Copy and extract the attached script (cemail.vol_backup-restore-delete.mscr in ZIP-file) to your computer. This script is based on hercule18's work (Thank you!). It didn't work on my device, so I fixed it and added two more options that we need for our purpose here.

The script backs up to and restores from the storage card. In the script, you find 3 occurrences of the word "Speicherkarte" which is the german translation of storage card. Open the script with any text editor and replace these 3 occurrences with the name of your storage card exactly as it shows up in the explorer on your new device.

Save the altered script, copy it to your new device and run it. Choose "Backup & Delete" or "Delete" (we won't need the old cemail.vol anymore, but maybe you feel better with a backup) and Reset Yes. When your device restarts, MortScript deletes the messaging file and WM replaces it with a fresh, empty one.

If you've manually copied the necessary directories with the MMS- and email-content (unlike step 3), you can copy these folders from your storage card to your device now:

Folder "\My Documents\UAContents\"

Folder "\Windows\Messaging\"

Now you are ready to start restoring all the messages. First you only restore the Inbox and Sent Items of SMS (text messages) in order to allow WM 6.1 the threading process (organizing received and sent items into conversation threads). Start PIM Backup, choose action "Restore" and load the text-mode based PIM Backup file that you've created before. Choose ONLY messages, and in the messages' options, only Inbox and Sent Items of the SMS/MMS-account. Go!

On completion of the restore, open Messaging (Start --> Messaging) and choose your SMS/MMS account. In the Inbox, you can watch WM now threading the messages. Give WM the time to complete this process until you feel that it's done and your device is reacting at normal speed again.

Check if TF3D shows the threaded messages if you click on a text message in the Messages tab or if you choose the Messages tab in a contact.

Start PIM Backup again, choose "Restore" and load your PIM Backup file again. Select ONLY messages, and in the messages' options, select all other folders of the SMS/MMS-account, EXCEPT Inbox and Sent Items. In the last dialogue, choose "Add messages..." Go!

If you want to restore email-accounts, you have to create this accounts in Messaging/Outlook first, because PIM Backup does not restore to accounts that don't exist. After creating the corresponding accounts, you can restart PIM Backup, "Restore", load your PIM Backup file and select all the folders of the email-accounts you want. Make sure you choose "Add..." instead of "Delete..." in the last dialogue!

Finally, we need to recreate the reference between the restored MMS and the MMS-dat-files in "\My Documents\UAContents\". This is how I did it:

Create a list of your dat-files with name, date and size, using the attached Excel-Sheet (MMS-Renaming.xls in ZIP-file). To populate the sheet with your data, you can access your storage card (where you have a copy of the above mentioned directory) with your computer and create a text-file by --> Start --> Execute --> "cmd" --> navigate to the drive and directory with your dat-files (e.g. E:\My Documents\UAContents\MMS UA\) --> "dir >list.txt". Copy the content of the list.txt into the xls-sheet and put the data into the right columns (yellow columns). Order by date (descending).

Connect your device to your computer via ActiveSync and open an explorer window showing the content of \My Documents\UAContents\MMS UA\ on your device (not on the storage card!).

Now you have to find out the new reference number that MMS Composer is looking for when you open an MMS. If you did the above steps (altering and restoring the PIM Backup file) correctly, you can find your MMS in the subfolders MMS\in and MMS\out in your SMS/MMS account. Navigate into these folders in Messaging/Outlook, order by date (descending) and do the following steps for each MMS:

Open the MMS (it doesn't show any content).

While leaving the MMS open, refresh the explorer window on your pc, showing the content of the MMS UA folder.

Now you will see that a new folder appeared - the name of this folder is the new reference number we are looking for.

Find the right row in the xls-sheet (same date and size as the open MMS) and note the new reference number in the green column (only number, no ".dat"-suffix).

When your xls-sheet is complete, you have a list of MortScript-commands in column F. These commands will rename your dat-files with the correct number so that MMS Composer will find its targets. Put the content of column F (without the title in row 1) into a normal text file and save it as "mms-renaming.mscr".

Close Messaging/Outlook.

Copy the mms-renaming.mscr to your device and start it. The script runs only a few milliseconds and does not inform you about completion.

You're DONE! When you open an MMS now, you should see its content as it used to be.

Mentioned tools for the solution:

PIM Backup (Freeware)

MortScript (Freeware)

Other tools mentioned:

Sprite Backup (Commercial)

Pocket dbExplorer (Commercial)