spaetz/offlineimap — Issue #8: Implement uidchange() method rather than saving and then deleting the mail...
Author: spaetz
Created: 2012-01-06 16:53:32+00:00
State: open

From: Sebastian Spaeth
To: Matthieu Rakotojaona , offlineimap-project@lists.alioth.debian.org
Subject: Re: Strange behavior when syncing two IMAP servers
In-Reply-To:
Fcc: /home/spaetz/mail/INBOX.Sent
--text follows this line--
<#part sign=pgpmime>
On Thu, 24 Nov 2011 03:05:56 +0100, Matthieu Rakotojaona wrote:
> Hello everyone !
>
> I am working on an IMAP server of my own. I'm trying to sync it with a
> GMail account, with a nametrans.
>
> System : Linux 3.1.1, OfflineIMAP v.6.3.4
>
> When I add a message to a mailbox on my side and want to sync
> (upstream), here is the behavior I see on my side, on the IMAP server:
>
>
> So, I have a few questions :
>
> * My custom server cannot append a message if it already exists. Why
> does it append exactly the same, to remove the first one ?
> All I can see from this is that the email should get a new UID, but I
> can't see the reason. And it doesn't serves my purpose =]
>
> * Where can I see the relevant parts (2, 3 ( I suppose OfflineIMAP
> verifies some things at that moment), and 4) in the code ? I couldn't
> find them.

Hi Matthieu,

interesting to see that sequence :-). First of all: offlineimap has
originally been designed for IMAP<->Maildir synchronization, and
IMAP<->IMAP has been bolted on later. And that shows, IMAP<->IMAP does
some suboptimal stuff.

> 1) SELECT, EXAMINE, FETCH FLAGS, SELECT, EXAMINE the mailbox

This would be getting a list of Mails & flags on that folder, not sure
why the send SELECT is there. this is the cachemessagelist() part in
folder.UIDMaps.py

Everything concerned with uploading the mail starts from folder.Base.py
in the copymessageto() function.

> 2) FETCH BODY.PEEK[] the new email

Well obviously reading in the mail via getmessage()

> 3) SELECT the mailbox
> 4) APPEND the exact same email
> 5) CHECK, SELECT the mailbox
> 6) DELETE (STORE +FLAGS \Deleted) the email (the one that was read,

THe above is clearly the suboptimal part, and this is what happens in
copymessageto():

# Got new UID, change the local uid.
#TODO: Maildir could do this with a rename rather than
#load/save/del operation, IMPLEMENT a changeuid()
#function or so.
self.savemessage(newuid, message, flags, rtime)
self.deletemessage(uid)

We save the message to record a new UID that we have received (this will
happen when the other side is an IMAP server) and then delete the message with
the old ID. In the case of a MappedIMAP server, all this serves to is to
record the UID mapping between the upstream and the local IMAP server.

Clearly this is less than optimal and as you can see, I have included
the TODO comment that we should simply implement
> not the one that was appended)
> 7) SELECT, EXPUNGE the mailbox