Repository: OfflineIMAP/offlineimap

Issue #384: maxage is respected only once
Created: 2016-10-14T12:54:44Z
General informations

system/distribution (with version): OSX Yosemite 10.10.5 (14F1912)
offlineimap version (offlineimap -V): homebrew offlineimap v7.0.7, imaplib2 v2.55 (bundled), Python v2.7.10

Configuration file offlineimaprc
ssl = yes
maxage = 100
status_backend = sqlite
type = GmailMaildir

Steps to reproduce the error

Empty maildir
sync -> mails newer than 100 days are synced normally
change maxage to 200
sync -> no new mails (there should be)
remove maxage from config
sync -> all mails coming... (17000)
interrupt
put maxage back
sync
all mails still coming, no way to get offlineimap respect maxage again...

Duplicate issue #581: Can't copy external mail older than maxage to local maildir (related to 681e27)
Created: 2018-11-01T22:48:24Z
General information

system/distribution (with version): Arch linux
offlineimap version (offlineimap -V): 7.2.1
Python version: 3.7.1
server name or domain: gmail
CLI options: -o

Configuration file offlineimaprc
[general]
accounts = test
fsync = False

[Account test]
localrepository = Local-test
remoterepository = Remote-test
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
maxage=5

[Repository Remote-test]
type = IMAP
maxconnections = 3
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
remoteuser = <...>
remotehost = imap.gmail.com
remoteport = 993
ssl = yes
ssl_version = tls1_2
trashfolder = '[Gmail].Trash'
remotepass = <...>
folderfilter = lambda folder: folder in ['INBOX']

[Repository Local-test]
type = Maildir
localfolders = <...>
status_backend = plain
trashfolder = '[Gmail].Trash'

Steps to reproduce the error

Copy an email that's older than maxage in another account, to the local maildir tracked by offlineimap
Run offlineimap once; this copies the new email to the remote account
Run offlineimap again; this deletes the local copy (which I don't want) and keeps the remote copy.

Desired behavior: offlineimap shouldn't be looking at this email at all, because it's older than maxage.
If I downgrade to 7.1.4, this behavior still happens, and if I revert commit 681e27 there, the problem goes away (nothing is copied, and nothing is deleted). I believe this commit is at fault in the more recent versions, but it seems something else has changed around it, and I can't revert cleanly.
Log messages
Running with -d imap,maildir,thread I get
Folder INBOX [acc: test]:
 [thread]: Register new thread 'Folder INBOX [acc: test]' (account 'test')
<...>
Account sync test:
Folder INBOX [acc: test]:
 Syncing INBOX: IMAP -> Maildir
 Loading message list for IMAP[INBOX]
 [imap]: calling imaplib2 fetch command: '5:9,11:12' (FLAGS UID INTERNALDATE)
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:23 +0000"', 'FLAGS': '(Old)', 'UID': '6500'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:26 +0000"', 'FLAGS': '(Old)', 'UID': '6501'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:29 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6502'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:30 +0000"', 'FLAGS': '(Old)', 'UID': '6503'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 22:03:50 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6505'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:57:36 +0000"', 'FLAGS': '(Old)', 'UID': '6507'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 20:13:30 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6508'}
 Message list for IMAP[INBOX] loaded: 7 messages
 Loading message list for Maildir[INBOX]
 Message list for Maildir[INBOX] loaded: 8 messages
 Syncing messages IMAP[INBOX] -> Maildir[INBOX]
 Syncing messages Maildir[INBOX] -> IMAP[INBOX]
 Copy message UID -1 (1/1) Local-test:INBOX -> Remote-test:INBOX
 [imap]: Write mail 'INBOX:-1' with flags set(['S'])
 []: deletemessageheaders: called to delete []
 []: getmessageheader: called to get message-id
 []: getmessageheader: eoh = 5051
 []: getmessageheader: headers = '<...>'
 [imap]: savemessage: date: "21-Aug-2010 00:04:35 -0500", content: <...>

on the first run, and
Folder INBOX [acc: test]:
 Syncing INBOX: IMAP -> Maildir
 Loading message list for IMAP[INBOX]
 [imap]: calling imaplib2 fetch command: '5:9,11:12' (FLAGS UID INTERNALDATE)
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:23 +0000"', 'FLAGS': '(Old)', 'UID': '6500'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:26 +0000"', 'FLAGS': '(Old)', 'UID': '6501'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:29 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6502'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:56:30 +0000"', 'FLAGS': '(Old)', 'UID': '6503'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 22:03:50 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6505'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 21:57:36 +0000"', 'FLAGS': '(Old)', 'UID': '6507'}
 [imap]: __options2hash returning: {'INTERNALDATE': '"01-Nov-2018 20:13:30 +0000"', 'FLAGS': '(\\Seen)', 'UID': '6508'}
 Message list for IMAP[INBOX] loaded: 7 messages
 Loading message list for Maildir[INBOX]
 Message list for Maildir[INBOX] loaded: 8 messages
 Syncing messages IMAP[INBOX] -> Maildir[INBOX]
 Deleting 1 messages (6515) in Maildir[INBOX]
 Syncing messages Maildir[INBOX] -> IMAP[INBOX]
[thread]: Unregister thread 'Folder INBOX [acc: test]'

on the second run. Note that 21-Aug-2010 00:04:35 -0500 is the date of the message copied.

Closing comment (2018-12-03T21:05:55Z): Your tests look right. I wonder we should revert this f37b97c commit because correctly copying emails between folders is more important than changing maxage as reported in #384. In this case I think there will be no other choice than enabling filename_use_mail_timestamp when maxage is set.