Package: lieer
LieerThis program can pull, and send, email and labels (and changes to labels) from
your GMail account and store them locally in a maildir with the labels
synchronized with anotmuchdatabase. The changes
to tags in the notmuch database may be pushed back remotely to your GMail
account.DisclaimerLieer will not and can not:Add or delete messages on your remote account (except syncing thetrashorspamlabel to messages, and those messages will eventually bedeleted)Modify messages other than their labelsWhile Lieer has been used to successfully synchronize millions of messages and tags by now, it comes withNO WARRANTIES.RequirementsPython 3notmuch >= 0.25python bindingsgoogle_api_python_client(sometimesgoogle-api-python-client)google_auth_oauthlibtqdm(optional - for progress bar)InstallationAfter cloning the repository Lieer can be installed through pip by using the commandpip install .UsageThis assumes your root mail folder is in~/.mailand that this folder isalreadyset up with notmuch.Make a directory for the lieer storage and state files (local repository).$cd~/.mail
$mkdiraccount.gmail
$cdaccount.gmail/All commands should be run from the local mail repository unless otherwise specified.Ignore the.jsonfiles in notmuch. Any tags listed innew.tagswill be added to newly pulled messages. Process tags on new messages directly after running gmi, or runnotmuch newto trigger thepost-newhook forinitial tagging. Thenew.tagsare not ignored by default if you do not remove them, but you can prevent custom tags from being pushed to the remote by using e.g.gmi set --ignore-tags-local new. In your notmuch config file (usually~/.notmuch-config):[new]
tags=new
ignore=/.*[.](json|lock|bak)$/Initialize the mail storage:$gmiinityour.email@gmail.comgmi initwill now open your browser and request limited access to your e-mail.The access token is stored in.credentials.gmailieer.jsonin the local mail repository. If you wish, you can specifyyour own api keythat should be used.You're now set up, and you can do the initial pull.Usegmi -horgmi command -hto get more usage information.Pullwill pull down all remote changes since last time, overwriting any local tag
changes of the affected messages.$gmipullthe first time you do this, or if a full synchronization is needed it will take longer. You can try to use the--resumeoption if you get stuck on getting the metadata and have to abort (this will cause local changes made in the interim to be ignored in the next push).Pushwill push up all changes since last push, conflicting changes will be ignored
unless-fis specified. these will be overwritten with the remote changes at
the nextpull.$gmipushNormal synchronization routine$cd~/.mail/account.gmail
$gmisyncThis effectively does apushfollowed by apull. Any conflicts detected
with the remote inpushwill not be pushed. After the nextpullhas been
run the conflicts should be resolved, overwriting the local changes with the
remote changes. You can force the local changes to overwrite the remote changes
by usingpush -f.Note: If changes are being made on the remote, on a message that is currently being synced withlieer, the changes may be overwritten or merged in weird ways.See below for morecaveats.SendingLieer may be used as a simple stand-in for thesendmailMTA. A typical configuration for a MUA send command might be:gmisend-t-C~/.mail/account.gmailLike the real sendmail program, the raw message is read fromstdin.Most sendmail implementations allow passing additional recipients in additional
arguments. However, the GMail API only supports the-t(--read-recipients) mode of
sendmail, without additional recipients.We try to support valid combinations from MUAs that make use of recipients
passed as arguments. Additional recipients are ignored, but validated. The
following combinations are OK:When-tis passed, we need to check for the CLI-passed recipients to be
equal or a subset of the ones passed in the headers.When-tis not passed, all header-passed recipients need to be provided in
the CLI as well.This avoids silently not sending mail to some recipients (pretending we did),
or sending mail to recipients we didn't want to send to again.One of the implication of-tis you have to keepBcc:header in your
message when passing it tosendmail. It is not enough to just put the
additional recipient on the command line. Formutt, this means settingwrite_bccoption.Lieer will try to associate the sent message with the existing thread if it has
anIn-Reply-Toheader. According to theGmail
APItheSubject:header must also match, but this does not seem to be necessary
(at least not where justRe:has been prepended).If the email address in theFrom:header does not match exactly the one of
your account, it seems like GMail resets the from to your accountaddressonly.Note that the following flags are ignored forsendmailcompatibility:-f(ignored, set envelopeFrom:yourself)-o(ignored)-i(always implied, not bothered by single.'s)There are instructions for using this in your email client (for example Emacs) in thewiki.SettingsLieer can be configured usinggmi set. Use without any options to get a list of the current settings as well as the current history ID and notmuch revision.Accountis the GMail account the repository is synced with. Configured during setup withgmi init.historyIdis the latest synced GMail revision. Anything since this ID will be fetched on the nextgmi pull(partial).lastmodis the latest synced Notmuch database revision. Anything changed after this revision will be pushed ongmi push.Timeoutis the timeout in seconds used for the HTTP connection to GMail.0means the forever or system error/timeout,whichever occurs first.File extensionis an optional argument to include the specified extension in local file names (e.g.,mbox) which can be useful for indexing them with third-party programs.Important:If you change this setting after synchronizing, the best case scenario is that all files will appear to not have being pulled down and will be re-downloaded (and duplicated with a different extension in the maildir). There might also be changes to tags. You should in theory be able to change it by renaming all files, but since this will update the lastmod you will get a check on all files.Drop non existing labelscan be used to silently ignore errors where GMail gives us a label identifier which is not associated with a label. SeeCaveats.Replace slash with dotis used to replace the sub-label separator (/) with a dot (.). I think this is easier to work with.Important: See note below onchanging this setting after initial sync.Ignore tags (local)can be used to specify a list of tags which should not be synced from local to remote (e.g.new). In addition to the user-configured tags these tags are ignored:'attachment', 'encrypted', 'signed', 'passed', 'replied', 'muted', 'mute', 'todo', 'Trash', 'voicemail'. Some are special tags in notmuch and some are unsupported by GMail. SeeCaveatsbelow for more explanations.Note:This setting expectstranslatedtags.Important: See note below onchanging this setting after initial sync.Ignore tags (remote)can be used to specify a list of tags (labels) which should not be synced from remote (GMail) to local. By default theCATEGORY_*typelabels which are mapped to the Personal/Promotions/etc tabs in the GMail interface are ignored. You can specify that no label should ignored by doing:gmi set --ignore-tags-remote "".Note:This setting expectsuntranslatedtags.Important: See note below onchanging this setting after initial sync.Local Trash Tag (local)can be used to set the local tag to which the remote GMail 'TRASH' label is translated.Important: See note below onchanging this setting after initial sync.Translation List Overlaycan be used to add or change entries in the translation mapping between local and remote tags. Argument is a comment-separated list with an even number of items. This is interpreted as a list of pairs of (remote, local), where each pair is added to the tag translation overwriting any existing translation for that tag if any. For example,--translation-list-overlay CATEGORY_FORUMS,my_forum_tagwill translate Google's CATEGORY_FORUMS tag to my_forum_tag.')Important: See note below onchanging this setting after initial sync.Changing ignored tags and translation after initial syncIf you change theignored tagsafter the initial sync this will not update already synced messages. This means that if a change is made locally on an already synced message the previously ignored remote labels may be deleted. Conversely, if a change occurs remotely on a message which previously which has local tags that were ignored before, these ignored tags may be deleted.The best way to deal with this is to do a full push or pull after having changed one of the settings.Do not change both--ignore-tags-locallyand--ignore-tags-remoteat the same time.Before changing either setting make sure you are fully synchronized. After changing e.g.--ignore-tags-remotedo first a dry-run and then a real run of fullgmi pull -f --dry-run. This will fetch the full tag list for all messages and overwrite the local tags of all your messages with the remote labels.When changing the opposite setting:--ignore-tags-local, do a full push (dry-run first):gmi push -f --dry-run.The same goes for the options--replace-slash-with-dotand--local-trash-tag. I prefer to dogmi pull -f --dry-runafter changing this option. This will overwrite the local tags with the remote labels.Translation between labels and tagsWe translate some of the GMail labels to other tags. The default map of labels to tags are:'INBOX':'inbox','SPAM':'spam','TRASH':'trash','UNREAD':'unread','STARRED':'flagged','IMPORTANT':'important','SENT':'sent','DRAFT':'draft','CHAT':'chat','CATEGORY_PERSONAL':'personal','CATEGORY_SOCIAL':'social','CATEGORY_PROMOTIONS':'promotions','CATEGORY_UPDATES':'updates','CATEGORY_FORUMS':'forums',The 'trash' local tag can be replaced using the--local-trash-tagoption.Using your own API keyLieer ships with an API key that is shared openly, this key shares API quota, butcannot be used to access dataunless access is gained to your privateaccess_tokenorrefresh_token.You can get anapi keyfor a CLI application to use for yourself. Store theclient_secret.jsonfile somewhere safe and specify it togmi auth -c. You can do this on a repository that is already initialized, possibly using-fto force reauthorizing with the new client secrets.Privacy policyLieer downloads e-mail and labels to your local computer. No data is sent elsewhere.Lieers use and transfer to any other app of information received from Google
APIs will adhere toGoogle API Services User Data Policy,
including the Limited Use requirementsCaveatsThe GMail API does not let you syncmutedmessages. Untilthis Google
bugis fixed, themuteandmutedtags are not synchronized with the remote.Thetodoandvoicemaillabels seem to be reserved and will be ignored.Thedraftandsentlabels are read only: They are synced from GMail to local notmuch tags, but not back (if you change them via notmuch).Only one of the tagsinbox,spam, andtrashmay be added to an email. For
the time being,trashwill be preferred overspam, andspamoverinbox.Trash(capitalT) is reserved and not allowed, usetrash(lowercase, see above) to bin messages remotely.archiveorarxivare reserved and not allowed; seeissue/109andissue/171. To archive e-mails remove theinboxtag.Sometimes GMail provides a label identifier on a message for a label that does not exist. If you encounter thisissueyou can get around it by usinggmi set --drop-non-existing-labelsand re-try to pull. The labels will now be ignored, and if this message is ever synced back up the unmapped label ID will be removed. You can list labels withgmi pull -t.Sometimes GMailindicates that there are more changeswhen doing a partial pull, but an empty set is returned. The default is to fail, but you can ignore empty history by setting:gmi set --ignore-empty-history.Youcannot add any new files(files starting with.will be ignored) to the lieer repository. Lieer uses the directory content an index of local files. Lieer does not push new messages to your account (note that if you send messages with GMail, GMail automatically adds the message to your mailbox).Make sure that you use the same domain for you GMail account as you initially created your account with: usually@gmail.com, but sometimes@googlemail.com. Otherwise you might get aDelegation deniederror.DevelopmentGithub actions are configured to check for python code formatted byblack.