VRT System/Mail Setup

==Mail setup==
E-mail is sent and received through a special Exim instance on the hosting server. Its configuration follows the lines of the setup described in Mail, but VRT specific configuration is listed below.
====Spam and Malware scanning====
SpamAssassin and ClamAV are used for spam/malware scanning, in Exim ACL which is run at the DATA phase during the SMTP connection. Should SpamAssassin fail for some reason, mail is let through.
acl_check_data:
 # skip spam-check for locally-submitted messages
 accept hosts = +relay_from_hosts
 set acl_m0 = trusted relay
 # skip if message is too large (>4M)
 accept condition = ${if >{$message_size}{4M}}
 set acl_m0 = n/a
 set acl_m1 = skipped, message too large
 # skip if whitelisted in exim
 accept condition = ${if eq{$acl_m2}{skip_spamd}}
 set acl_m0 = n/a
 set acl_m1 = skipped, exim whitelist
 # add spam headers...
 warn spam = nonexistent:true
 set acl_m0 = $spam_score ($spam_bar)
 set acl_m1 = $spam_report
 set acl_m3 = $spam_score_int
 # silently drop spam at high scores (> 12)
 discard log_message = spam detected ($spam_score)
 condition = ${if >{$spam_score_int}{120}{1}{0}}
 # silently discard messages with malware attached
 discard log_message = malware detected ($malware_name)
 demime = *
 malware = *
 accept
==== Message tagging ====
We use Exim filters to tag messages with headers that VRT can match for automatic queue routing. The Exim filters are in /etc/exim4/system_filter (see the inline comments):
# Exim filter
if first_delivery then
 # Remove headers that control OTRS - we don't want these
 headers remove X-OTRS-Priority:X-OTRS-Queue:X-OTRS-Lock:X-OTRS-Ignore:X-OTRS-State
 if $acl_m0 is not "trusted relay" then
 # Remove any SpamAssassin headers and add local ones
 headers remove X-Spam-Score:X-Spam-Report:X-Spam-Checker-Version:X-Spam-Status:X-Spam-Level:X-Spam-Flag
 endif
 if $acl_m0 is not "" and $acl_m0 is not "trusted relay" then
 headers add "X-Spam-Score: $acl_m0"
 headers add "X-Spam-Report: $acl_m1"
 # Add header for OTRS filters
 if $acl_m1 is not "" and $acl_m1 begins "yes" then
 headers add "X-Spam-Flag: YES"
 # overload X-Spam-Flag since OTRS doesn't do numeric comparison
 elif $acl_m3 is not "" and $acl_m3 is above 20 then
 headers add "X-Spam-Flag: MAYBE"
 else
 headers add "X-Spam-Flag: NO"
 endif
 # add a hook for OTRS to filter list mail
 if
 ($message_headers contains "\nList-Id:" or
 $message_headers contains "\nList-Help:" or
 $message_headers contains "\nList-Subscribe:" or
 $message_headers contains "\nList-Unsubscribe:" or
 $message_headers contains "\nList-Post:" or
 $message_headers contains "\nList-Owner:" or
 $message_headers contains "\nList-Archive:") and
 $header_precedence: does not match "^(bulk|junk|list)"
 then
 headers remove Precedence
 headers add "Precedence: bulk"
 endif
 endif
endif
====VRT mail routing====
Mail destined for VRT is served by a simple accept router otrs, which does a MySQL database query to determine the validity of the recipient address being routed, similar to the check done earlier by the main MX hosts.
# Mail destined for OTRS
otrs:
 driver = accept
 condition = ${lookup mysql{SELECT value0 FROM system_address WHERE value0='${quote_mysql:$local_part@$domain}'}{true}fail}
 transport = otrs
On success, the message is handed over to the otrs pipe transport:
# OTRS pipe transport
otrs:
 driver = pipe
 command = OTRS_POSTMASTER
 current_directory = OTRS_HOME
 home_directory = OTRS_HOME
 user = OTRS_USER
 group = OTRS_GROUP
 freeze_exec_fail
 log_fail_output
 timeout = 1m
 timeout_defer
This transport pipes the full contents of the message to the command/path specified in the macro OTRS_POSTMASTER (defined at the top of the file). A current and home directory will be set as specified, and the command will be run as the otrs user and group. If the actual execution/invocation fails for some reason, the message will be frozen on the queue with a warning message sent to root. If the command invocation succeeds, but the return code is EX_TEMPFAIL (e.g. when OTRS cannot access the database), the message is deferred/queued, and will be retried later. Any output will be logged.
====Outbound mail====
Any mail destined for an address that is not an OTRS address, e.g. mail submitted by OTRS itself, will be forwarded to an outbound MX.
===ClamAV ===
The server runs its own ClamAV instance, using the stock clamav-daemon package. The daemon runs as user clamav which has read access to the mail queue via membership in group Debian-exim. Per the stock config, the freshclam daemon is used to update virus definitions.
Exim accesses ClamAV via unix socket at /var/run/clamav/clamd.ctl and silently drops and logs messages containing an infected attachment.
===SpamAssassin ===
The server runs its own SpamAssassin instance. The stock spamassassin package is used, with 
daily updates enabled. Stock rules/scores are kept and we make a few local modifications which are listed below.
Multiple user profiles are not used, SpamAssassin reads global configuration settings and runs as user otrs. Training databases are stored in that user's homedir.
====/etc/default/spamassassin====
# Change to one to enable spamd
ENABLED=1
# Options
# See man spamd for possible options. The -d option is automatically added.
# SpamAssassin uses a preforking model, so be careful! You need to
# make sure --max-children is not set to anything higher than 5,
# unless you know what you're doing.
OPTIONS="--max-children 8 --nouser-config --listen-ip=127.0.0.1 -u otrs -g otrs"
# Pid file
# Where should spamd write its PID to file? If you use the -u or
# --username option above, this needs to be writable by that user.
# Otherwise, the init script will not be able to shut spamd down.
PIDFILE="/var/run/spamd.pid"
# Set nice level of spamd
NICE="--nicelevel 10"
# Cronjob
# Set to anything but 0 to enable the cron job to automatically update
# spamassassin's rules on a nightly basis
CRON=1
====/etc/spamassassin/local.cf====
Non-stock sections are shown here:
# Set which networks or hosts are considered 'trusted' by your mail
# server (i.e. not spammers)
#
trusted_networks 91.198.174.0/24 208.80.152.0/22 2620:0:860::/46 10.0.0.0/8
# short-format report template, starting with Yes/No, used for OTRS filters
clear_report_template
report _YESNO_, score=_SCORE_ | host: _HOSTNAME_ | scores: _TESTSSCORES(,)_ | autolearn=_AUTOLEARN_
# Set file-locking method (flock is not safe over NFS, but is faster)
#
lock_method flock
# Set the threshold at which a message is considered spam (default: 5.0)
#
required_score 3.5
score RP_MATCHES_RCVD -0.500
score RCVD_IN_RP_SAFE 2.000
score RCVD_IN_RP_CERTIFIED 2.000
score SPF_SOFTFAIL 2.000
score SUSPICIOUS_RECIPS 2.000
Category:Mail
====SpamAssassin Training====
There a few steps to spam training:
#user moves spammy messages to the Junk queue
#OTRS Generic Agent:"Export_Spam" runs nightly, filtering for tickets which are not in state "Closed successful" passing MessageIDs to otrs.TicketExport2Mbox.pl
#otrs.TicketExport2Mbox.pl writes the messages to /var/spool/spam/spam, and changes the ticket's state to "Closed successful"
#/usr/local/bin/train_spamassassin picks up /var/spool/spam/spam and feeds it to sa-learn as spam
Ham training is similar:
#OTRS Generic Agent:"Export_Ham" runs nightly, filtering for tickets in non-Junk queues which are in states Open, or Closed successful" and feeding those TicketID's to otrs.TicketExport2Mbox.pl
#otrs.TicketExport2Mbox.pl writes the messages to /var/spool/spam/ham
#/usr/local/bin/train_spamassassin picks up /var/spool/spam/ham and feeds it to sa-learn as ham
The two scripts mentioned above are custom and are installed by puppet from operations/puppet/files/otrs/* in the git repository.