Package: dmarc-metrics-exporter
dmarcs-metrics-exporterExport metrics derived from DMARC aggregate reports to Prometheus.
This exporter regularly polls
for new aggregate report emails
via IMAP.
The following metrics will be collected
and exposed at an HTTP endpoint
for Prometheus:dmarc_total: Total number of reported messages.dmarc_compliant_total: Total number of DMARC compliant messages.dmarc_quarantine_total: Total number of quarantined messages.dmarc_reject_total: Total number of rejected messages.dmarc_spf_aligned_total: Total number of SPF algined messages.dmarc_spf_pass_total: Total number of messages with raw SPF pass.dmarc_dkim_aligned_total: Total number of DKIM algined messages.dmarc_dkim_pass_total: Total number of messages with raw DKIM pass.Each of these metrics is subdivided by the following labels:reporter: Domain from which a DMARC aggregate report originated.from_domain: Domain from which the evaluated email originated.dkim_domain: Domain the DKIM signature is for.spf_domain: Domain used for the SPF check.In addition, there is admarc_invalid_reports_totalmetric with a count of
DMARC report emails from which no report could be parsed. It is subdivided by
a singlefrom_emaillabel.InstallationThis describes the manual setup fo dmarc-metrics-exporter.
An Ansible role for automated deployment is provided inroles.
Further instructions for Ansible are given in the readme file
provided in that directory.It is best to run dmarc-metrics-exporter under a separate system user account.
Create one for example withadduser--system--groupdmarc-metricsThen you can install dmarc-metrics-exporter withpipfrom PyPI for that
user:sudo-udmarc-metricspip3installdmarc-metrics-exporterYou will need a location to store themetrics.dbthat is writable by that
user, for example:mkdir/var/lib/dmarc-metrics-exporterchowndmarc-metrics:dmarc-metrics/var/lib/dmarc-metrics-exporterConfigurationTo run dmarc-metrics-exporter a configuration file in JSON format is required.
The default location is/etc/dmarc-metrics-exporter.json.Because the configuration file will contain the IMAP password,
make sure to ensure proper permissions on it,
for example:chownroot:dmarc-metrics/etc/dmarc-metrics-exporter.jsonchmod640/etc/dmarc-metrics-exporter.jsonAn example configuration file is provided in this repository inconfig/dmarc-metrics-exporter.sample.json.The following configuration options are available:listen_addr(string, default"127.0.0.1"): Listen address for the HTTP endpoint. Use"0.0.0.0"if running in a dockerized environment.port(number, default9797): Port to listen on for the HTTP endpoint.imap(object, required): IMAP configuration to check for aggregate reports.host(string, default"localhost"): Hostname of IMAP server to connect to.port(number, default993): Port of the IMAP server to connect to.username(string, required): Login username for the IMAP connection.password: (string, required): Login password for the IMAP connection.use_ssl: (boolean, defaulttrue): Whether to use SSL encryption for the connection. Disabling this will transmit the password in clear text! Currently, there is no support for STARTTLS.verify_certificate: (boolean, defaulttrue): Whether to verify the server’s SSL certificate. You might have to set this tofalseif you are using a self-signed certificate. If this is disabled, someone else could impersonate the server and obtain the login data.folders(object):inbox(string, default"INBOX"): IMAP mailbox that is checked for incoming DMARC aggregate reports.done(string, default"Archive"): IMAP mailbox that successfully processed reports are moved to.error: (string, default"Invalid"): IMAP mailbox that emails are moved to that could not be processed.storage_path(string, default"/var/lib/dmarc-metrics-exporter"):
Directory to persist data in that has to persisted between restarts.poll_interval_seconds(number, default60): How often to poll the IMAP server in seconds.deduplication_max_seconds(number, default604800which is 7 days): How long individual report IDs will be remembered to avoid counting double delivered reports twice.logging(object, default{}): Logging configuration, see the “Logging configuration” section below.Logging configurationWhen providing a custom logging configuration, it must follow the dictionary
schema (version 1) described in thelogging.config documentation.
In general, a provided top-level key will replace the default configuration,
but there are some exceptions. The following keys are always fixed:versionwill always be1.incrementalwill always befalse.formattersis fixed and provides the following formatters:plainrenders human-readable log messages without colors.coloredrenders human-readable log messages with colors.jsonrenders structured JSON log messages.In addition, therootkey has some special handling. If it is overridden,
but nothandlerskey is provided,handlers: ['default']will be inserted
automatically. Also, thelevelkey will be set to'DEBUG'if the
application is started with the--debugflag.Configuring log levelTo change the log level globally:{"logging":{"root":{"level":"WARNING"}}}Configuring logging formatTo change the logging format:{"logging":{"handlers":{"default":{"class":"logging.StreamHandler","formatter":"json"}}}}Valid formats areplain,colored, andjson.Disabling Uvicorn access logsTo disable the Uvicorn access logs:{"logging":{"loggers":{"uvicorn.access":{"propagate":false}}}}UsageTo run dmarc-metrics-exporter with the default configuration in/etc/dmarc-metrics-exporter.json:sudo-udmarc-metricspython3-mdmarc_metrics_exporterTo use a different configuration file:sudo-udmarc-metricspython3-mdmarc_metrics_exporter--configuration<path>You can enable debug logging with the–debugif you do not want to provide your own logging configuration:sudo-udmarc-metricspython3-mdmarc_metrics_exporter--debugsystemdInstead of manually starting the dmarc-metrics-exporter,
you likely want to have it run as a system service.
An example systemd service file is provided in this repository inconfig/dmarc-metrics-exporter.service.
Make sure that the paths and user/group names match your configuration
and copy it to/etc/systemd/systemto use it.
To have systemd pick it up asystemctldaemon-reloadmight be necessary.You can than start/stop dmarc-metrics-exorter with:systemctlstartdmarc-metrics-exportersystemctlstopdmarc-metrics-exporterTo have dmarc-metrics-exporter start on system boot:systemctlenabledmarc-metrics-exporterDockerA new docker image is build for each release
with GitHub Actions as described in this yaml-file:.github/workflows/docker-publish.yml.Note that you should configure thelisten_addrto0.0.0.0to be able to
access the metrics exporter from outside the container.Example docker-compose file:version: "3"

services:

  dmarc-metrics-exporter:
    # source: https://github.com/jamborjan/dmarc-metrics-exporter/pkgs/container/dmarc-metrics-exporter
    container_name: dmarc-metrics-exporter
    hostname: dmarc-metrics-exporter
    image: jgosmann/dmarc-metrics-exporter:0.3.0
    restart: unless-stopped
    user: 1000:1000 #PUID=1000:PGID=1000
    expose:
      - 9797
    volumes:
      - '/host/folder/dmarc-metrics-exporter.json:/etc/dmarc-metrics-exporter.json'
      - '/host/folder/dmarc-metrics-exporter/metrics:/var/lib/dmarc-metrics-exporter:rw'
    logging:
      driver: "json-file"
      options:
        tag: "{{.ImageName}}|{{.Name}}|{{.ImageFullID}}|{{.FullID}}"
    networks:
      - YourDockerLan

# $ docker network create -d bridge --attachable YourDockerLan
networks:
  YourDockerLan:
    external:
      name: YourDockerLanPrometheusExample prometheus config file:global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:

scrape_configs:

  - job_name: 'dmarc-metrics-exporter'
    static_configs:
      - targets: ['dmarc-metrics-exporter:9797']GrafanaAn example configuration file is provided in this repository inconfig/dmarc-metrics-exporter.grafana.sample.json. This example dashboard displays the collected metrics as shown in the screenshot below.Example grafana dashboardHintsYou should not use your normal email and password credentials for the dmarc-metrics-exporter.
If you are not able to create a dedicated service account email account, you should use an app password.Microsoft Exchange OnlineApp passwords are available when you are using Multi Factor Authentication (MFA).Manage app passwords for two-step verificationIf you don’t see the app passwords option or get an error,check if MFA is enabledfor the user.If you still don’t see the app passwords option,check if app passwords are allowed in your organizationFinally,ensure that IMAP is enabled for the user.DevelopmentPrerequisitesPythonpre-commitPoetryDockerSetup development environmentpre-commitinstallpoetryinstallRun testsdocker-composeup-dpoetryrunpytest