1. Periodic Observation Report

Messaging Technology

1.1 Introduction

and breaches of confidential and personal information re-

The IIR has continued to report on quantitative trends insulting from the capturing of IDs and passwords, malware

spam and its content, and as described in IIR Vol. 39, from

infections, and the like.

here out we will be focusing on explaining and chronicling
the spread of messaging technologies, including technolo-

A number of incidents have spurred these sorts of occur-

gies designed to combat spam.

rences on. A spate of information breaches from a variety of
Web services have occurred, with email addresses included

In this issue, we go over the results of a survey on the spreadin the information exposed in almost all cases, making itof sender authentication technologies, particularly DMARC,

possible for spammers to direct spam with precision. Newsand explain MTA-STS, a mechanism described in an RFC lastalso came of a massive breach of personal information fromyear that relates to TLS encryption connection policies fora major hotel chain last year, and reams of spam have sub-email delivery channels, as well as SMTP TLS Reporting, asequently made their way into the inboxes of the exposedmechanism for reporting of TLS connection information. In

addresses. Some such spam messages even contain a login

relation to messaging, we report on the JPAAWG 1st Generalpassword. Services available via the Web can be conve-

Meeting, held last year and co-hosted alongside the Anti-nient, but the service provider’s security cannot always be

spam Conference, as well as on JPAAWG itself.

trusted. Users need to properly understand the strength ofpasswords and other information they set on Web services,

1.2 Spoofed Emails and Information Breaches

as well as the types of services for which the same pass-

Emails spoofed to appear as though they were sent bywords are used.

someone else cause so many kinds of problems that they
are given names like phishing emails and BECs (Business1.3 Sender Authentication Rates

Email Compromises). The damage caused by such emails is

We have noted previously that sender authentication is anboth serious and wide ranging, including financial damages

effective countermeasure against email spoofing. Settings

none

14.3%

temperror

0.1%

permerror

1.4%

neutral0.7%

softfail

7.0%

hardfail

6.5%

pass 70.1%

Figure 1: Breakdown of SPF Authentication Results

4© Internet Initiative Japan Inc.

Sep.2019

Vol.

43

1. Periodic Observation Reportneed to be configured on both the sending and receiving

authentication was not possible, so turning this around, it

ends. Email recipients can use authentication to detectmeans that 85.7% of emails received were from senders

spoofed emails, while senders can configure their sys-that have implemented SPF. The year-earlier (April 2018)tems to ensure that their emails can be distinguished fromfigure for “none” was 16.0%, around the same level,spoofed ones.

which indicates that sender authentication has spread to apoint that the vast majority of received emails can be SPF

If we are to promote the spread of sender authentication,authenticated.

we first need to understand how far it currently permeatesthe space. Here, we report on two sets of survey results,

Figure 2 shows a breakdown of DKIM authentication results.

one looking at volume-based deployment rates among send-

As a proportion of the total, the figure for “none” is 62.2%,

ers from a recipient perspective, and the other looking at

which indicates that less than 40% of emails received were

the proportion of registered domain names on which senderfrom senders that have implemented DKIM. The year-earlier

authentication is implemented.

figure for “none” was 62.4%, so DKIM deployment rates
have not changed that much.

1.3.1 Volume-based Deployment rates
Here, we go over the spread of sender authentication among

Figure 3 breaks down DMARC authentication results. And

senders from an email recipient perspective based on emailby the same measure, the figure for “none” here is 76.9%,

authentication data for emails received via IIJ’s email ser-indicating that around 20% of emails received were from do-vices in April 2019.

mains where the sender has implemented DMARC. DMARC
authentication uses the results of SPF and DKIM, so it is

Figure 1 shows a breakdown of SPF authentication re-predictable that authentication rates here will be lower thansults. Of all emails received, SPF authentication returnedfor SPF and DKIM. That said, and although the resending of“none” for 14.3%. A value of “none” indicates that SPF

emails is an issue, emails can be DMARC authenticated via

pass 35.6%

pass 15.2%
fail 7.7%
permerror 0.1%
temperror 0.1%

none

62.2%

none

76.9%

fail 1.1%
neutral 1.1%

Figure 2: Breakdown of DKIM Authentication Results

Figure 3: Breakdown of DMARC Authentication Results© Internet Initiative Japan Inc.

5

SPF alone, so the figure is really quite low considering thatfor the purpose of gauging the spread of sender authentica-

the SPF deployment rate is above 80%.

tion technology. I am taking part in the studies as a visiting
researcher for the Japan Data Communications Association.

Figure 4 shows the results for DMARC since January
2016. Initially, the figure for “none” was 87.5%, so on a

Figure 5 shows the results for March 2018 onward. For

volume basis, the deployment rate has risen by around 10

domain names configured with an MX record, which indi-

percentage points over almost three years. The proportion

cates the domain name is used for email, the graph shows

has roughly doubled. Although changes in the proportion

what proportion had a DMARC record configured, brokenof “fail” results have fluctuated over time, the data show

down by type of jp domain. According to the latest datathat the proportion of all emails that can be authenticated,from May, this was 0.95% of jp domains overall. By type,including those for which authentication fails, is graduallyad.jp tops the list, but still only with a figure of 3.4%. Next

rising.

down the list with 2.1% is go.jp, which has a step-function
look to it on the graph because the number of such domains

1.3.2 Deployment Rates Based on Registered Domain Names

registered is small.

Next, for registered jp domain names, we look at whether
SPF or DMARC is implemented. As noted previously in Vol.

Materials*1 disclosed by NISC (the Cabinet Office’s cyber-

39, we have a joint research agreement with the Japansecurity center) indicate that the use of SPF, DKIM, andRegistry Services (JPRS)—which manages jp domain

DMARC on the sender and receiver sides is listed as a mea-

names—and the Japan Data Communications Association

sure for preventing email spoofing within the information

(%)
100

75

50
■none
■permerror■temperror
■fail
■pass

25

0

Fe
b.
M
ar
.
Ap
r.

Fe
b.
M
ar
.
Ap
r.
M
ay
Ju
n.
Ju
l.
Au
g.
Se
p.
O
ct.
No
v.
De
c.
Ja
n.
’19

Ja
n.
’1
Fe 6
b.
M
ar
.
Ap
r.
M
ay
Ju
n.
Ju
l.
Au
g.
Se
p.
O
ct.
No
v.
De
c.
Ja
n.
’1
Fe 7
b.
M
ar
.
Ap
r.
M
ay
Ju
n.
Ju
l.
Au
g.
Se
p.
O
ct.
No
v.
De
c.
Ja
n.
’18

(Date)

Figure 4: Breakdown of DMARC Authentication Results Over Time

*1

“Guidelines on the Formulation of Information Security Measures for Government Agencies and Related Bodies (2018 Edition)” (in Japanese at https://www.nisc.
go.jp/active/general/pdf/guide30.pdf).

6© Internet Initiative Japan Inc.

Sep.2019

Vol.

43

1. Periodic Observation Reportsecurity strategy for government bodies. This means we canindustries surveyed. As I reported last time, this increase

expect the proportion of go.jp domains with a DMARC re-likely traces to a legally binding order*3 issued by the United

cord configured to increase ahead. Note also that registered

States Department of Homeland Security. And according togo.jp domains top the list for the proportion with an SPF

DMARC.org, a group that advocates for the use of DMARC

record configured (Figure 6).

technology, the number of domains in the DNS with DMARC
records increased by over 2.5-fold in 2018*4.

Similarly, the proportion of all jp domain names with an
SPF record configured was 59.7%. This is a 2.8-percent-

1.4 Encryption of Email Delivery Channels

age-point increase vs. the previously reported figure of

Email is used not only to exchange simple messages but

56.9% (Vol. 39). The fact that this SPF adoption rate isalso as a means of transferring various types of data via

still rising seems to indicate that awareness of SPF is quite

attachment capabilities (MIME). Meanwhile, it does seemhigh. Unfortunately, the rate of increase for DMARC is quitethat users do not give much consideration to what route

low compared with that for SPF, so we will need to boost

an email that contains data will take when being delivered

awareness of DMARC somehow.

nor to what level of data leakage risk exists. The SMTP
email delivery protocol can use TLS (STARTTLS) as an ex-

1.3.3 Deployment rates Overseas

tension. Here, we discuss issues with this conventional

According to a survey

by the US-based Valimail, 80%

STARTTLS protocol and standards established to addressof federal government domains in the United States havethose issues, namely MTA-STS and SMTP TLS Reporting.

*2

DMARC records. This was the highest rate among the(%)
4■average
■gen
■geo
■lg
■ed
■gr
■ne
■or
■go
■co
■ac
■ad

3

2

1

3.
9.
20
18
3.
16
3.
23
3.
30
4.
6
4.
13
4.
20
4.
27
5.
4
5.
11
5.
18
5.
25
6.
1
6.
8
6.
15
6.
22
6.
29
7.
6
7.
13
7.
20
7.
27
8.
3
8.
10
8.
17
8.
24
8.
31
9.
7
9.
14
9.
21
9.
28
10
.
10 5
.1
10 2
.1
10 9
.2
6
11
.2
11
.9
11
.1
11 6
.2
11 3
.3
0
12
.
12 7
.1
12 4
.2
1 1
1. 2.28
4.
20
19
1.
11
1.
18
1.
25
2.
1
2.
8
2.
15
2.
22
3.
1
3.
8
3.
15
3.
22
3.
29
4.
5
4.
12
4.
19
4.
26
5.
3
5.
10

0

(Date)

Figure 5: Proportion of jp Domains with a DMARC Record
（％）
100

75

50

25

0
ad

ac

co

go

or

ne

gr

ed

lg

geo

gen average

Figure 6: Proportion of Domains with an SPF Record Declaration*2

Email Fraud Landscape, Q4 2018 (https://www.valimail.com/resources/email-fraud-landscape-q4-2018/).

*3

DHS, “Binding Operational Directive 18-01” (https://cyber.dhs.gov/bod/18-01/blank).

*4

DMARC Policies Up 250% In 2018 (https://dmarc.org/2019/02/dmarc-policies-up-250-in-2018/).

© Internet Initiative Japan Inc.

7

1.4.1 Issues with STARTTLS

1.4.2 MTA-STS and TLSRPTThe SMTP extension STARTTLS (TLS) is used to encrypt

MTA-STS*5 is a mechanism in which recipient domains use

the channel when emails are delivered. The procedure is

a combination of DNS and HTTPS to publish their receiving

as follows: if the Recipient mail server supports STARTTLS

policies. This mechanism allows you to determine whether(determined by the response when connecting), the sender

TLS authentication is supported before sending an email andsends a STARTTLS command to start a TLS session. So thewhat action the sender should take if a TLS connection can-channel cannot be encrypted under the following conditions.

not be established.

• Recipient mail server does not have STARTTLS (doesRecipient domains should make the following settings.

not return a response to STARTTLS)• The STARTTLS command is sent in order to start a
TLS session but the available TLS version and cipher(1) Configure an MTA-STS record(2) Set a “well-known” path so that the MTA-STS policy
can be fetchedsuites do not match
Cipher suites are combinations of encryption algorithms,

The MTA-STS record is usually a TXT record that is named

key length, and so on. Encrypted communications are notby adding “_mts-sts” to the destination domain and thatpossible unless both sending and receiving ends are able tostarts with the string “v=STSv1”. So if the mail destina-

use the same cipher suite. If the STARTTLS command can-tion domain is “example.com”, the record is configured as

not be executed, many sending email servers will switch

follows.

to conventional unencrypted plaintext email transmission.
This setup exposes email to a sort of man-in-the-middle_mta-sts.example.com. IN TXT "v=STSv1; id=20160831085700Z;"

attack because by intercepting the SMTP session and deleting the intended recipient server’s STARTTLS response,

The id parameter is a string that can be used to determine

an attacker can force plaintext transmission and snoop thewhen the policy has been updated. By first referring to thiscontents of email. This sort of technique is also called a

MTA-STS record, the sender can check whether the recipi-

downgrade attack.

ent domain supports MTA-STS.

*5

8

SMTP MTA Strict Transport Security, RFC 8461

© Internet Initiative Japan Inc.

Sep.2019

Vol.

43

1. Periodic Observation Report

To fetch the MTA-STS policy, the sender refers to the

The TLSRPT*6 specification is used to report to the sender“well-known” path on the policy domain prepended with

if the policy verification succeeds or fails under MTA-STS or“mta-sts”. The “well-known” path is described in RFC

other mechanisms such as DANE*7. Senders use the DNS

5785. In the case of MTA-STS, it is fetched via an HTTPS

to publish a TLSRPT policy for receiving reports. Email re-

GET request for the following path.

cipients that support TLSRPT first determine whether thisTLSRPT policy has been specified by the sender domain,and if fetchable, a report is sent to the report recipient, if

https://mta-sts.example.com/.well-known/mta-sts.txt

specified, in that policy. The TLSRPT policy settings can be
The MTA-STS policy contains line feed-separated (CRLF-

retrieved by prepending “_smtp._tls” to the target domain.

separated)

The parameters are quite similar to those for DMARC*8 but

key/value

pairs.

The

currently

allowableparameters are shown in Table 1.

differ in that “v=TLSRPTv1” specifies version 1 of TLSRPT
and the “rua=” field, which specifies where the report is“max_age” specifies how long the policy should be cached.

to be submitted, can specify the mailto schema (“rua=-“mx” specifies patterns matching hostnames given in themailto:”) as well as HTTPS (“rua=HTTPS:”). An example of

MX record. Multiple hosts and patterns can be set. Table 2

a TLSRPT policy record appears below.

shows the allowable values for operation mode (“mode”).
The sending MTA determines whether to continue sending
emails based on the value of this “mode” field.
An example of an MTA-STS policy appears below.

̲smtp.̲tls.example.com. IN TXT "v=TLSRPTv1;rua=mailto:reports@example.com"

When emailing reports to destinations specified using “rua=mailto:”, the report must contain a DKIM signature by
the sender domain. The DKIM record of the sender providing

version: STSv1
mode: enforce
mx: mail.example.com
mx: *.example.net
mx: backupmx.example.com
max_age: 604800

the DKIM signature SHOULD contain the “s=tlsrpt” service
type declaration.

Table 1: MTA-STS Policy Parameters

Table 2: MTA-STS Policy Modes

Parameter

Meaning

Operation mode

Meaning

versionThe version (currently only “STSv1”)

enforce

Messages are not delivered to hosts that fail policy validation or TLS

mode

Expected behavior of sender if policy validation fails

testing

max_age

Max lifetime of the policy (in seconds)

Report sent if sending MTA implements TLSRPT*6;
messages continue to be deliveredmx

Allowed MX record patterns

none

Indicates that no explicit MTA-STS policy is applied*6

SMTP TLS Reporting, RFC8460*7

The DNS-Based Authentication of Named Entities (DANE) Transport Layer Security (TLS) Protocol: TLSA, RFC 6698

*8

Domain-based Message Authentication, Reporting, and Conformance (DMARC), RFC 7489

© Internet Initiative Japan Inc.

9

Based on this, Figure 16 shows plots of total traffic of theselector._domainkey.example.com IN TXT"v=DKIM1; k=rsa; s=tlsrpt; p=Mlf4qwSZfase4fa=="

past 10 years. The data series are stacked. The outbounddata are observations made at entry points, and the inbounddata are observations made at exit points. Some traffic is

Reports sent via email are sent as attachments (MIME)

eliminated within the backbone, such as that involved in at-in the same manner as DMARC reports. An example of atacks, but generally all traffic that comes into the backbone

TLSRPT policy record for sending reports over HTTPS also

also exits at some point, so the totals are almost the same.

appears below.

1.5 About JPAAWG_smtp._tls.example.com. IN TXT "v=TLSRPTv1; rua=https://reporting.example.com/v1/tlsrpt"

The IIR has mentioned the international antispam organization M3AAWG*9 several times in the past. Recently, it has
also become a forum for a range of discussion on highly

Report data should be compressed for both email and HTTPS

relevant security issues beyond that of email. It has alsotransport. Whether applying compression or not, the media

been supporting the establishment of regional organizations

type should be consistent with the format (“application/tl-

beyond North America and Europe, where many M3AAWG

srpt+gzip” or “application/tlsrpt+json”). Reports are sent

members reside. A recently formed group is LAC-AAWG forin JSON format, unlike DMARC reports. We do not go overLatin America and the Carribean. The organization is also

the parameters given in the report data here, but details canworking toward and supporting AFR-AAWG for Africa. Thisbe found in RFC 8460.

leaves only the issue of Asia and what to do there.

*910

Messaging, Malware and Mobile Anti-Abuse Working Group© Internet Initiative Japan Inc.

Sep.2019

Vol.

43

1. Periodic Observation Report

IIJ has long been an active member in M3AAWG since it

and key members of M3AAWG. With the event’s success,was established, but the number of participants from Japan

we made preparations for ongoing JPAAWG activities, cul-

has not really risen as much the number from the US andminating in JPAAWG being formally established on May 30,

Europe. To increase the number of participants, we have2019. We hope JPAAWG’s future activities will be of inter-been publicizing the M3AAWG’s activities in Japan andest to you.

sounding out the prospects of holding an M AAWG General
3

Meeting in Japan or Asia from time to time. Against that

1.6 Conclusion

backdrop, M AAWG has been making efforts to support

In this issue, we described MTA-STS, a technical specifi-

M3AAWG-linked activities in other regions. And out of thatcation for reliably ensuring encryption of email deliveries,

process emerged efforts among M3AAWG and participants

and TLSRPT as a means of ascertaining what operations

from Japan to set up JPAAWG.

have taken place. So far, the IIR has looked at sender au-

3

thentication technologies including DMARC, ARC, andAs an organization, JPAAWG (Japan Anti-Abuse Working

DANE, but email-related technical specifications continue toGroup) is entirely independent of, but receives considerable

evolve along with new specifications such as BIMI (Brand

support from, M3AAWG. The JPAAWG 1st General Meeting

Indicators for Message Identification) and JMAP (JSON Meta

on November 8, 2018, was held in conjunction with theApplication Protocol). Going forward, the IIR will continue toInternet Association Japan’s Anti-Spam Conference, an

discuss new technical specifications and the background toevent that has been running for over a decade, and attracted

their development.

many speakers and participants. Speakers included the chair

Shuji Sakuraba
Senior Manager, Application Service Department, Network Division, IIJ. Mr. Sakuraba is engaged in the research and development of communication
systems. He is also involved in various activities in collaboration with related external organizations aimed at bringing about safe and secure messaging
environments. He has been a member of M3AAWG since its establishment. He is the chair of the Japan Anti-Abuse Working Group (JPAAWG).
He is acting chairperson of the Anti-Spam mail Promotion Council (ASPC) and a member of its administrative group, as well as chief examiner for
the Technology Workgroup. He is chairman of Internet Association Japan’s Anti-Spam Measures Committee. He is a member of the Email Security
Conference program. He is a visiting researcher for the Japan Data Communications Association. And he is a visiting researcher at JIPDEC.

© Internet Initiative Japan Inc.

11