2. Focused Research (1)Research on a Method of Constructing Sender
Reputation
2.1 Introduction

paper*7 published in the journal of the Information Processing

Twenty years ago, in January 2004, IIJ joined MAAWG

Society of Japan. The paper describes a sender reputation(Messaging Anti-Abuse Working Group), which works on

construction method and feedback loop. In this article,

measures to combat spam globally. I participated in the

I focus on the sender reputation aspect of the paper. The

group’s founding meeting in April 2004, and I have con-

paper was also recognized as a specially selected paper bytinued to attend the general meetings since. The group’s

the Information Processing Society of Japan.

name has been changed slightly to M3 AAWG*1, and thescope of its activities has been expanded. The 60th general

2.2 Sender Reputation

meeting, marking the group’s 20th anniversary, took place

DNSBL (DNS Blocklist) has long been used as a mecha-

in February 2024.

nism for determining whether to accept email based on
sender information. It uses a DNS query to look up the

MAAWG technical discussions were initially focused on

source host’s IP address. While the source IP address isevaluating and popularizing sender authentication technolo-

not an appropriate way to identify the sender of an email,gies, particularly SPF

and DKIM , to address the fact that

DNSBL has so far been used because the email address

it was not possible to accurately identify the sender of an

specified in the email headers and during the email de-email, which really should be regarded as a flaw in the email

livery process to indicate who the sender is cannot besystem. Those technical discussions continued over therelied on. With the spread of SPF and DKIM for senderyears, with members of M3 AAWG playing a central role inauthentication, there are moves to use domain names ascreating technical specifications like DMARC , ARC , andauthenticated, trustworthy information for determining

BIMI*6. From the start, technical discussions about sender

whether to accept or reject email—this is the concept ofauthentication were premised on the idea that, as a next

domain-name sender reputation.

*2

*3

*4

*5

step, we would need to be able to determine if we should
accept email using authenticated domain names—in otherIn addition to domains with a negative reputation fromwords, sender reputation. Indeed, the first SPF specifica-

which email should not be accepted, one can also imagine

tion, RFC4408, mentioned domain reputation, and Google

there to be legitimate domain names from which email

and Yahoo in the US have recently been pushing heavily forshould be accepted. When quantified, the factors behindemail senders to get on board with sender authentication tothis determination result in a reputation score. In simpler

tighten up the way incoming email is handled. Indeed, eventerms, reputation can be thought of as the basis for ain Japan the number of domains supporting DMARC rose

Block List and Allow List of domain names.

sharply after the companies announced these tougher measures. As a visiting researcher at IAjapan, I study domain jp

Alongside the rise of sender authentication technology,

names, and as of February 2024, around a quarter of the

we have also seen an increase in the volume of spam fromdomain names used for email had a DMARC record set, indi-

actors who have registered their own domain names andcating a roughly three-fold rise in the proportion of domains

properly configured SPF and DKIM. The domain nameswith such a record.

registered for this type of spam are used as throwaways,
so building a Block List based on domain reputation is,

The IIJ Research Laboratory conducts research on meth-unfortunately, not all that effective. It may thus be more

ods for building sender reputation. This article discusses a

effective to use an approach that involves building an*1

Messaging, Malware and Mobile Anti-Abuse Working Group.

*2

Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1 (RFC7208).

*3

DomainKeys Identified Mail (DKIM) Signatures (RFC6376).

*4

Domain-based Message Authentication, Reporting, and Conformance (RFC7489).

*5

Authenticated Received Chain.

*6

Brand Indicators for Message Identification (Internet-Draft).

*7

Shuji Sakuraba et al., Sender Reputation Construction Method And Feedback Loop Using Sender Authentication, Journal of the Information Processing Society ofJapan, Vol. 64, No. 1, pp. 13–23 (2023).

14© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)Allow List and combining this with email filters and theas the inability to properly authenticate forwarded emails.

like to assess the content of emails not covered by theBut because sending mail servers need to perform the ad-

Allow List. Spam as a proportion of all email has declined

ditional steps of creating the digital signature and adding

from what it was in the past, and so if the bulk of email

the DKIM signature information to emails, DKIM has not

can be assessed using the simpler and easier methods of

become as widespread as SPF.

sender authentication and sender reputation, then morecomputing resources can be devoted to assessing the

2.4 A Method of Constructing Sender Reputation

content of email messages.

Here, I describe a method that uses sender authentication
to collect SPF-authenticated domain names from which

With this background now laid out, this article describes

email should be accepted. In general, it is relatively easy

a method for collecting the domain names of legitimate

to collect this information in the case of spam because

emails and building reputations on that basis.

this sort of email is itself unwanted and shouldn’t be accepted. The approach has been to collect data for spam

2.3 Characteristics of Sender Authentication
Technologies

filters and block lists by extracting salient characteristics

Detailed descriptions of the SPF and DKIM sender authen-comes to legitimate emails, however, a challenge is thattication technologies can be found in sources such as thethe messages may contain highly confidential informa-

Sender Authentication Technology Deployment Manual*8.

tion, making it generally difficult to collect the desired

Here, therefore, I focus on parts of the paper related todata. And because the information is used to determine

the method of constructing sender reputation.

whether or not an email should be accepted, erroneously

and sender information from the collected spam. When it

recording an email sender as a spam emitter can cause
SPF authenticates the domain name of an email address

substantial damage, so accuracy is required when making

representing the sender of an email over the Simple Mailsuch entries.

Transfer Protocol (SMTP). On the sending side, an SPF
record listing email sender IP addresses and the like is

Here, we take forwarded emails to be emails that should

published on the domain’s DNS server, and on the receiv-

be accepted, and describe a method for extracting theing end, when an email is received, the receiving server

senders of forwarded emails and creating a list on thatlooks up the IP address to determine whether the email

basis.

is from the correct sender. This mechanism means that
implementing SPF on sending servers is relatively easy as

2.4.1 The Nature of Forwarded Emailall the administrator needs to do is publish an SPF DNSEmail forwarding is often used as a way of consolidatingrecord, and use of SPF thus continues to spread. One

emails, such as when you use multiple email accounts andproblem, however, is that recipients cannot properly au-

want to view them in a single place. This mechanism has

thenticate emails when they were sent by an entity otherlong been used in email systems such as the opensource

than the original email sender.

Sendmail, which can be configured to automatically redirect received emails by adding the forwarding address toWith DKIM, the server creates a digital signature from thethe .forward file in the user’s home directory. Hence, theemail header and body for each email sent and affixes thisforwarder’s email forwarding settings point to the recipientalong with other relevant information to the email header.

of the forwarded emails, and so from the forwarded email

Because it uses an authentication method that does notrecipient’s perspective, the forwarding email sender can bedepend on the route by which an email was delivered,regarded as an email sender from whom email should be

DKIM is not subject to the problems inherent in SPF, suchaccepted.

*8

Anti-Spam Consultation Center, Japan Data Communications Association, Sender Authentication Technology Deployment Manual (https://www.dekyo.or.jp/soudan/aspc/report.html#dam, in Japanese).

© Internet Initiative Japan Inc.

15

If you can collect a list of such email forwarders, you should

as a means of tightening defenses against email spoofing.

be able to construct reputations for email senders from

For this reason, when forwarding email, some forwarding

which email should be accepted.

sources rewrite the envelope-from field to contain the domain name of the forwarding source. When this is done,

2.4.2 Forwarded Email and Sender Authentication

emails will pass both SPF and DKIM authentication at theIn basic email forwarding, the email address set by theforwarding destination. However, the domain names authen-original sender is used in the envelope-from field*9, whichticated in this case usually differ. The results are illustrated

corresponds to the domain authenticated by SPF. Because

in Figure 2.

of this mechanism, SPF authentication at the forwarded
email destination fails. DKIM, meanwhile, does not use the2.4.3 Assessing Forwarded Email Source using Sendersender’s IP address for authentication, so emails to which a

Authentication Results

DKIM signature has been added can be DKIM-authenticatedI have explained that there are two email forwarding meth-at the forwarding destination. The results are illustrated in

ods and that the SPF and DKIM authentication results differ

Figure 1. The SDID (Signing Domain Identifier) in the figure

across those methods. Thus, we use the sender authen-is the domain name authenticated by DKIM.

tication results to determine whether an email has been
forwarded, and collect this as reputational information on

Recently, an increasing number of email-receiving servers are

the forwarded email senders. First, we identify forwarding

refusing to accept emails that cannot be SPF-authenticatedsources that do not rewrite the RFC5321.From field when

RFC5321.From: example.jp

RFC5321.From: example.jp

RFC5321.To: example.com

RFC5321.To: example.org

SDID: example.jp

SDID: example.jp

forwarding mailexample.jp

example.com

example.org

SPF : pass (example.jp)

SPF : fail (example.jp)

DKIM : pass (example.jp)

DKIM : pass (example.jp)

Figure 1: Results of Sender Authentication of Forwarded Email

RFC5321.From: example.jp

RFC5321.From: example.net

RFC5321.To: example.net

RFC5321.To: example.org

SDID: example.jp

SDID: example.jp

rewrite forwarding mailexample.jp

example.net

example.org

SPF : pass (example.jp)

SPF : pass (example.net)

DKIM : pass (example.jp)

DKIM : pass (example.jp)

Figure 2: Sender Authentication of Forwarded Email with Sender Information Rewritten*916

The sender email address according to the email delivery protocol (SMTP). May appear as RFC5321.From, referring to the SMTP RFC.

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)forwarding emails based on the following authentication

related—for instance, usually they will have the same do-

criteria, and thus build a list of SPF-authenticated domains

main name or the same upper domain name. For example,

from which email should be accepted.

DMARC defines an organizational domain name and assumes
that the SPF- or DKIM-authenticated domain name and the• Sending IP addresses for which SPF authentication fails
and DKIM authentication passes

sending domain name in the header are the same or havethe same organizational domain name. Given this specifica-• SPF-authenticated domains that send email from thetion, the SPF- and DKIM-authenticated domain names areabove IP address and for which SPF authentication

closely related, even for ordinary email. If the original email

passessender supports DKIM, and the SPF-authenticated domainname is rewritten when the email is forwarded, it is common

With the first criterion, we collect the source IP address

for there to be no relation between the original DKIM-

of forwarded emails. There can be multiple outgoing mailauthenticated domain name and the SPF-authenticatedservers, so to capture a broader range of legitimate email

domain name at the forwarding destination. To identify for-senders, we collect a list of SPF-authenticated domains for

warding sources that rewrite the envelope-from field when

which email is sent from the forwarded email’s source IP

forwarding email, we focus on the relationship between

address and for which SPF authentication passes, the objec-

SPF- and DKIM-authenticated domain names. To automat-

tive being to identify the administrative origin of sent emails.

ically collect these sender IP addresses, we look at emails

This is the second criterion. Both the forwarding IP address

sent from the same IP address, which pass SPF authen-and the SPF-authenticated domain name sent from that ad-

tication, and for which the email sender IP address yields

dress constitute sender reputation. Such IP addresses are

multiple DKIM-authenticated domain names. These are de-legitimate sources from which email should be accepted,

termined to be the email forwarding sources. These email

so any email sent from such addresses, including email thatforwarding source IP addresses and the SPF-authenticatedhas not been forwarded, is considered acceptable. Thisdomain names that they send constitute sender reputation

makes it possible to use not only IP addresses but also SPF-

indicating that emails should be accepted.

authenticated domain names for sender reputation.

2.5 Constructing and Verifying Sender Reputationn
Next, we identify forwarding sources that rewrite the en-

To assess the effectiveness of these methods, we con-velope-from field when forwarding emails according to thestructed sender reputations and applied this to incoming

following criteria and again collect a list of SPF-authenticatedemails. We used the incoming email logs from a real-world

domains from which emails should be accepted.

email service. This email service performs SPF and DKIM
sender authentication when receiving emails, and applies a• Source IP addresses for which both SPF and DKIM

spam filter to all emails, so the results of these operations

authentication passes and the domain names areare available in the logs. We used the results of this spam

unrelatedfilter as the basis for evaluating the judgements made ac-• Of those above, sending IP addresses for which multi-

cording to sender reputation.

ple DKIM-authenticated domain names can be obtained
via DKIM authentication

That is, we construct sender reputation from the SPF and• SPF-authenticated domains sent from the above IP addresses for which SPF authentication passes

DKIM authentication results. Next, we check received email
against the sender reputations, compare it with the resultsof the spam filter, and measure the volume of email classi-

Outgoing emails that have not been forwarded and thatfied as non-spam (ham) and as spam.

support both SPF and DKIM are expected to be closely© Internet Initiative Japan Inc.

17

We constructed sender reputations from around 340 mil-spam by the spam filter for which sender reputation waslion incoming email log entries for the month of September

misapplied. That is, this is the false positive rate (FPR).

2019. At the time, spam accounted for 11.7% of email,This differs from the meaning of a positive result from an

the SPF authentication pass rate was 71.1%, and the DKIM

email filter’s assessment of spam. Here, a positive result

authentication pass rate was 38.1%. From these data,means that email should be accepted, so it is important towe were able to extract 15,169 forwarding IP addresses,

be aware of the relationship between true positives and744,660 SPF domain names sent from ordinary forwarding

false positives based on reputation.

sources, and 11,164 domain names that rewrite the sender

2.6 Discussion

domain name when forwarding email.

By assessing email forwarding source using sender
We applied these sender reputations (indicating that email

authentication technology and constructing sender repu-

should be accepted) to the roughly 36 million emails re-tations based on this, we were able to correctly identify

ceived in the week of October 2019 immediately following

around 58% of acceptable email (ham). At the time, thethe week from which we collected the reputation data

SPF authentication rate was around 70%, so a large(Table 1). We used the same incoming email logs when

portion of that can be attributed to the use of sender

doing this. Differences between two reputation types are

reputation. Detecting forwarding sources that rewrite theshown below.

sender information when forwarding emails and using thisto add to sender reputation increased the effectiveness(1) Ordinary forwarding sources (IPs) and SPF-authenticatedof our method. We were able to increase TPR by overdomain names that do not rewrite sender information

10 points while holding down the increase in FPR to only

when forwarding

0.25pt. During the period to which we applied our sender(2) In addition to (1), sendes (IPs) and SPF-authenticatedreputations, spam accounted for around 9% of received

domain names that rewrite sender information whenemail, so the actual number of false positives was quite

forwarding

low. We also understand, to an extent, why these false
positives occurred, so we think it will be possible to fur-

In Table 1, the ham column indicates the proportion ofther reduce FPR.

email determined not to be spam by the spam filter forwhich sender reputation was successfully applied. In other

This method of constructing sender reputation only

words, this is the true positive rate (TPR). The spam col-uses the results of sender authentication and does not

umn indicates the proportion of email determined to belook at the content of email. Even though it is simple

Table 1: Results of Applying Sender Reputation

18

Reputation

ham(％)

spam(％)

(1)

47.45

3.01

(2)

58.01

3.26

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)in comparison with common email filtering methods,

method, which uses information on sender trustworthi-

we still achieved high spam-detection accuracy. We doness to determine whether email should be accepted or

use the results of DKIM authentication, which does notnot, is a significant contribution. The fact that our method

have a high uptake rate, in identifying the sender of for-of constructing sender reputation does not involve looking

warded emails, but the sender does not necessarily haveat the content of emails also makes it valuable from ato support DKIM for reputation to be useful, and we only

privacy perspective. Further, as demonstrated when weuse a few DKIM-authenticated emails to determine for-tested this method, the fact that it makes it possible towarding source. So even though DKIM uptake is low, it

construct sender reputation using, for example, incoming

is possible to construct adequate sender reputations. In

email logs means that it is possible to produce senderconstructing and applying sender reputations, we used

reputations geared to the emails that your organization

SPF-authenticated domain names, which are widespread,

receives, opening up the prospect of greater sorting ac-

and if SPF uptake increases further, this should enable ac-

curacy. While this method may be inapplicable to a small

curate determinations about even more emails. If DKIM or

number of incoming emails, it should make more com-

DMARC uptake were to increase, we could also consider

puting resources available, which could then be used tousing those authenticated domain names for the purposes

perform deeper assessments based on the content ofof sender reputation.

those email and so forth.

The fact that SPF authentication fails at email forwarding

It has long been the case that emails would be delivered

destinations has until now been considered a shortcoming

even if you had not deployed some form of sender au-of SPF authentication. However, we believe the favor-

thentication, and relatively new technologies like DMARC

able results we obtain using the method for constructing

have thus struggled to gain traction. Yet the recent

sender reputation that we describe here—using the fea-

announcement of new countermeasures for incomingtures of SPF as a network-based method and DKIM as aemail from the likes of Google and Yahoo in the US has

digital-signing method—are actually positive for the up-

prompted more uptake of DMARC, as well as SPF andtake of SPF.

DKIM, on which it is based. This will make it possible tocombat email spoofing while also increasing opportunities

2.7 Conclusion

to apply measures of domain reputation. We will continue

With phishing and other forms of spam becoming more

to pursue research relevant to achieving more accurate

sophisticated these days, such that it is hard to tell le-measures of domain reputation.

gitimate emails from malicious ones, we believe that our

Shuji Sakuraba, Ph.D.
Senior Research Engineer, Technology Coordination Office, IIJ Research Laboratory
Dr.Sakuraba is engaged in research and development related to messaging security. He is also involved in various activities in collaboration
with related external organizations aimed at bringing about safe and secure messaging environments.He has been a member of M3 AAWG
(Messaging, Malware and Mobile Anti-Abuse Working Group) since its establishment. He is the chair of JPAAWG (Japan Anti-Abuse
Working Group). He is acting chairperson of the Anti-Spam mail Promotion Council (ASPC) and a member of its administrative group, as well
as chief examiner for the Technology Workgroup. He is chairman of the Internet Association Japan’s Anti-Spam Measures Committee
and a visiting researcher at the association. He is a cooperating researcher at the University of Electro-Communications.

© Internet Initiative Japan Inc.

19