Internet
Infrastructure
Review
Jun.2024

Vol.

62

Periodic Observation Report

SOC Report
Focused Research (1)Research on a Method of Constructing
Sender Reputation
Focused Research (2)IIJ and the Evolution of Data Centers—Commemorating 30 YearsInternet Inf rast ruc ture Review
June 2024 Vol.62

Executive Summary ........................ ................................................ 3
1. Periodic Observation Report .......... ................................................ 4
1.1
1.2
1.3

Introduction ............................................................................... 42023 Security Summary ................................................................... 4
Security Topics ............................................................................ 7

1.3.1 Challenges in operating and performing analysis

on the Data Analytics Platform and the deployment of dbt ................................... 7

1.3.2 Visualizing C&C communications with a focus on time interval variability ................ 9
1.4

Conclusion ................................ ............................................... 13

2. Focused Research (1) ................................................................. 14
2.1
2.2
2.3
2.4

Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Sender Reputation . ...................................................................... 14
Characteristics of Sender Authentication Technologies ................................... 15

A Method of Constructing Sender Reputation ............................................... 15

2.4.1 The Nature of Forwarded Email ........................................................... 15
2.4.2 Forwarded Email and Sender Authentication ............................................... 16
2.4.3 Assessing Forwarded Email Source using Sender Authentication Results . ............. 16
2.5
2.6
2.7

Constructing and Verifying Sender Reputationn . .......................................... 17
Discussion . ............................... ............................................... 18
Conclusion ................................ ............................................... 19

3. Focused Research (2) ................................................................. 20
3.1
3.2
3.3
3.4
3.5

2

1990s—It Started with the Effective Use of Space ......................................... 20
2000s—Internet Data Centers ............................................................ 20
2010s—Accommodating Cloud Services ... ............................................... 21

2020s—Next-generation Data Centers ................................................... 23
Conclusion ................................ ............................................... 27© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

Executive Summary

Executive Summary

We present IIR Vol. 62, the first edition for 2024. For Japan, the year began with a huge earthquake that hit the
Noto Peninsula. We offer our deepest sympathies to everyone affected by the 2024 Noto Peninsula earthquake.
On New Year’s Day afternoon, a magnitude 7.6 earthquake with a maximum seismic intensity of 7 destroyed local residents’ infrastructure in the blink of an eye. Telecommunications infrastructure was no exception. The four mobile carriers held a joint press conference to disclose the damage caused by the earthquake, including power outages to buildings and base stations and the disconnection of trunk and backhaul circuits. As has been pointed out during previous
disasters, the ability to use the Internet via smartphone as one would during normal times is crucial during emergencies. Reports coming out of the press conference, too, highlighted the fact that mobile communications are essential tousers, and that electrical power and optical fiber constitute the foundations on which such communications depend.
Physical steps, such as deploying mobile power supply vehicles and generators, are needed to furnish electric power,
but in this case, satellite communications stepped in as an alternative to backhaul circuits. Satellite mobile phones
have also been used during past disasters, but in the case of the recent earthquake, high-speed data communications
using low-orbit satellites were made available to function as base station backhaul circuits and Wi-Fi hotspots, andreports indicate this to have been extremely useful. This experience showed that, even in the broadband era, satellite-based wireless communications can be used to supplement optical fiber and terrestrial wireless communications.
The IIR introduces the wide range of technology that IIJ researches and develops, comprising periodic observation
reports that provide an outline of various data IIJ obtains through the daily operation of services, as well as focusedresearch examining specific areas of technology.
Chapter 1 presents our SOC Report, our periodic observation report for this edition. As usual, this report looks at security
incidents on which IIJ’s SOC focused its attention from among those that occurred in 2023. This is followed by two
discussions looking at the SOC’s efforts to address certain challenges. The first looks at challenges with the Data
Analytics Platform that the SOC operates and the introduction of a tool called dbt. The second discusses a methodof enabling the visualization of C&C communications with a focus on time interval variability. I think you will findboth to be interesting accounts of some of the actual work that IIJ’s SOC carries out.
Our focused research report in Chapter 2 discusses a method of constructing sender reputation based on a paper titled“Sender Reputation Construction Method And Feedback Loop Using Sender Authentication,” presented at the Information
Processing Society of Japan. Email has been in widespread use since the Internet’s early days, and that remains true
today, and so there are strong social reasons for maintaining the security of email. The method proposed here onlyuses the results of sender authentication. There is no need to look at the content of email, which makes the method
valuable from a privacy perspective as well. The team behind this method obtained good results when testing it on real-world
emails too, which seems to validate the effectiveness of send authentication technology here.
The focused research report in Chapter 3 continues our series commemorating IIJ’s 30-year history, this time with a lookat data centers. The article walks through IIJ’s data center initiatives through the years, from the 1990s and the dawn of
the Internet era through to the 2020s and the rise of demands for carbon neutrality and AI in the data center. Technology
has evolved substantially over the past 30 years, and the Internet and society at large have undergone huge changes.
Data centers are an important piece of the infrastructure that has underpinned those changes, and we look back on those
30 years with the strong sense that they will continue to play that role in the future. I encourage you to read through.
Through activities such as these, IIJ strives to improve and develop its services on a daily basis while maintaining
the stability of the Internet. We will continue to provide a variety of services and solutions that our customers cantake full advantage of as infrastructure for their corporate activities.

Junichi Shimagami
Mr. Shimagami is a Senior Executive Officer and the CTO of IIJ. His interest in the Internet led to him joining IIJ in
September 1996. After engaging in the design and construction of the A-Bone Asia region network spearheaded by IIJ,
as well as IIJ’s backbone network, he was put in charge of IIJ network services. Since 2015, he has been responsible
for network, cloud, and security technology across the board as CTO. In April 2017, he became chairman of theTelecom Services Association of Japan’s MVNO Council, stepping down from that post in May 2023. In June 2021,
he also became a vice-chairman of the association.

© Internet Initiative Japan Inc.

3

1. Periodic Observation Report

SOC Report

1.1 Introduction

operating the Data Analytics Platform under the headings

IIJ maintains the wizSafe security brand and works constantly“Challenges in operating and performing analysis on theto create an environment in which the Internet can be

Data Analytics Platform and the deployment of dbt” andused safely. One of its initiatives in this regard is the“Visualizing C&C communications with a focus on time

regular dissemination of information on security issues

interval variability”. As part of this discussion, Sectionin blog form via wizSafe Security Signal . IIJ also uses

1.3.1 describes the events that led us to deploy dbt toits position as an ISP to perform data analysis on its Data

address the issues of data freshness and data mart creation

Analytics Platform , which aggregates backbone traffic

and maintenance. And in Section 1.3.2, we share the results

as well as security appliance logs, focusing its effortsof our evaluation of the use of a statistical measure called

on preventive measures and swift follow-up responses

the moving coefficient of variation to deal with three issues

against the increasingly sophisticated cyberattacks out

related to the evaluation of communications time interval

there.

variability.

Section 1.2 of this report looks back at security incidents

1.2 2023 Security Summarythat occurred in 2023 and provides a security summary

Tables 1 and 2 show the security incidents that the SOC

covering key incidents of note in calendar format. Section

focused on from among those that rose to prominence in

1.3 then discusses our latest efforts to deal with emerging

2023.

*1

*2

challenges faced when performing data analysis and

4*1

wizSafe Security Signal (https://wizsafe.iij.ad.jp/).

*2

Internet Infrastructure Review (IIR) Vol. 38 (https://www.iij.ad.jp/en/dev/iir/038.html).

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

1. Periodic Observation Report

Table 1: Incident Calendar (January–June)
Month

Summary

January

Access to a local government’s official website was disrupted, and it was revealed that a group calling itself Anonymous may have been to blame
for the disruption (DDoS attack). A user claiming to be Anonymous posted content on Twitter (now X) hinting at the attack.

January

IIJ’s SOC observed an increase in attack emails with Microsoft OneNote attachments. Multiple malware infection campaigns using OneNote files
were seen from January onward.

February

Multiple organizations reported ransomware campaigns (ESXiArgs) exploiting an OpenSLP heap-overflow vulnerability (CVE-2021-21974) inVMware ESXi products. IIJ’s SOC also confirmed there to be an increase in scanning on port 427, which is used by OpenSLP, around the same time.

February

A crypto asset exchange disclosed that its employees had been subject to a social engineering attack. The attacker targeted multiple employees
via smishing and attempted to log into the exchange’s internal network using stolen employee credentials but was blocked by multi-factor
authentication (MFA). The attacker attempted to keep the attack going by calling the employees and posing as an IT staff member, but theexchange’s CSIRT was able to intervene and foil the attack.

February

Ransomware group Clop claimed to have stolen data from over 130 organizations by exploiting a zero-day vulnerability in the file transfer tool
GoAnywhere MFT. The vulnerability is identified as CVE-2023-0669. Thereafter, several organizations, including Japanese companies, announced
that they have been impacted.

March

The JPCERT Coordination Center (JPCERT/CC) issued an alert on mailouts aiming to infect systems with Emotet, which had not been observed
since November 2022. In some cases, the attached ZIP files contained a Word file that exceeded 500MB once extracted, which is likely acontrivance designed to avoid detection by security products(Note 1).

March

A foreign company that develops business communications software revealed there to be a risk of information-stealing malware infections fromthe official installer of software it provides. An investigating security vendor subsequently reported that the company had suffered a supply chain
attack, compromising its software development environment, which led to other supply chain attacks.

March

A telecommunications carrier disclosed that customer information associated with personal internet services and video streaming services had
been leaked from one of its contractors. A follow-up report revealed that the cause of the breach was not a malware infection but that a former
temp employee who worked on the services at the contractor had exfiltrated the information.

March

A company that provides information communications and system integration services disclosed that an incident occurred in which a local
government certificate issuing service it provides was issuing certificates to residents other than the certificate applicants. The service wastemporarily suspended, but in the subsequent process of fixing the system program that caused the incident, it was found that the patch had notbeen applied at some local governments, so the company said it was carrying out inspections and moving quickly to apply the patch.

April

An IT company that works on public services as a contractor disclosed that servers used in solution services it provides for local government
meetings had been subject to unauthorized access. The unauthorized access prompted several local governments to report a temporar ysuspension of Internet-based meeting live-streaming services.

May

A major automaker’s business strategy firm disclosed that customer information, including vehicle data and footage captured by drive recorders,
had been made externally accessible due to a cloud misconfiguration. A subsequent investigation of other cloud environments revealed new casesof some customer information having been made accessible.

May

Progress Software disclosed that the web application of its MOVEit Transfer file transfer service contained an SQL injection vulnerability(CVE-2023-34362), and that the vulnerability had already been exploited. The company initially advised customers to check for indicators of
unauthorized access over “at least the past 30 days”, but other security organizations subsequently disclosed that exploitation dated back to morethan 30 days prior.

June

For t inet disclosed t hat t he SSL-V PN func t ionali t y of For t iOS and For t iProx y cont ained a heap - based buf fer over f low vulnerabili t y(CVE-2023-27997), and that it may have been exploited in some limited cases. Multiple security vendors reported that the attack group Volt
Typhoon uses a zero-day vulnerability in Fortinet products to gain initial access, but it was revealed that no evidence of the heap-based buffer
overflow vulnerability being used had been found.

June

From June, there were ongoing reports of people who had made bookings through a travel booking site receiving messages directing them tophishing sites. This was traced back to unauthorized access to the accommodation booking information management system, and it was revealed
that devices used to manage the system at some accommodation facilities were infected with malware. Multiple security vendors reported that theattacker was posing as a customer and sending enquiry emails and the like to get operators to open attachments that would infect the devices withinformation-stealing malware.

Note 1: Alert Regarding Re-emergence of Emotet Malware Infection Activities (https://www.jpcert.or.jp/english/at/2022/at220006.html).

© Internet Initiative Japan Inc.

5

Table 2: Incident Calendar (July–December)
Month

Summary

July

A port transportation group announced that its terminal system was down due to a ransomware infection and that it was working to restore it.
After the outage, media outlets reported that the group had received threats from an attack group going by the name Lockbit. Operations wereresumed in short order, over the course of about two and a half days.

July

Microsoft disclosed that the attack group Storm-0558 had gained unauthorized access to Exchange Online and Outlook.com and was able toaccess emails and personal accounts of around 25 organizations, including government agencies. It revealed that the group had used a Microsoft
Service Account (MSA) signing key to forge authentication tokens that enabled it to access the services. While it did not have any concrete
evidence as to how the group obtained the signing key, Microsoft issued a subsequent report that described likely methods.

AugustMedia outlets reported that an Indonesian man had been arrested for using a phishing kit called 16shop to steal and fraudulently use otherpeople’s credit card information. This was the first case in which Japan’s National Police Agency was involved in a joint cyber investigation withanother country: it worked with INTERPOL and Indonesian police to arrest the suspect.

AugustThere were multiple incidents of login screen tampering on mobile network-compatible IoT routers used in Japan. IIJ’s SOC confirmed that theaffected login screens contained content protesting the discharge of treated water from nuclear power plants. An account on X thought to be
involved in the tampering posted an explanation of the vulnerability exploited.

September

A security company revealed that when AI researchers published open-source AI models on GitHub, a misconfiguration of the tokens that grant
access to the public data meant that 38TB of data, including researchers’ computer backups, was also exposed. It also revealed that the exposed
data included passwords to the AI researchers’ organization’s services, secret keys, and employees’ internal messages.

September

It was revealed that a domain used by a remittance and payment service had been listed on a domain registrar’s auction site, causing a stir, withpeople raising concerns of the domain potentially being acquired and misused by a third party. After speaking to the company that owned thedomain, some media outlets reported that the incident was due to internal mishandling.

September

The Zero Day Initiative (ZDI), a vulnerability discovery community, disclosed a vulnerability (CVE-2023-42115) in the mail transfer agent (MTA) Exim
that could allow remote code execution. The vulnerability had been reported by a security researcher in June of the previous year, and in
September, ZDI informed the vendor that it intended to publish it as a zero-day advisory, and subsequently did so(Note 2).

October

Google, Cloudflare, and Amazon Web Services all announced that a large-scale DDoS attack exploiting a vulnerability in the HTTP/2 protocol(CVE-2023- 44487) had been obser ved in August. The vulnerability exploits the HTTP/2 protocol’s stream multiplexing, which enables theconcurrent processing of multiple HTTP requests and responses within a single TCP connection. The attacking client sends a high volume of
HEADERS frames and RST_STREAM frames to drain server resources. This is called an HTTP/2 Rapid Reset attack.

October

Cisco Systems disclosed a vulnerability (CVE-2023-20198) in the web UI feature of Cisco IOS XE. By exploiting this vulnerability, an attacker canobtain the hightest level of access without authentication and create new local users. A few days later, it disclosed another vulnerability(CVE-2023-20273) in the web UI feature that would allow local users to execute commands with root privileges. Reports of damage from attacks
combining these two vulnerabilities were reported in Japan as well, and IIJ’s SOC observed traffic related to this vulnerability.

October

A contact center company disclosed that a person involved in the operation and maintenance of the call center system had been exfiltrating
customer information and leaking it to third parties. This breach is thought to have been ongoing for around 10 years. The root cause was that it
was possible to download customer information on operation and maintenance devices and write it to external storage media, and these
operations were not detectable via timely detection measures or routine log checks.

October

A company that provides identity management and authentication services disclosed that its support case management system had been subject
to unauthorized access. It initially said that only files uploaded by some customers had been affected, but it subsequently revealed that all users
had been affected. An investigation into the account used for unauthorized access revealed that an employee had signed in to a personal Google
account via a device managed by the company, and the account information had been synced to the employee’s personal device, suggesting thatthe account may have been stolen because the personal device was compromised.

November

It was disclosed that ownCloud, open-source software for building online storage, contained a vulnerability (CVE-2023- 49103) that allows
confidential information and settings in containerized environments to be leaked. Some days after this was revealed, several organizations
reported that the vulnerability was being exploited.

November

Akamai’s SIRT disclosed that a Mirai variant called InfectedSlurs was using a zero-day vulnerability in attacks designed to build out DDoS botnets.
A follow-up report revealed that one of the products with this vulnerability is a wall-jack-compatible wireless LAN router sold in Japan, and that it
is possible to exploit the authentication details in this device’s factory default settings. IIJ has observed DDoS attacks emanating from suchbotnets as ongoing since November.

December

Microsoft and Arkose Labs announced they had seized infrastructure used by Storm-1152, a group that sells fraudulent Microsoft accounts and
CAPTCHA bypass tools, identified the ringleaders, and filed criminal charges with law enforcement. It was also revealed that Storm-1152 had soldaround 750 million fraudulent Microsoft accounts, and was providing tutorial videos and support chat services for the tools it was selling.

December

Security consulting firm SEC Consult disclosed an SMTP implementation vulnerability called SMTP smuggling. The SMTP protocol defines an
end-of-data sequence, but many products will recognize a sequence as signaling the end of data even when an email message deviates from thestandard sequence. So by adding a second email message in the data following the sequence, an attacker can, for example, bypass sender
spoofing checks that would normally occur per the SPF sender authentication method.

Note 2: (0Day) Exim AUTH Out-Of-Bounds Write Remote Code Execution Vulnerability (https://www.zerodayinitiative.com/advisories/ZDI-23-1469/).

6© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

1. Periodic Observation Report

1.3 Security Topics

While more fine-grained distinctions may be used at times,

This section discusses the IIJ SOC’s efforts to tackle somehere we will say that a data engineer is someone whoof the challenges it faces.

creates the data platform, and a data analyst is someone
who uses the data platform. With IIJ’s Data Analytics

1.3.1 Challenges in operating and performing analysis on
the Data Analytics Platform and the deployment of dbt
At IIJ, we operate a data platform that we call our Data

Platform too, we break the overall staff up into a teamresponsible for development and a team responsible foranalysis, and those teams collaborate with each other.

Analytics Platform, with the objective being to implement
appropriate preventive measures and post-incident re-

The current mainstream approach is to manage the data

sponses to cybersecurity threats. For additional details

stored on data platforms using the following layeredon these initiatives, refer also to the SOC Report in IIR Vol.

structure, and we manage data on our Data Analytics

38 (https://www.iij.ad.jp/en/dev/iir/038.html), published

Platform in a similar manner (Figure 1).

March 2018. We continue to operate the Data Analytics
Platform, continuously rolling out software updates and• Data lake layer

expanding data sources. Here, we describe some challenges• Data warehouse layerthat have emerged through the operation of the platform• Data mart layer

and our efforts to introduce open-source software called
dbt (data build tool) to address them.

The data lake layer stores data collected from or transferredby data sources, which generate the data, and to the extent

Before delving into the challenges, an explanation of thepossible, keeps it in the form in which it was received. The

underlying data platform operations and architecture isdata warehouse layer transforms data from the data lake

in order. Note that the discussion here is not limited tolayer into structured data to facilitate analysis. The data

the Data Analytics Platform. Firstly, in general, work onmart layer transform data from the data warehouse layer

the data platform is performed by team members serving

into specific forms tailored to specific use cases.

in the following two types of roles.
The data mart layer will be relevant to the challenges• Data engineer

discussed below, so an example from the domain of se-• Data analyst

curity is in order. Say that data collected from an IPS/IDS on a particular data platform is imported into the

Data source

Collect/transfer

Transform

Collect/transfer

Transform

Log

Collect/transfer

Data lake layer

Data warehouse layer

Data mart layer

Email filtering

Figure 1: Illustration of How Data is Managed in a Layered Structure on a Data Platform© Internet Initiative Japan Inc.

7

data warehouse layer. And let’s say that a particular data

The second challenge relates to creating and maintaining

point (record) contains information about the signature ofdata marts. Data marts are crucial when it comes to routine

and time at which a threat was detected. If you want totasks that are performed over and over again. This is because

look at daily trends based on this data, you might consider

the presence or absence of a data mart can greatly impact

producing daily counts of the type and number of detected

on work efficiency.

signatures. In this scenario, data counts that have been
generated in advance, rather than created ad hoc by a data

On the other hand, when creating data marts on the Data

analyst, constitute a data mart. For important information

Analytics Platform, there was a tendency for engineers tothat is frequently looked up, you might also consider

develop them individually. With this style of development,

adding visualization features using a dashboard or the like.

significant resources are spent on everything from development through to maintenance. And the lead time required to

Next, we look at two challenges that have emerged in

actually put the data mart into use tends to be long.

the course of operating and performing analysis on the
Data Analytics Platform. The first relates to data quality.

To address this challenge on our Data Analytics Platform,

For data analysts to perform appropriate analyses, data

we deployed the open-source version of dbt, called dbt

on the platform must remain in a healthy state. Yet there

Core. A SaaS version called dbt Cloud is also available.

is no such thing as a perfect system, and on our Data

dbt is specifically geared to the T (data transformation) in

Analytics Platform, we experienced the following types

ETL (extract, transform, load), the basic functionality of aof events.

data platform. One of its key traits is that most features
are available simply via SQL and YAML configurations*5.

• Decline in freshness of certain data

The fact that SQL and YAML configurations are all that is• Unexpected circumstances befalling certain dataneeded is quite a boon for data analysts, who tend to use
SQL extensively when performing analyses.

The concept of data freshness concerns whether data
is imported into the data platform in a timely manner

The first challenge related to data quality is addressed bywithout any delays. A decline in data freshness occurs

testing the data using dbt. Automated testing is a bestwhen data is imported into the data platform slower

practice in software engineering. And this is also a valid

than expected or when the data flow temporarily stops.

approach in data engineering.

A decline in data freshness may mean that data analysts
are unable to perform timely analysis*3.

dbt includes functionality for testing data, and the tests
can also be written in SQL and YAML. Tests in dbt are

Saying that unexpected circumstances can befall data

SELECT statements that return 0 records in a normallyis somewhat vague, but what this means is that, fromfunctioning system. In other words, if the SELECT statement

a data analyst’s perspective, something appears to be

for a particular test returns one or more records, the test

amiss with the data. Easy-to-visualize examples include

fails. You can of course write your own SELECT state-values that are supposed to be unique appearing multiple

ments from scratch, or you can use built-in test macros

times and unexpected values popping up in the data*4.

and test macros implemented by third-party plugins.

Overlooking these sorts of phenomena can affect theAdditionally, the source freshness feature can be usedreliability of analysis results.

to detect when data freshness has declined.

*3*4

The degree to which a decline in freshness can be tolerated depends on the nature of the work performed by the data analyst.
Situations that actually occur range from abnormalities that are obvious from the data definition to cases that are difficult to detect, such as changes in the distributionof data when viewed through summary statistics.

*58Differences between databases (SQL dialect etc.) are handled by adapters tailored to each database.

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

1. Periodic Observation Report

In the past, if something unexpected had happened tofrom what they were before. And laying out a more concrete

data on the Data Analytics Platform, data analysts would

path for creating data marts has streamlined our operations

typically notice this during the course of performing

and made it easier to come up with new ideas for analysis.

analyses. But when data analysts perform analyses,
they do this with an objective in mind. That is, theyusually need that data right at that moment. So discovering

1.3.2 Visualizing C&C communications with a focus ontime interval variability

something unexpected with the data at that point tended

When infecting a system, some types of malware use an

to heavily impact on workflow. Deploying dbt, however,Internet-connected command and control (C&C) server tomade it possible to efficiently discover and address suchreceive instructions from an attacker. The basic behavior

occurrences by incorporating data definitions and dataof the communications that take place (the C&C commu-

analysts’ experience-based rules into automated tests.

nications) is predetermined according to what malware
program is running. For this reason, certain patterns are

The second challenge relating to creating and maintaining

more likely to emerge than those seen in, for example,

data marts is addressed by using dbt to transform data

communications that occur when humans use a browser

with SQL. In dbt, objects called models can be definedor other such application in an ad hoc fashion.

using SQL SELECT statements. Results transformed by
the model’s SELECT statement can be accessed in the

One key characteristic that tends to reflect particularform of views and tables. The Jinja templating language

patterns is the communication time interval. A typical

can also be used to define models. So it is possible to groupexample that should make this easy to see is polling thatfrequently used content into macros, and to perform looping

occurs at regular intervals, such as a heartbeat used

and branching that is difficult to accomplish in pure SQL.

in system health monitoring. Time interval variability
tends to be low with these sorts of communications.

Deploying dbt on the Data Analytics Platform has madeTo enable security analysts to efficiently capture C&C

it possible, in many cases, to create data marts simply

communication patterns, we therefore tried out the use

by writing SQL. In some cases, we have been able toof a statistical measure that focuses on time interval

create data marts for quickly searching for security

variability to generate visualizations of communications.

IoCs (Indicators of Compromise) within a short period oftime, making certain tasks dozens of times more efficient.

It must be noted, however, that small time interval

Some processes are difficult to implement using SQL alone,variations can occur even with legitimate, non-malware

however, so individual development has not been renderedapplications. If you detect small communication timeentirely unnecessary. Meanwhile, it has become easier thaninterval variations in a production environment, it’s best

before to divide labor between data engineers and datato assume that most of that is attributable to legitimate

analysts in some respects. Specifically, data engineers canapplications. The method described here is thus intended

implement processes that are difficult to perform using SQL

to be used to help analysts visually understand what’salone as SQL user-defined functions. Data analysts cangoing on when breaches occur.

then write routines for transforming the data in dbt using
SQL that includes those user-defined functions. With thisFirst, let’s go over the problem and the tools that canworkflow, tasks follow an existing framework, so we canhelp us solve it. C&C communications can be viewed as

expect to reduce resource requirements and lead times

events that occur over the passage of time. Each event© Internet Initiative Japan Inc.

9

belongs to a specific session*6, typically identified by■ (b) Outliers have a substantial impact

the below elements in combination. The source is theIf, for example, a malware-infected computer is usedsystem infected with malware, and the destination is afor business purposes, it might only be turned on during

C&C server set up by the attacker. Note that the source

weekday daytimes. And naturally, when it’s not powered

port number is not included in the session identifier

on, no C&C communications will occur. Including the

because it may be an ephemeral port that changes fromtimes when the computer was off in the calculations

connection to connection.

would result in a long interval between events, as if.

the readings were outliers. The calculated variance andstandard deviation are likely to be larger than expected• Source IP address

as a result.

• Destination IP address■ (c) Event time intervals can change partway through• Transport layer protocol• Destination port number

sessions
Some malware shortens the time interval between events

Next, we turn to the descriptive statistical measures ofwhen the attacker is actively issuing commands. This isvariance and standard deviation, which is based on variance,

possibly because a lot of malware has autonomy overto quantify the variability in our figures. However, there arewhen it goes to the C&C server to receive commands*7,

some challenges to address when it comes to assessing

so if the polling interval is long, it will take time for opera-

variability in communication time intervals with variancetions to be reflected in the malware’s behavior. When longand standard deviation using all events in a particularand short time intervals are mixed like this, it is difficult

session.

to express the variability in terms of a single measure likevariance or standard deviation.

■ (a) Comparing values from different sessions is difficult
Consider the following situation, for example. Say that in

To address these challenges, we looked at a statisticalsession A, the average time interval between events ismeasure called the moving coefficient of variation. This

100 seconds and the standard deviation (a measure of

takes the ordinary coefficient of variation and introduces

variability) is 10 seconds. Meanwhile, say that in session

the idea of moving it over time. The smaller the moving

B, the average time interval between events is 1,000

coefficient of variation of the communications time interval,

seconds and the standard deviation is 50 seconds. If wethe more we can say that the communications have low

only compare the absolute values, we will decide thatvariability and are thus regular.

the standard deviation is lower for session A than for

10

session B. But if we consider the magnitude of variability

First, the coefficient of variation is obtained by dividing

relative to the average, we will find this to be smaller

the standard deviation by the mean. As mentioned infor session B than for session A. Hence, variance andchallenge (a) above, the standard deviation alone is notstandard deviation alone are insufficient when comparing

enough to determine variability relative to the mean size

different sessions.

of the observation. We therefore obtain a dimensionless*6

For convenience, we refer to a series of exchanges between any particular nodes as a session. This is different from a TCP session.

*7

This is possibly because if the communications originate from a C&C server on the Internet, they are likely to be hindered or blocked by NAT and firewalls.

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

1. Periodic Observation Reportmeasure by dividing the standard deviation by the mean.

We will now look at how malware-generated C&C commu-

This makes it possible to compare the variability ofnications from past infection incidents can be visualized

different sessions.

using the moving coefficient of variation. The first casedeals with malware communicating with multiple C&C

Next, in statistics, a moving measure is something thatservers at the same time. In recent times, malware

is calculated using a moving set of local (as opposed toprograms often attempt to communicate with multiple

global) data points. Such measures are used mainly for

C&C servers at the same time in the manner discussed

time series and other continuous data sets and are based

here.

on a predetermined number of data points. The range
covered by that predetermined number of data points is

In Figure 3, time elapsed since the reference time is plottedcalled the window. The window is progressively moved

on the horizontal axis, and the moving coefficient of vari-along the data set so that all of the data is eventuallyation is plotted on the vertical axis. Each line represents aused.

session, all of which constituted communications to a C&C

So the moving coefficient of variation is obtained bymoving the calculation of the coefficient of variation
along the data set. The moving coefficient of variation
is calculated by dividing the moving standard deviationby the moving average. This reduces the impact of theregards (b), outliers only affect the windows in which
they occur. And as for (c), even if the time intervals at
which events occur change during the observation period,
this should show up as changes in the statistical measureover time.

Destination IP address AS number A
Moving coefficient of variation

issues discussed in challenges (b) and (c) above. As

3
2

1
0
Destination IP address AS number B
3

2

1

Attacker0
Destination IP address AS number C

Commands
3

C&C servers

2

1

0

Internet

Destination IP address AS number D
3

C&C communications

2

1

0
500

Infected device

1000

1500

2000

2500

Time elapsed since reference time (seconds)

Figure 2: Malware Communicatingwith Multiple C&C Servers at the Same Time

Figure 3: Typical Sawtooth Wave Patternof Malware Communications to C&C Servers© Internet Initiative Japan Inc.

11

server. In this case, each session used the same source

In Figure 5, as before, time elapsed since the reference

IP address and transport layer protocol but had different

time is plotted on the horizontal axis, and the moving

destination IP addresses and port numbers. For ease ofcoefficient of variation is plotted on the vertical axis.

viewing, the graph is divided into panels by the AS number

Each line again represents a session, and the windowof the destination IP address. The window size used tosize used to calculate the moving coefficient of variation

calculate the moving coefficient of variation was 10 data

is 10 data points here also. Note, however, that elapsed

points.

time is calculated in minutes here, as opposed to seconds
in the previous graphs.

The graphs in each panel of Figure 3 show the typicalsawtooth wave pattern, oscillating around a value of 2,

Two sessions are plotted on the graph, and you can see

in the moving coefficient of variation. The waves are nottimes during which the moving coefficient of variation

perfectly aligned with each other, but they are close.

remained stable at close to zero in both cases. In other

This means that attempts to communicate with C&C

words, there are times when communication was occuring

servers happened at similar times, even across different

at regular intervals such that there was very little vari-

C&C servers.

ability in the communication time interval. In fact, during
these times, the malware was repeatedly communicat-

The second example shows malware falling back to

ing with a C&C server at intervals of around 317–320

another C&C server when the C&C server it was originallyseconds.

communicating with becomes unavailable. While thismalware only communicates with one C&C server at a

In the session indicated by the orange line, the mov-time, it likely has the ability to switch to a different C&C

ing coefficient of variation increases from around theserver if it loses contact with its current C&C server for

200-second mark*8 and the line subsequently cuts out.

some set amount of time.

A little while after this, the purple line appears. This is

Attacker
Commands
Communications with C&C server fail2

Moving coefficient of variation

C&C servers

Internet

C&C communications

Fallback

1

0

0

100

Infected device
Figure 4: Malware Falling Back to Another C&C Server

*8

300

400

500

600

700

Figure 5: Plot of Communications
When Malware Falls Back to Another C&C Server

An increase in the moving coefficient of variation indicates a breakdown in the regularity of communication intervals partway through the session. This corresponds
to challenge (c) above.

12

200

Time elapsed since reference time (minutes)© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

1. Periodic Observation Reportwhat it looks like when the malware falls back to another

1.4 Conclusion

C&C server.

In the orange session, communications

In this article, we discussed several incidents that ourwith the C&C server had failed about an hour before the

SOC focused on out of those that we observed in 2023.

line cut out. The malware, it seems, therefore switched

The annual summary in Section 1.2 indicated that soft-

to sending its communications to a different C&C server.

ware vulnerabilities continue to come to light, and thatransomware attacks as well as information breaches due

Note that the examples covered here are typical ones andto configuration errors and external attacks continue tothat there is a whole range of malware communication

occur. Section 1.3.1 described how we used dbt to address

patterns in the wild. It is not uncommon, for instance,

issues related to data freshness and quality and the creation

for malware to vary communication intervals depending

and maintenance of data marts through automated testingon which C&C server it is communicating with, or toand the use of SQL for data conversion in a way that does

introduce some degree of randomness. These sorts of

not require individual development. Finally, Section 1.3.2

behaviors are likely attempts to avoid detection based on

discussed how we arrived at the idea of using the moving

communications time interval variability. We have observed

coefficient of variation, along with some actual use cases,

that, even in such cases, the shape of the graphs of the

as a means of addressing challenges that arise whenmoving coefficient of variation exhibit similarities, andassessing the variability in communications time intervals

that the coefficient will move within a particular range.

based on variance and standard deviation—namely, thechallenges of comparing different sessions, addressing

Here, we have described our work to enable visualizations

the impact of outliers, and addressing changes in timeof communications traffic using a statistical measureintervals.

that focuses on time interval variability. IIJ’s SOC willcontinue to study techniques for efficient data-based

IIJ will continue to publish information to address the

security analysis.

ever-changing array of threats out there. We hope thatyou will continue to turn to the IIR and wizSafe Security
Signal for such information and that it will prove useful
to you in your security responses and operations.

Junya Yamaguchi
Data Analytics Section, Security Operation Department, Advanced Security Division, IIJ

Hiroyuki Kamogawa
Data Analytics Section, Security Operation Department, Advanced Security Division, IIJ

Satoshi Kobayashi
Data Analytics Section, Security Operation Department, Advanced Security Division, IIJ© Internet Initiative Japan Inc.

13

2. Focused Research (1)Research on a Method of Constructing Sender
Reputation
2.1 Introduction

paper*7 published in the journal of the Information Processing

Twenty years ago, in January 2004, IIJ joined MAAWG

Society of Japan. The paper describes a sender reputation(Messaging Anti-Abuse Working Group), which works on

construction method and feedback loop. In this article,

measures to combat spam globally. I participated in the

I focus on the sender reputation aspect of the paper. The

group’s founding meeting in April 2004, and I have con-

paper was also recognized as a specially selected paper bytinued to attend the general meetings since. The group’s

the Information Processing Society of Japan.

name has been changed slightly to M3 AAWG*1, and thescope of its activities has been expanded. The 60th general

2.2 Sender Reputation

meeting, marking the group’s 20th anniversary, took place

DNSBL (DNS Blocklist) has long been used as a mecha-

in February 2024.

nism for determining whether to accept email based on
sender information. It uses a DNS query to look up the

MAAWG technical discussions were initially focused on

source host’s IP address. While the source IP address isevaluating and popularizing sender authentication technolo-

not an appropriate way to identify the sender of an email,gies, particularly SPF

and DKIM , to address the fact that

DNSBL has so far been used because the email address

it was not possible to accurately identify the sender of an

specified in the email headers and during the email de-email, which really should be regarded as a flaw in the email

livery process to indicate who the sender is cannot besystem. Those technical discussions continued over therelied on. With the spread of SPF and DKIM for senderyears, with members of M3 AAWG playing a central role inauthentication, there are moves to use domain names ascreating technical specifications like DMARC , ARC , andauthenticated, trustworthy information for determining

BIMI*6. From the start, technical discussions about sender

whether to accept or reject email—this is the concept ofauthentication were premised on the idea that, as a next

domain-name sender reputation.

*2

*3

*4

*5

step, we would need to be able to determine if we should
accept email using authenticated domain names—in otherIn addition to domains with a negative reputation fromwords, sender reputation. Indeed, the first SPF specifica-

which email should not be accepted, one can also imagine

tion, RFC4408, mentioned domain reputation, and Google

there to be legitimate domain names from which email

and Yahoo in the US have recently been pushing heavily forshould be accepted. When quantified, the factors behindemail senders to get on board with sender authentication tothis determination result in a reputation score. In simpler

tighten up the way incoming email is handled. Indeed, eventerms, reputation can be thought of as the basis for ain Japan the number of domains supporting DMARC rose

Block List and Allow List of domain names.

sharply after the companies announced these tougher measures. As a visiting researcher at IAjapan, I study domain jp

Alongside the rise of sender authentication technology,

names, and as of February 2024, around a quarter of the

we have also seen an increase in the volume of spam fromdomain names used for email had a DMARC record set, indi-

actors who have registered their own domain names andcating a roughly three-fold rise in the proportion of domains

properly configured SPF and DKIM. The domain nameswith such a record.

registered for this type of spam are used as throwaways,
so building a Block List based on domain reputation is,

The IIJ Research Laboratory conducts research on meth-unfortunately, not all that effective. It may thus be more

ods for building sender reputation. This article discusses a

effective to use an approach that involves building an*1

Messaging, Malware and Mobile Anti-Abuse Working Group.

*2

Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1 (RFC7208).

*3

DomainKeys Identified Mail (DKIM) Signatures (RFC6376).

*4

Domain-based Message Authentication, Reporting, and Conformance (RFC7489).

*5

Authenticated Received Chain.

*6

Brand Indicators for Message Identification (Internet-Draft).

*7

Shuji Sakuraba et al., Sender Reputation Construction Method And Feedback Loop Using Sender Authentication, Journal of the Information Processing Society ofJapan, Vol. 64, No. 1, pp. 13–23 (2023).

14© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)Allow List and combining this with email filters and theas the inability to properly authenticate forwarded emails.

like to assess the content of emails not covered by theBut because sending mail servers need to perform the ad-

Allow List. Spam as a proportion of all email has declined

ditional steps of creating the digital signature and adding

from what it was in the past, and so if the bulk of email

the DKIM signature information to emails, DKIM has not

can be assessed using the simpler and easier methods of

become as widespread as SPF.

sender authentication and sender reputation, then morecomputing resources can be devoted to assessing the

2.4 A Method of Constructing Sender Reputation

content of email messages.

Here, I describe a method that uses sender authentication
to collect SPF-authenticated domain names from which

With this background now laid out, this article describes

email should be accepted. In general, it is relatively easy

a method for collecting the domain names of legitimate

to collect this information in the case of spam because

emails and building reputations on that basis.

this sort of email is itself unwanted and shouldn’t be accepted. The approach has been to collect data for spam

2.3 Characteristics of Sender Authentication
Technologies

filters and block lists by extracting salient characteristics

Detailed descriptions of the SPF and DKIM sender authen-comes to legitimate emails, however, a challenge is thattication technologies can be found in sources such as thethe messages may contain highly confidential informa-

Sender Authentication Technology Deployment Manual*8.

tion, making it generally difficult to collect the desired

Here, therefore, I focus on parts of the paper related todata. And because the information is used to determine

the method of constructing sender reputation.

whether or not an email should be accepted, erroneously

and sender information from the collected spam. When it

recording an email sender as a spam emitter can cause
SPF authenticates the domain name of an email address

substantial damage, so accuracy is required when making

representing the sender of an email over the Simple Mailsuch entries.

Transfer Protocol (SMTP). On the sending side, an SPF
record listing email sender IP addresses and the like is

Here, we take forwarded emails to be emails that should

published on the domain’s DNS server, and on the receiv-

be accepted, and describe a method for extracting theing end, when an email is received, the receiving server

senders of forwarded emails and creating a list on thatlooks up the IP address to determine whether the email

basis.

is from the correct sender. This mechanism means that
implementing SPF on sending servers is relatively easy as

2.4.1 The Nature of Forwarded Emailall the administrator needs to do is publish an SPF DNSEmail forwarding is often used as a way of consolidatingrecord, and use of SPF thus continues to spread. One

emails, such as when you use multiple email accounts andproblem, however, is that recipients cannot properly au-

want to view them in a single place. This mechanism has

thenticate emails when they were sent by an entity otherlong been used in email systems such as the opensource

than the original email sender.

Sendmail, which can be configured to automatically redirect received emails by adding the forwarding address toWith DKIM, the server creates a digital signature from thethe .forward file in the user’s home directory. Hence, theemail header and body for each email sent and affixes thisforwarder’s email forwarding settings point to the recipientalong with other relevant information to the email header.

of the forwarded emails, and so from the forwarded email

Because it uses an authentication method that does notrecipient’s perspective, the forwarding email sender can bedepend on the route by which an email was delivered,regarded as an email sender from whom email should be

DKIM is not subject to the problems inherent in SPF, suchaccepted.

*8

Anti-Spam Consultation Center, Japan Data Communications Association, Sender Authentication Technology Deployment Manual (https://www.dekyo.or.jp/soudan/aspc/report.html#dam, in Japanese).

© Internet Initiative Japan Inc.

15

If you can collect a list of such email forwarders, you should

as a means of tightening defenses against email spoofing.

be able to construct reputations for email senders from

For this reason, when forwarding email, some forwarding

which email should be accepted.

sources rewrite the envelope-from field to contain the domain name of the forwarding source. When this is done,

2.4.2 Forwarded Email and Sender Authentication

emails will pass both SPF and DKIM authentication at theIn basic email forwarding, the email address set by theforwarding destination. However, the domain names authen-original sender is used in the envelope-from field*9, whichticated in this case usually differ. The results are illustrated

corresponds to the domain authenticated by SPF. Because

in Figure 2.

of this mechanism, SPF authentication at the forwarded
email destination fails. DKIM, meanwhile, does not use the2.4.3 Assessing Forwarded Email Source using Sendersender’s IP address for authentication, so emails to which a

Authentication Results

DKIM signature has been added can be DKIM-authenticatedI have explained that there are two email forwarding meth-at the forwarding destination. The results are illustrated in

ods and that the SPF and DKIM authentication results differ

Figure 1. The SDID (Signing Domain Identifier) in the figure

across those methods. Thus, we use the sender authen-is the domain name authenticated by DKIM.

tication results to determine whether an email has been
forwarded, and collect this as reputational information on

Recently, an increasing number of email-receiving servers are

the forwarded email senders. First, we identify forwarding

refusing to accept emails that cannot be SPF-authenticatedsources that do not rewrite the RFC5321.From field when

RFC5321.From: example.jp

RFC5321.From: example.jp

RFC5321.To: example.com

RFC5321.To: example.org

SDID: example.jp

SDID: example.jp

forwarding mailexample.jp

example.com

example.org

SPF : pass (example.jp)

SPF : fail (example.jp)

DKIM : pass (example.jp)

DKIM : pass (example.jp)

Figure 1: Results of Sender Authentication of Forwarded Email

RFC5321.From: example.jp

RFC5321.From: example.net

RFC5321.To: example.net

RFC5321.To: example.org

SDID: example.jp

SDID: example.jp

rewrite forwarding mailexample.jp

example.net

example.org

SPF : pass (example.jp)

SPF : pass (example.net)

DKIM : pass (example.jp)

DKIM : pass (example.jp)

Figure 2: Sender Authentication of Forwarded Email with Sender Information Rewritten*916

The sender email address according to the email delivery protocol (SMTP). May appear as RFC5321.From, referring to the SMTP RFC.

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)forwarding emails based on the following authentication

related—for instance, usually they will have the same do-

criteria, and thus build a list of SPF-authenticated domains

main name or the same upper domain name. For example,

from which email should be accepted.

DMARC defines an organizational domain name and assumes
that the SPF- or DKIM-authenticated domain name and the• Sending IP addresses for which SPF authentication fails
and DKIM authentication passes

sending domain name in the header are the same or havethe same organizational domain name. Given this specifica-• SPF-authenticated domains that send email from thetion, the SPF- and DKIM-authenticated domain names areabove IP address and for which SPF authentication

closely related, even for ordinary email. If the original email

passessender supports DKIM, and the SPF-authenticated domainname is rewritten when the email is forwarded, it is common

With the first criterion, we collect the source IP address

for there to be no relation between the original DKIM-

of forwarded emails. There can be multiple outgoing mailauthenticated domain name and the SPF-authenticatedservers, so to capture a broader range of legitimate email

domain name at the forwarding destination. To identify for-senders, we collect a list of SPF-authenticated domains for

warding sources that rewrite the envelope-from field when

which email is sent from the forwarded email’s source IP

forwarding email, we focus on the relationship between

address and for which SPF authentication passes, the objec-

SPF- and DKIM-authenticated domain names. To automat-

tive being to identify the administrative origin of sent emails.

ically collect these sender IP addresses, we look at emails

This is the second criterion. Both the forwarding IP address

sent from the same IP address, which pass SPF authen-and the SPF-authenticated domain name sent from that ad-

tication, and for which the email sender IP address yields

dress constitute sender reputation. Such IP addresses are

multiple DKIM-authenticated domain names. These are de-legitimate sources from which email should be accepted,

termined to be the email forwarding sources. These email

so any email sent from such addresses, including email thatforwarding source IP addresses and the SPF-authenticatedhas not been forwarded, is considered acceptable. Thisdomain names that they send constitute sender reputation

makes it possible to use not only IP addresses but also SPF-

indicating that emails should be accepted.

authenticated domain names for sender reputation.

2.5 Constructing and Verifying Sender Reputationn
Next, we identify forwarding sources that rewrite the en-

To assess the effectiveness of these methods, we con-velope-from field when forwarding emails according to thestructed sender reputations and applied this to incoming

following criteria and again collect a list of SPF-authenticatedemails. We used the incoming email logs from a real-world

domains from which emails should be accepted.

email service. This email service performs SPF and DKIM
sender authentication when receiving emails, and applies a• Source IP addresses for which both SPF and DKIM

spam filter to all emails, so the results of these operations

authentication passes and the domain names areare available in the logs. We used the results of this spam

unrelatedfilter as the basis for evaluating the judgements made ac-• Of those above, sending IP addresses for which multi-

cording to sender reputation.

ple DKIM-authenticated domain names can be obtained
via DKIM authentication

That is, we construct sender reputation from the SPF and• SPF-authenticated domains sent from the above IP addresses for which SPF authentication passes

DKIM authentication results. Next, we check received email
against the sender reputations, compare it with the resultsof the spam filter, and measure the volume of email classi-

Outgoing emails that have not been forwarded and thatfied as non-spam (ham) and as spam.

support both SPF and DKIM are expected to be closely© Internet Initiative Japan Inc.

17

We constructed sender reputations from around 340 mil-spam by the spam filter for which sender reputation waslion incoming email log entries for the month of September

misapplied. That is, this is the false positive rate (FPR).

2019. At the time, spam accounted for 11.7% of email,This differs from the meaning of a positive result from an

the SPF authentication pass rate was 71.1%, and the DKIM

email filter’s assessment of spam. Here, a positive result

authentication pass rate was 38.1%. From these data,means that email should be accepted, so it is important towe were able to extract 15,169 forwarding IP addresses,

be aware of the relationship between true positives and744,660 SPF domain names sent from ordinary forwarding

false positives based on reputation.

sources, and 11,164 domain names that rewrite the sender

2.6 Discussion

domain name when forwarding email.

By assessing email forwarding source using sender
We applied these sender reputations (indicating that email

authentication technology and constructing sender repu-

should be accepted) to the roughly 36 million emails re-tations based on this, we were able to correctly identify

ceived in the week of October 2019 immediately following

around 58% of acceptable email (ham). At the time, thethe week from which we collected the reputation data

SPF authentication rate was around 70%, so a large(Table 1). We used the same incoming email logs when

portion of that can be attributed to the use of sender

doing this. Differences between two reputation types are

reputation. Detecting forwarding sources that rewrite theshown below.

sender information when forwarding emails and using thisto add to sender reputation increased the effectiveness(1) Ordinary forwarding sources (IPs) and SPF-authenticatedof our method. We were able to increase TPR by overdomain names that do not rewrite sender information

10 points while holding down the increase in FPR to only

when forwarding

0.25pt. During the period to which we applied our sender(2) In addition to (1), sendes (IPs) and SPF-authenticatedreputations, spam accounted for around 9% of received

domain names that rewrite sender information whenemail, so the actual number of false positives was quite

forwarding

low. We also understand, to an extent, why these false
positives occurred, so we think it will be possible to fur-

In Table 1, the ham column indicates the proportion ofther reduce FPR.

email determined not to be spam by the spam filter forwhich sender reputation was successfully applied. In other

This method of constructing sender reputation only

words, this is the true positive rate (TPR). The spam col-uses the results of sender authentication and does not

umn indicates the proportion of email determined to belook at the content of email. Even though it is simple

Table 1: Results of Applying Sender Reputation

18

Reputation

ham(％)

spam(％)

(1)

47.45

3.01

(2)

58.01

3.26

© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

2. Focused Research (1)in comparison with common email filtering methods,

method, which uses information on sender trustworthi-

we still achieved high spam-detection accuracy. We doness to determine whether email should be accepted or

use the results of DKIM authentication, which does notnot, is a significant contribution. The fact that our method

have a high uptake rate, in identifying the sender of for-of constructing sender reputation does not involve looking

warded emails, but the sender does not necessarily haveat the content of emails also makes it valuable from ato support DKIM for reputation to be useful, and we only

privacy perspective. Further, as demonstrated when weuse a few DKIM-authenticated emails to determine for-tested this method, the fact that it makes it possible towarding source. So even though DKIM uptake is low, it

construct sender reputation using, for example, incoming

is possible to construct adequate sender reputations. In

email logs means that it is possible to produce senderconstructing and applying sender reputations, we used

reputations geared to the emails that your organization

SPF-authenticated domain names, which are widespread,

receives, opening up the prospect of greater sorting ac-

and if SPF uptake increases further, this should enable ac-

curacy. While this method may be inapplicable to a small

curate determinations about even more emails. If DKIM or

number of incoming emails, it should make more com-

DMARC uptake were to increase, we could also consider

puting resources available, which could then be used tousing those authenticated domain names for the purposes

perform deeper assessments based on the content ofof sender reputation.

those email and so forth.

The fact that SPF authentication fails at email forwarding

It has long been the case that emails would be delivered

destinations has until now been considered a shortcoming

even if you had not deployed some form of sender au-of SPF authentication. However, we believe the favor-

thentication, and relatively new technologies like DMARC

able results we obtain using the method for constructing

have thus struggled to gain traction. Yet the recent

sender reputation that we describe here—using the fea-

announcement of new countermeasures for incomingtures of SPF as a network-based method and DKIM as aemail from the likes of Google and Yahoo in the US has

digital-signing method—are actually positive for the up-

prompted more uptake of DMARC, as well as SPF andtake of SPF.

DKIM, on which it is based. This will make it possible tocombat email spoofing while also increasing opportunities

2.7 Conclusion

to apply measures of domain reputation. We will continue

With phishing and other forms of spam becoming more

to pursue research relevant to achieving more accurate

sophisticated these days, such that it is hard to tell le-measures of domain reputation.

gitimate emails from malicious ones, we believe that our

Shuji Sakuraba, Ph.D.
Senior Research Engineer, Technology Coordination Office, IIJ Research Laboratory
Dr.Sakuraba is engaged in research and development related to messaging security. He is also involved in various activities in collaboration
with related external organizations aimed at bringing about safe and secure messaging environments.He has been a member of M3 AAWG
(Messaging, Malware and Mobile Anti-Abuse Working Group) since its establishment. He is the chair of JPAAWG (Japan Anti-Abuse
Working Group). He is acting chairperson of the Anti-Spam mail Promotion Council (ASPC) and a member of its administrative group, as well
as chief examiner for the Technology Workgroup. He is chairman of the Internet Association Japan’s Anti-Spam Measures Committee
and a visiting researcher at the association. He is a cooperating researcher at the University of Electro-Communications.

© Internet Initiative Japan Inc.

19

3. Focused Research (2)IIJ and the Evolution of Data Centers—Commemorating 30 Years
3.1 1990s—It Started with the Effective Use ofSpace

ISPs use a huge number of dedicated lines and telephone

Even before the privatization of the Nippon Telegraph andwithin communication stations and connect them toTelephone Public Corporation in 1985 (an event referred to astransmission equipment and exchanges on-site, but NTT’s

Japan’s telecommunications deregulation), system integrators

station facilities were not yet made broadly available,

were engaged in the business of housing other companies’

and because ISPs needed to connect both domesticallycomputers in their computing centers in space freed up

and internationally, they naturally decided to make use

by computer downsizing. International telecommunicationsof the communication stations of KDD and the NCCs.

companies were also in the business of housing dealer phone

NSPIXP-3 was opened in Osaka, and 1997 saw Japansystems and private telephone exchanges for foreign finan-Internet Exchange Co., Ltd. and Internet Multifeed Co.,cial institutions in space freed up in their telecommunications

Ltd. established, marking the beginning of commercial IX

spaces by the downsizing of exchange and transmission

services. This was the era in which Japan’s underlying

equipment. Both types of buildings were sturdier thanInternet framework was formed.

lines, so it makes sense to install network equipment

ordinary buildings, and people recall them as the predecessor
to today’s data center facilities.

The term data center was still not in use at this point. The
services were called “co-location,” because equipment was

Before Japan’s telecommunications deregulation, international

located alongside transmission equipment, and “housing,”

communications were handled exclusively by Kokusai

because they involved housing customer equipment. The

Denshin Denwa Corporation (KDD, now KDDI), but postrooms were heavily cooled and seemed cold and inorganic,

deregulation, several new communications carriers (NCCs)

and at the time, a supply of 1–2KVA per rack was quite

entered the market, and computer communications alsosufficient.

started to spread, driven by Type 2 Communications Carriers(General, Special), which do not have their own transmission

3.2 2000s—Internet Data Centers

equipment.

With the explosive growth of Internet connectivity services
in the late 1990s, all at once many ISPs began leasing

IIJ was founded in 1992 and, after much effort, was licensed

communications station buildings and providing services.

as a Special Type 2 Communications Carrier and thus per-Late in the era, in October 1998, IIJ established a joint

mitted to provide Internet services. IIJ’s first installation

venture with Sony and Toyota Motor Corporation calledof communications equipment was in the KDD Otemachi

Crosswave Communications Inc. (CWC), which set up

Building (now the KDDI Otemachi Building) in Otemachi,a nationwide network as an NCC. The following year, it

within Tokyo’s Chiyoda Ward. That was followed by theopened access points across Japan.

installation of NSPIXP-1 in the basement of Iwanami Shoten,
Publishers’ premises in Kanda Jimbocho, Chiyoda Ward, andAt the same time, we selected buildings with a fairly large

when Internet exchange (IX) interconnectivity tests began,

floor load capacity in major Japanese cities (Sapporo,

IIJ set up routers on dedicated lines. Once NSPIXP-2 was

Sendai, Tokyo, Nagoya, Osaka, and Fukuoka) and set up

subsequently installed at the KDD Otemachi Building,data centers equipped with uninterruptible power supplies

Internet service providers (ISPs) began to congregate atand emergency generators. Later, we built our own datathat location.

centers—encompassing land and structure—in Kawaguchi

Table 1: Differing Features of Computing Centers and Communications Stations
FeatureComputing center

Communications station (telephone & telegraph offices)

Air conditioning

Water cooled (large, general-purpose coolers)

Air cooled (transmission equipment, telephone exchanges)

Power supply

Main supply: AC (3-phase 200V)Generator backup

Main supply: DC (48V DC)

Building structure

Designed on par with office buildings

Designed as a telegraph and telephone office

Free-access flooring

Slab floor

Mainly space leasing

Mainly rack leasing

CVCF also allows AC supply

Provisioning

20© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

3. Focused Research (2)City, Saitama Prefecture, and Yokohama City, Kanagawa

3.3 2010s—Accommodating Cloud Services

Prefecture. Our guiding concept at the time was to create

With cloud services, which provide computing resources

human-friendly data centers, with integrated people flow

over a network, service providers and the like own andlines, cafeterias for break times, conference spaces, kitting

operate the servers and other IT equipment essentially on

rooms, and more. This marked a shift away from inorganic

behalf of users. To achieve high equipment densities andcomputing centers and communications stations to intelli-efficiently run the huge amount of IT equipment involved,

gent buildings for the Internet era.

in the 2010s, operators began building hyperscale data
centers capable of adequately cooling the equipment andWhile it was opening data centers, CWC was also providing

supplying 10KVA or more per rack.

wide area LAN services to which customers could connect
via Ethernet, which was the standard LAN interface. This

Power receiving capacities were now on the order of 50MW,

communications service (called a “virtual building”) allowed

equivalent to some 10,000 ordinary households (on 100V

users to connect as if on a LAN even when physically distant

50A contracts). The facilities were not only large in scale but

and use the network as if they were all in the one building.

also designed to be energy efficient. The rise of hyperscale

Combined with a fee structure that was independent ofdata centers also saw improvements in an indicator of energy

physical distance, this service took the world by storm at

efficiency called PUE (power usage effectiveness; found by

the time, and it is fair to say that this setup has now become

dividing the power consumption of the entire data center

the de facto standard for L2 data services.

by the total power consumption of its IT equipment, witha score of 1 being optimal and the Japan average being

As networks and data centers evolved, companies began

1.7). To build data centers that would accommodate

installing the equipment needed for email services, web

cloud service platforms, IIJ developed the IZmo contain-

services, firewalls for security, and remote access. The

erized modules capable of providing 10KVA per rack in a

facilities attracted not only ISPs but also OTT (over-the-high-density environment with outside-air cooling for hightop) businesses such as content providers, and with theenergy efficiency, and this culminated in IIJ becoming theadvent of rental server services and hosting services,

first in Japan to operate a commercial outside-air-cooled

the facilities became not just places to locate equipment

containerized data center facility in 2011. At the time, IIJ’s

but places for providing the functionality to connectbusiness model involved reselling data center capacity

such systems to the Internet.

rented from other companies. We had no experiencebuilding our own data centers or developing electrical

And thus they came to be called Internet data centers.

and air conditioning equipment, so we had to start fromIn this era, floor load capacities exceeded one ton per

scratch. We visited the data centers of industry leaderssquare meter, multiple power feeds were installed, and(GAFA and the like) in North America and determined thatthe power supply and air conditioning equipment wasoutside-air cooling would be the most effective way to re-designed to facilitate 4–6KVA per rack. The Internet hasduce power consumption, and we settled on the concept

long been likened to a cloud, and it was from aroundof combining this with containerized modules that would

this time that Internet data centers would transform into

allow us to add capacity in stages. Working with our part-cloud service and connectivity hubs.

ners, we designed and built test facilities, and after a year

Table 2: Key Considerations when Constructing a Data Center
Consideration

Details

Building structure

Earthquake-resistant structure, seismic-isolation structure, seismic-isolation flooring

Power receiving system

3-spot network, receipt of power from substations

Power supply

UPS redundancy (standby redundancy, parallel redundancy, common standby method)

Communication circuits

Multi-carrier, carrier neutral, conduit redundancy

Disaster preparedness

Distance from active faults, avoidance of locations directly under air routes, hazard mapping

GroundGround strength (N value), liquefaction potential index

Indicators

PUE, WUE, PLM value, Tier level classification© Internet Initiative Japan Inc.

21

of testing, we were able to build the Matsue Data Center

fact that infrastructure is well developed by virtue of thePark (Matsue DCP).

area having long been a site for computing centers. Even
overseas, though, data centers do tend to cluster in certain

We named IZmo after the Izumo region, where it was firstareas, so Japan is no exception here. Other cluster sites

implemented, and the kanji for “cloud” (which is part of the

include Osaka’s Saito region and the Keihanna area, and withJapanese spelling of Izumo). IZmo’s excellent energy-saving

multiple large-scale development plans underway in Japan,

performance and ease of installation saw it not only used inthis trend can be expected to persist.

Japan but also exported overseas for use at a hydroelectric
power plant in Russia, a national data center in Laos, andAs the scale of IIJ’s business expanded, in 2019, we also

elsewhere. And a project is currently underway to provide

built and began operating a hyperscale data center, dubbed

data centers to Uzbekistan’s national telecommunications

Shiroi Data Center Campus (Shiroi DCC), in the Inzai district

carrier. With edge computing becoming more widespread,(Shiroi City is next to Inzai City). Located on a 40,000sqm site,

we now offer the modules both in Japan and internationallythe facility can be expanded to a maximum power receiving

under the DX edge moniker as a solution for easily andcapacity of 50MW. Matsue DCP can be expanded in container

rapidly building edge computing environments and digital/

increments (nine racks), but to allow for the expansion of

IT infrastructure.

Shiroi DCC’s service infrastructure in larger increments, wefirst constructed a system module building with a 1,000-

Matsue DCP is a medium-sized center with a power receiving

rack capacity, and this is divided into four modules to allow

capacity of 4MW, but from the late 2010s, the construction

for expansion in stages. The use of outside-air cooling systemsof multiple hyperscale data centers for foreign cloud providers

on a large scale also means that the facility achieves excellent

began in earnest. These were initially clustered in the Inzai

energy savings on par with Matsue DCP. In addition to out-

district around Chiba New Town Chuo Station. This areaside-air cooling, Shiroi DCC also uses side-flow systems,

is so well known as a data center hotspot that the namewhich are often used in hyperscale data centers. The low-

Inzai is recognized internationally. This clustering can be

speed, high-capacity airflow of this system is more energy

attributed to a confluence of factors, including the area’s

efficient than the conventional method of vigorously blowing

solid ground, the fact that it is within 30km of Otemachi,

cold air out from under the double floor and creates a com-easy access to a submarine cable landing station, and thefortable working environment.

AHU : Air Handling Unit

Outside-air cooled container module concept at initial planning stage in 2009
(excerpt from internal documents)

Shiroi Data Center Campus, launched in 2019

Figure 1: Creating a New Form of Data Center, Something from Nothing

22© Internet Initiative Japan Inc.

Jun.2024

Vol.

62

3. Focused Research (2)In the 2010s, as cloud services became widespread, the■ Carbon Neutrality

data center industry saw substantial change, including the

The use of data centers helps to save energy across society

ongoing build up of hyperscale data centers and the entry of

as a whole through the centralized, efficient operation offoreign players into the market. In response, IIJ developed

IT equipment, but even so, the energy consumption of thecontainer modules from scratch (Figure 1) and was eventuallyfacilities themselves is a focus of attention. Japan’s revisedable to build its own hyperscale data center. In 2023,

Energy Saving Act was enacted in 2023, introducing a

the Shiroi DCC Phase 2 building went into operation, and IIJ

benchmark system for the data center industry (PUE of 1.4

is currently working on plans for the facility’s Phase 3

or less), and requiring operators to submit a plan for convertingbuilding with a view to putting it into operation in 2026.

to non-fossil energy. Companies listed on the Tokyo Stock

We continue to explore new data center forms as we look

Exchange’s Prime market are also now effectively required

further into the 2020s and the prospect of even more drastic

to disclose information on climate change risks based onchange than we have seen so far.

the TCFD recommendations, and thus carbon neutrality is an
issue of some urgency for data centers. At IIJ, data centers

3.4 2020s—Next-generation Data Centersaccount for around 80% of greenhouse gas emissions (Scope

After all the changes so far, what will we ask of data centers

1 and 2)*1, and we have set and are working toward goals

going forward? The data center equipment life cycle is overconcerning the use of renewable energy and improvements

10 years, longer than the 3–5 years for IT equipment, so

in energy-efficiency (Figure 2).

societal and technological trends must be assessed from a
medium- to long-term perspective. As calls for sustainability

The use of renewable energy is a new initiative for IIJ, and takes

intensify and technological innovations like generative AI,

into account not only the quantity of power but also types oflarge language models (LLMs), and post-5G move forward,

power generation that do not produce CO2 emissions. When

IIJ will be focusing on the following three themes around

it comes to IIJ data centers, we evaluate power procurement

next-generation data centers.

methods based on delivery times, cost trends, and the

Scope1.

381(4%)• Usage of renewable energy*1
The target is to increase the renewable energy usage rate of data
centers (Scope 1 and 2) to 85% in FY2030
Scope2. 10,129(96%)• Improvements in energy conservation
The target is to keep the PUE of data centers at or below the industry’s

Roughly 80% comesfrom data centershighest level (1.4)*2 until FY2030 through continuous technologicalinnovation.

(t CO2)

FY2022 unconsolidated data for IIJ

Disclosures based on TCFD (Task Force on Climate-related Financial Disclosures) recommendations

Calculation method: “Basic Guidelines for Calculating Greenhouse Gas
Emissions Throughout the Supply Chain (Ver. 2.3)” (Ministry of the Environment,
Ministry of Economy, Trade and Industry)*1. Renewable energy usage includes substantial renewable energy through the use of non-fossil fuel certificates.
*2. As of April 2022, the Agency for Natural Resources and Energy set the benchmark index and target level for the data center industry to be a PUE of 1.4 or lower; businesses that achieve thisare regarded as energy-saving leaders.

Figure 2: IIJ’s Carbon Neutrality Initiatives*1

Scope 1, 2 (company’s own greenhouse gas emissions): Direct emissions from fuel used or industrial processes, and indirect emissions from energy/heat purchased (GHG
Protocol definition).

© Internet Initiative Japan Inc.

23

like, and we are combining multiple methods in an effort

We are also working on initiatives to develop the environmental

to increase our renewable energy usage rates, increase ourvalue trading business, with plans to start providing digital

proportional usage of power with high “additionality,” andassets (tokens) representing environmental value in Julystabilize our procurement costs (Figure 3). We have joined

2024 (Figure 5). And we are developing new roles for thethe Japan Electric Power Exchange (JEPX)’s renewable energy

data center to help solve societal issues, such as power

value trading market, obtaining certificates directly fromgrid stabilization and resilience bolstering. For instance,

the market, and we have installed solar power generation

we are part of a VPP (virtual power plant) project to use

systems on our data center premises, reducing our renew-

storage batteries in data centers (launched in FY2023),

able energy procurement costs. We aim to achieve carbon

and we are working with Matsue City, Shimane Prefecture

neutrality by combining our traditional approach to energy

as a co-proposer in the Ministry of the Environment’s

savings with offsite power purchasing and the like. We have

Decarbonization Leading Regions program, with plans toalso launched a new service that utilizes these resources touse storage batteries to supply power to local communities

provide environmental value to data center users (Figure 4).

in the event of disasters.

Increasing renewable energy usage rates early while also working to increase additionality*1 ratio and stabilize costs
Step 1. Raise renewable usage rates early through use of non-fossil certificates, green power certificates, etc.
Shiroi DCC: Directly procured non-fossil certificatesin FY2023

Matsue DCP: Achieved 100% renewable energy usage
in Apr. 2022

Non-fossil certificates
Bidding

Electricity retailer

Japan Electric Power Exchange (JEPX)

Shiroi DCC

Procuring directly from the market resultsin cost reductions (equiv. to commissions)

Step 2. Increase ratio of high-additionality renewable electricity and stabilize renewable energy costs
Shiroi DCC: Introduced onsite private
power generation in Feb. 2023
Matsue DCC: Introduced onsite private
power generation in Mar. 2023

Matsue DCP: 293kWp

Cost effective but only covers
a few percent of needs

Procure renewable electricity via offsite PPAs*(incl. wheeling for own use)

Shiroi DCC: 336kWp(400kW planned for Phase 2
building)*1. Has the effect of encouraging growth and investment in new renewable energy equipment.
*2. Power Purchase Agreement. Electricity sales agreement between an electricity user (consumer) and an electric power company (PPA operator) that sells electricity to the consumer.

Figure 3: IIJ’s Efforts to Use Renewable Energy

24© Internet Initiative Japan Inc.

2

Data center

Jun.2024

Vol.

62

3. Focused Research (2)■ AI-based Control and Automation

operator workloads at data centers. We are, for instance,

Reducing energy usage is crucial to achieving carbon neutrality

working to automate the various procedures and tasks thatin the data center. The air conditioning systems used to cool

are crucial to providing data center services, a key example

IT equipment account for the greatest share of energy

being the use of automated reception systems, which auto-

consumed by equipment other than IT equipment like servers.

mate the procedure of admitting people into data centerTo achieve data center PUEs close to the optimal value of 1.0,

buildings. We are moving in this direction because, amid

we are actively using outside air to cool IT equipment, andthe decline in Japan’s working population and the country’s

we select highly efficient equipment. At Shiroi DCC, we arework-style reforms, we expect it to become more and morealso using AI technologies to further improve energy-saving

difficult to maintain the quality of our product offerings as theperformance. We aim to achieve more effective control ofscale of our data centers expands if we continue to rely on

the overall air conditioning system by training AI on datasolely on human resources through the hiring and training offrom sensors and IT equipment. In addition to automating

operations personnel. We are looking to progressively expand

air conditioning systems, we are also taking steps to reduce

automation to other areas. For example, with respect to our

Pursuing initiatives to realize carbon-neutral data centers
Using the resources to return new value to customers and society
Conventional data centers

IIJ’s carbon-neutral data centers
Offset renewable power

Offsite power derived
from fossil fuels
Generators + UPS

Non-fossil
certificates

PPA, wheelingfor own use

Generators + UPS

Air conditioning systems

Onsite renewable power

High capacity storage batteries
Outside-air cooling systems
IT demand control

July 2022
Joined VPP (Virtual Power Plant) project,contributing to power grid stabilization

October 2023
Began procuring non-fossil certificates onbehalf of customers to support theirdecarbonization efforts
Revised Enery Saving
Act mandates regular
reporting to gov’t*1

Shiroi DCC
Electricity retailer

Certificates

For details, see IIJ’s July 28, 2022 press release, “IIJ to Participate in‘Virtual Power Plant (VPP)’ Project of Kansai Electric Power”(https://www.iij.ad.jp/en/news/pressrelease/2022/0728.html).
Source: Agency for Natural Resources and Energy website(https://www.enecho.meti.go.jp/category/saving_and_new/advanced_
systems/vpp_dr/about.html).

Renewable energy
value trading market(JEPX)

Data center
(IIJ)

User

July 2024 (planned)
Link environmental value to digital assets +digital currency payments to fuelcustomers’ GX/DX efforts
FutureEnvironmental
value+
payments

Data center(IIJ)

Secondarytrading

User

Company B

Electricity
Environmental value*1. Agency for Natural Resources and Energy, Key points for regular
reporting under the Energy Saving Act—inclusion of IT equipment
energy usage of tenant businesses, May 2023
(https://www.enecho.meti.go.jp/category/saving_and_new/saving/
enterprise/factory/faq/pdf/a1-18.pdf).

DCJPY Network

Figure 4: Beyond Carbon-Neutral Data Centers© Internet Initiative Japan Inc.

25

plans to supply environmental value-linked power to data

are set to come out with TDP (thermal design power) ratingscenter users, as described in the previous section, we hopein excess of 300W, and this is only expected to rise ahead

to go beyond simple automation to provide more sophisticated,in response to demand for AI. According to the American

higher-quality offerings by developing an electricity supply/Society of Heating, Refrigerating and Air-Conditioning

demand matching platform, introducing digital currency

Engineers (ASRAE), once TDP exceeds 300W, facilities will

payments, and the like.

need to adopt water-cooled systems instead of conventional
air-cooled systems (Figure 6).

■ Supporting Higher Server Densities and Water Cooling
As the use of technologies like generative AI and LLMs in

Because they will also house network devices and other sucha range of fields continues to advance and the processing

IT equipment that is air coolable, future data centers will need

power of CPUs and GPUs increases, next-generation data

to adopt hybrid cooling systems that combine air and water

centers will need to be able to efficiently accommodate hugecooling while also providing excellent energy-saving perfor-

numbers of these CPUs and GPUs. Ahead, data center CPUs

mance in order to achieve carbon neutrality. IT equipment

Source: Ashrae Emergence and Expansion of Liquid Cooling in Mainstream Data Centers(https://www.ashrae.org/file%20library/technical%20resources/bookstore/emergence-and-expansion-ofliquid-cooling-in-mainstream-data-centers_wp.pdf).

Figure 6: TDP (CPU Power) and Cooling Methods

Electricity supply/demand matching platform
Matching
payments

Grid (retailers,
generation firms, power
trading market, etc.)

Matching
payments

DC operator (IIJ)

Onsite PV
Ph-3-a: Use in otherindustries, applications(commercial buildings,
condos, etc.)

Power

Fossil-derived

Power
Wind
power

Solar
power

DC user
User-specified
renewable energy

User A
Power
StoragebatteryPower

Generation firmElectricity retailer
Environmental value

Data center

Low-cost, no
renewables
Additionality
renewable energy

Renewableenergy trading
market(JEPX)

User C

Ph-2: Offsite grid links

Ph-1: Data center onsite

DCJPY Network
Figure 5: Addressing the Rising Need for Decarbonization

26

User B© Internet Initiative Japan Inc.

Ph-3-b: Purchase of other
IIJ services using
environmental value
tokens (network, cloud,security, MVNO, etc.)

Jun.2024

Vol.

62

3. Focused Research (2)and facilities will become more closely intertwined, and wedevelopment of a high-density data center reference model,

believe this will be an area in which IIJ will be able to leverage

establishing hybrid cooling methods that combine air-coolingits strengths, given the company’s experience building andand water-cooling technologies, and formulating and developing

operating large-scale data centers and cloud services. IIJ,

evaluation methods for energy-saving indicators geared toPreferred Networks Inc., and the Japan Advanced Institute

AI computing platforms.

of Science and Technology (JAIST) jointly put forward a
project, which was subsequently accepted, for research

3.5 Conclusionand development into ultra-high-efficiency AI computation

While the users of IT systems may not often give much thought

infrastructure for inclusion in the Ministry of Economy, Tradeto data centers, there is growing recognition that data centers

and Industry and the New Energy and Industrial Technology

are a key component of the infrastructure underpinning a

Development Organization (NEDO)’s program of research

digitalized society. Drawing on our data center journey so far,and development of enhanced infrastructure for post-5G

from computing centers and communications stations,information and communication systems, specificallythrough to colocation services, the cloud, and hyperscaling,

addressing the “Development of post-5G information andwe will continue to pursue new forms of data centers andcommunication systems (commissioned project)” category. IIJ

as we go about operating the infrastructure that supports

is responsible for research and development of basic technol-

our society.

ogy for high-density data centers. We will be working on the

Yoshikazu Yamai
Managing Executive Officer and Director of the Infrastructure Engineering Division, IIJ

Isao Kubo
General Manager, Infrastructure Services Department, Infrastructure Engineering Division, IIJ
Mr. Kubo joined IIJ in 2008. He oversees the data center business and the construction of Matsue DCP and Shiroi DCC. His aim is toachieve carbon neutrality as soon as possible.

Yusuke Tsutsumi
Data Center Infrastructure Engineering Section, Infrastructure Services Department, Infrastructure Engineering Division, IIJ
Mr. Tsutsumi joined IIJ in 2015 and works on the construction of data centers in Japan and abroad. He is engaged in the development oftechnologies for next-generation data centers, which includes studying new technologies in the field of electric power.

Takahiro Mimura
Data Center Infrastructure Engineering Section, Infrastructure Services Department, Infrastructure Engineering Division, IIJ© Internet Initiative Japan Inc.

27

62
June 2024 Vol.
About Internet Initiative Japan Inc. (IIJ)
IIJ was established in 1992, mainly by a group of engineers who
had been involved in research and development activities related
to the Internet, under the concept of promoting the widespreaduse of the Internet in Japan.
IIJ currently operates one of the largest Internet backbones
in Japan, manages Internet infrastructures, and provides
comprehensive high-quality system environments (includingInternet access, systems integration, and outsourcing services,etc.) to high-end business users including the government andother public offices and financial institutions.
In addition, IIJ actively shares knowledge accumulated throughservice development and Internet backbone operation, andis making efforts to expand the Internet used as a social
infrastructure.­­The copyright of this document remains in Internet Initiative Japan Inc.
(“IIJ”) and the document is protected under the Copyright Law of Japanand treaty provisions. You are prohibited to reproduce, modify, or makethe public transmission of or otherwise whole or a part of this document
without IIJ’s prior written permission. Although the content of thisdocument is paid careful attention to, IIJ does not warrant the accuracy andusefulness of the information in this document.
Initiative Japan Inc. All rights reserved.
© Internet
IIJ-MKTG020-0060

Internet Initiative Japan Inc.
Address: Iidabashi Grand Bloom, 2-10-2 Fujimi, Chiyoda-ku,
Tokyo 102-0071, Japan
Email: info@iij.ad.jp URL: https://www.iij.ad.jp/en/