2‑82 Telecommunication Radio Broadcast

Flow Rate Analysis of Syslog Collectedin the Tokyo 2020 GamesHIROSE Masato

KANAI Akira

In a network for events in which multiple vendor network devices are integrated it is hard to analyze the semantics oflogs or detect anomalies by keywords We collected device logs from some operational organizations during the Olympic
and Paralympic Games Tokyo 2020 We focused on the file size of the logs to follow the scale of the network of the games
And we categorized the logs by device type and installation site contributing to real‑time analysis and discovery ofmultiple problems
Keywords：Network monitoring Log analysis Tokyo 2020 Games

network We decided to conduct a logging analysis using

1．Introduction
For network operators

this method and tried to apply it to the environment ofdefining the anomaly or

the Tokyo 2020 Games However it was not possible tonormal state and having the monitoring system notify

prepare the necessary and sufficient compute resources

operators in the event of anomaly states contribute toin a short time to implement this method in real‑time A

the early detection of problems

To determine anparticular challenge was the amount of memory required

anomaly or normal state from device logs it is necessary

to count the number of log lines In the Tokyo 2020

to define anomaly string patterns according to theenvironment nearly 1 GB of logs were collected in thecontents of the target device logs before the operational

monitoring environment every minute Reading theseterm These preparations are time‑consuming tasks forlogs into memory and counting the number of lines in

network operators In addition considering the scale ofreal‑time required much memory

the Tokyo 2020 Olympic and Paralympic Games(hereinafter referred to as “Tokyo 2020”) network the

2．Log Analysis Environment for Tokyo
2020 Games

ever‑changing nature of the events and the building andoperation period of each venue it is very difficult todefine anomaly or normal states in advance (1) proposed

To solve this problem the total size of the log file

to perform log analysis in such an environment This

every few minutes was used in the analysis instead of themethod uses a Bollinger band graph to analyze thenumber of lines in the log To obtain the log file size wenumber of lines in the logs of devices in an eventchecked the file size every five minutes on a log storage
server and took the records In Linux such as CentOS itis possible to obtain the target file size by getting theHIROSE Masato
Innovation Center NTT Communications Corporation
E‑mail masato hirose@ntt com
KANAI Akira
Innovation Center NTT Communications Corporation
E‑mail a kanai@ntt com

The Journal of The Institute of Electronics Information and Communication Engineers Vol 105 No 8 Supplement pp 67‑70 August 2022
© 2022 The Institute of Electronics Information and Communication Engineers

inode information managed by the file system Therefore

obtaining the file size requires much fewer

computing resources than counting the number of lines
in the log file
While performing the operations a Bollinger bandSpecial Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 202067

Figure 1

Bollinger Band Graph Generated from Log Flow

Table 1graph is generated The moving average and standard
deviation were calculated using data from 12 intervals

Category

Examples of Categorization
Example of division

ago i e from one hour ago
Figure 1 shows a Bollinger band graph generated
based on the log file size increase data for one day during
the Tokyo 2020 Games The blue line shows the moving
average the orange line shows the standard deviation

Number ofdivision

Type ofdevice

Router Switch UTM etc

40

By DC andBy Venue

Primary DC Secondary DC National
Stadium etc

64＋3σ and the green line shows the standard deviation−3σ This figure shows that the log flow rate gradually
increases from 06 : 00 and progressively decreases from18 : 00

we categorized the logs of devices by device type DC

The section between 03 : 00 and 05 : 00 isand venue and generated a graph for each Table 1

characterized as shown in the figure by an increase inshows the actual categorization This categorization

logs generated when logging into the device for periodic

enabled the operators to notice anomalies even in

backup of settings The rapid growth in log flow fromdevices with a low log flow and quickly narrow down the

06 : 00 was due to the sudden increase in network usage

list of devices suspected of having problems

because of the entry of staff and press into each venue
Then we confirmed that the log flow rate of loadbalancers and proxies increased rapidly From around3．Results Obtained from the Analysis

18 : 00 network usage and the log flow rate decreased

The analysis of the Tokyo 2020 Games was conducted

On the other hand the flow rate increases in bursts

using the method described in Section 2 3 The first isaround 22 : 00 When analyzing this graph the operator

external scanning activity and the second is missing

was notified when the moving average exceeded thesecurity appliance logs

standard deviation of ＋3σ or when it fell below theThe first event is related to external scanning activity

standard deviation of ‑3σ Upon receiving a notification

We found that the moving average of the log flow on thethe operator manually searched the logs for the device

Unified Threat Management (UTM) log flow graphthat had significantly increased or decreased log flow

(Figure 2) exceeded the standard deviation of 3σ The

during the relevant period and escalated the situation if a

UTM logs contain many filtered logs of scanned packets

problem was suspectedduring the time in question and the system detected thisAs mentioned above when a single Bollinger graph isevent by capturing changes in the log flow rate

generated for all logs collected the logs of devices thatThe second event was the log loss of the security

generally have a higher log flow than those of otherappliance On the graph of the collected logs (Figure 3)

devices occupy most of the total graph and the log flow

the log flow rate decreased by approximately 1 GB Theof different devices is not visible as a trend Therefore

moving average of the log flow fell below the standard

68

THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS

Figure 2

Figure 3

Graph of External Scanning Activity Detected

Graph of Missing Security Appliance Logs

deviation of ‑3σ which led to the discovery of this event
After a detailed investigation by the team operating the4．Conclusion

device it was found that the logs were not being

This article describes our analysis of the overall logs

transmitted to the outside world for a certain period due

data collected from devices across the Tokyo 2020

to the performance limitations of the device Fortu-

Games To keep up with the scale of the network for thenately the logs for that period were stored inside theGames we focused on the file size of the logs to realize

device and there was no impact on business operations

real‑time analysis In addition categorizing the logs by

However it was only by focusing on the logsʼ flow rate

device type and installation site made it possible forthat we could detect and discover anomalies in theoperators to identify suspected problems quickly The

logging system of the device

system contributed to the detection of multiple issues
and increased the speed of response

Special Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 202069

References()

H. Abe, M. Shikida, and Y. Shinoda,“The anomaly detection method
analyzing syslog data using bollinger bands algorithm on eventnetwork,” the Information Processing Society of Japan, vol. 59, no. 3,pp. 1006-1015, 2018.

KANAI Akira
KANAI Akira completed the Graduate
School of Media and Governance at Keio
University in 2009 He joined NTT Communications Corporation in the same year He is a
CISSP CISA CISM and a Professional Engineer (Information Technology)(Received 28 February 2022 ; Revised 16 March 2022)

HIROSE Masato
Completed the masterʼs program in theGraduate School of Information Science andTechnology at JAIST in 2017 He joined NTTCommunications Corporation in the same year
Since then he has been researching anddeveloping Internet technologies and services㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇㍇70THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS