Data Access Pattern Analysis for dCache
Storage System
112

2

Amanda Pereira , Alex Sim , John Wu ,Shinjae Yoo, Hiro Ito
2

Drake University, Lawrence Berkeley National Laboratory, Brookhaven National Laboratory

ABSTRACTRESEARCH QUESTION

dCache is a storage management system withdisk space for temporary and permanent ﬁlestorage and connectivity to a larger highperformance storage system (HPSS) for
additional ﬁle storage. Ideally, all ﬁles will be
stored locally on the dCache disks when theyneed to be accessed. Analysis of past dataset
access patterns for the ﬁles stored on dCacheis necessary to determine how dataset
popularity varies over time and what factors
might aﬀect the ﬂuctuation. Findings suggestthat datasets are generally accessed only afew days at a time with an inconsistent day to
day popularity measurement, and therefore
could only be stored locally for the few days
that they are popular.

How can future dataset popularity be predicted given
information on past dataset popularity?

ANALYSIS METHODS
The analysis of data transfer logs and metadata information are used to address thisresearch question by grouping ﬁles into datasets and studying the popularity ﬂuctuation.

Datasets are made up of ﬁles that all contain information about the same instance• Energy measurement of particles being
studied is described using
Teraelectronvolts (TeV).

• 69.68 % of the 13TeV files include“mc15_13TeV” or “mc16_13TeV in the file path
• 88.51% of mc15_13TeV files contain “EVNT” or“log” + 8-digit number• A majority of accesses in mc16_13TeV
grouping are for files that contain “DOAD” or“AOD” in their name

BACKGROUND● Particle collision events being
studied produce a very high volumeof data
○ A majority of the data is stored on
HPSS tapes, and the rest is stored
on dCache disks.

Data is retrieved through a request tothe dCache System• 83% of data files in the logs are files withinformation about particle collisions at anenergy level of 13TeV

Local copy notfound in cache: acopy of the ﬁlehas to be moved
for access.

Motivation:
Caching data shortens access timeand reduces latency. Forecastingdataset popularity for a future timeframe allows the dataset to becached prior to when it needs to beaccessed.

● Search for patterns in popular andunpopular files may help what factors
may lead to a high popularity● For maximum storage efficiency, datasets
could be stored in the local cache during
the time period in which they are popular,
and then moved to the HPSS for storage
during the time they are not popular.
FUTURE WORK:● Develop a machine learning model topredict the popularity of a given file in thenext time frame● Expand study to longer time periods

FURTHER READING

All ﬁlepaths with the same 8 digit number are grouped together.

RESULTSACKNOWLEDGMENTS

Mc15 group plots:- dCache searches local cache for theﬁleLocal copy foundin cache: File canbe read from orwritten to with noHPSS interaction.

SUMMARY

April 3 2021 event popularity

April 3-9 event popularity● spike in graph - popularevent● More colors - accessedon more days
○ Greater ﬂuctuation in
popularity of event
datasets than log
datasets.

This work was supported in part by theOffice of Advanced Scientific Computing
Research, Office of Science, of the U.S.
Department of Energy under Contract
No. DE-AC02-05CH11231, and also used
resources of the National Energy
Research Scientific Computing Center(NERSC). This work was also supported
in part by the U.S. Department of Energy,
Office of Science, Office of Workforce
Development for Teachers and Scientists(WDTS) under the Science
Undergraduate Laboratory Internship(SULI) program.