Big Data Analysis in Geoscience
Frontier of Understanding Earth’s Interior and Dynamics
Tohoku University, Sendai, Japan
August 8-9th, 2022

Karianne J. BergenAssistant Professor of Data ScienceBrown University, Providence, RI, USA

Big Data Analysis is helping Earth scientists
extract more knowledge & insightsfrom larger, more complex data sets than ever before.

K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Drivers of Big Data Geoscience
1) Massive datasets
2) Advances in computingData

3) New techniques and algorithms
ComBig DataGeoscienceputingthirogl
Asm

adapted from Arrowsmith et al. (2022)
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Massive geoscience data setsLong-duration continuous observations(Image: SCEC, SDSC)[Nakata et al. 2015][Smith-Konter et al. 2018]33˚48'

Large-scale simulations
Crowdsourced data
Remote sensingobservations
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

33˚45'

1 km−118˚06'

Dense sensor arrays

Terabytes (TB)

Benchmarkdatasets

Terabytes (TB)

Data Volumes: IRIS-DMC Archive

Arrowsmith et al. (2022)
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

The challenges of big geoscientific data analysis
Volume: Data-gathering capabilities – GB to TBs per day• Extracting information – automated analysis, scalability

Velocity: Near real-time analysis, e.g. for hazard assessment• Streaming data – fully automated (no configuration)

Variety:Multimodal datasets, e.g. seismometers + GNSS
• Sensor fusion – combining multiple data sources

Veracity: Data quality, e.g. noisy environments, instrument error• automatic data cleaning, quality control, denoising
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Algorithms for big scientific data
• Efficient algorithms: linear / sub-quadratic scaling with data volume• randomized algorithms, streaming algorithms, etc.
• Data-driven algorithms: large-scale machine learning (e.g. deep learning)• Custom, task-specific algorithms• Data reduction, data compression• More computation: parallel and distributed computing, cloud computing

K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Can new Big Data algorithms detect small earthquakes
and identify seismic phases?

FAST: How can we detect more small earthquakes?

Yoon et al., (2015)

Leverages technology for efficient audio recognition
Discovers new event waveforms (without labeled data):
10 –100× earthquakes detected
Computationally efficient:
500× more data with reduced runtime
P. Bailis

G. Beroza

P. Levis

O. O’Reilly

K. Rong

C. YoonFAST: scalable “Large-T” earthquake detection
Data mining: extract similar waveforms from large datasetsNaïve (slow, exact) search: small data

2013-OctEfficient (fast, approximate) search:

2010-Sep
2009-Mar

vs.

2008-Apr
2007-Nov

Searching a well-organized database is faster – cluster similar waveforms for quick retrieval
Sacrificing (a little) accuracy can substantially reduce runtime.
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Algorithms for big scientific data
• Efficient algorithms: linear / sub-quadratic scaling with data volume• randomized algorithms, streaming algorithms, etc.
• Data-driven algorithms: large-scale machine learning (e.g. deep learning)• Custom, task-specific algorithms• Data reduction, data compression• More computation: parallel and distributed computing, cloud computing

K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

What is Machine Learning?
Automating and scaling data analysis

Machine learning (ML)a set of tools for recognizing complex
patterns and building predictive models
automatically from data
Unsupervised
Discovering
structure in dataydata
linear modelSupervised
Learning a patternfrom examples
xK.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Machine Learning is a key
tool for high-volume data
• diverse datasets & tasks• dense & sparse data
• high- or low-knowledge

Arrowsmith et al. (2022)
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

How is machine learning being used by geoscientists today?

Dis

ing
delMo

cov
er y

How might it be used in the near future?

Automation
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

How is machine learning being used by geoscientists today?

Dis

ing
delMo

Extract new information,
patterns, structure, orrelationships from data

cov
er y

How might it be used in the near future?

Automation
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

How is machine learning being used by geoscientists today?

DisingdelMo• Learn representations• Build surrogate models• ML + simulations

cov
er y

How might it be used in the near future?

Automation
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

How is machine learning being used by geoscientists today?

DisingdelMo• Perform a complex or
repetitive task
• High accuracy predictions

cov
er y

How might it be used in the near future?

Automation
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Benchmark & training datasets for supervised ML
STEAD dataset[Mousavi et al. (2019), IEEE Access.][Michelini et al. (2021), DOI: 10.13127/instance]
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

1D CNN for detection and phase-picking[e.g. Perol et al. (2018)][Ross et al., 2018]

K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

EQTransformer: Attention-based model for detection & phase picking[Mousavi et al. (2020), Nature Communications]PhaseNet: U-net for phase picking[Zhu & Beroza (2018), Geophys J. Int.]

Bergen et al. (2019), Science; Kong et al. (2019), BSSA; Dramsch (2020), Adv. in Geophys; Yu & Ma (2021), Rev. Geophys.
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Flexibility of Neural Networks

Neural Network Model Zoo (F. van Veen & S. Leijnen)
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Can we build an interpretable deep NN for detection?
Chen et al. (2019)

𝑧Domain- or task- specific
NN architectures• Design humaninterpretable prediction
systems𝑑 (") =𝑥• Incorporate physics into
data-driven NN model𝑦!

• Jointly analyze data frommultiple sources𝑧 − 𝑇 (")

𝑇
Prototype layer (pairwise distances)
Conv (+ BN + ReLU) + MaxPool

Dense + Softmax

Flatten

Dense

Future of Machine Learning for the solid Earth[Bergen et al., Science 2019]
Conferences& Workshops

Joint workwith ML
experts

Interpretable
Learning
PhysicsGuided
Learning

Open access

Geo-DataScience
Education

OpenScience
Open source

Advances in our
understanding ofthe solid EarthNew ML
architectures& models

Benchmark
Data sets

ChallengeProblems

Data Science
CompetitionsK.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics

Further reading

REVIEW◥

GEOPHYSICS

Machine learning for data-driven
discovery in solid Earth geoscienceereaisfseru
R ES E A RC H | R E V IE Wr
Karianne J. Bergen1,2, Paul A. Johnson3, Maarten V. de Hoop4, Gregory C. Beroza5*
dlaUnderstanding the behavior of Earth through the diverse fields of the solid Earth geosciencespetitive
tconsum
is an increasingly important task. It is made challenging by the complex, interacting, andvolcanic
multiscale processes needed to understand Earth’s behavior and by the inaccessibility of nearly
ML ac
all of Earth’s subsurface to direct observation. Substantial increases in data availability anda repree
in the increasingly realistic character of computer simulations hold promise for accelerating
structule
progress, but developing a deeper understanding based on these capabilities is itself
buildina
challenging. Machine learning will play a key role in this effort. We review the state of the fieldor incotand make recommendations for how progress might be broadened and accelerated.
tweenmtionshis
and peap
such that aspects of that knowledge are not
he solid Earth, oceans, and atmospherepredict
constrained by direct measurement.
together form a complex interacting geoapprox
For this reason, solid Earth geoscience (sEg)system. Processes relevant to understando
computc
is both a data-driven and a model-driven field
ing its behavior range in spatial scale from(23, 24)
the atomic to the planetary, and in tempowith inverse problems often connecting the two.
snect ob
ral scale from milliseconds to billions of years.
Unanticipated discoveries increasingly will comedand phPhysical, chemical, and biological processes infrom the analysis of large datasets, new developsystemc
teract and have substantial influence on thisments in inverse theory, and procedures enabledlearninf
by computationally intensive simulations. Over
complex geosystem. Humans interact with it
folems (2the past decade, the amount of data available totoo, in ways that are increasingly consequenis
tures ingeoscientists has grown enormously, throughtial to the future of both the natural world andlearn as
traditio
alarger deployments of traditional sensors andcivilization as the finiteness of Earth becomesgeneraltthrough new data sources and sensing modes.
increasingly apparent and limits on available
into theComputer simulations of Earth processes areenergy, mineral resources, and fresh water inDataa
rapidly
increasing
inscale
andsophistication
creasinglyaffectthehumancondition.
EarthisFig. 1. How scientists analyze data: the conventional versus the ML lens for scientific analysis.
newsuch that they
are increasingly
and rele- ins
subject
to atovarietyof at
geohazards
that area poorly
ML is akinlooking
the data throughnew lens. Conventional
approaches
applied realistic
by domain
citing dc
to predicting
Earth’s
behavior.
understood,increasingly
as our exexperts (e.g.,yetFourieranalysis)impactful
are preselected
and testvant
a hypothesisor simply
display
data Among thetions. aM
foremost
facinggeoscientists
posuregrows
increasing
urbanization,differently.
ML throughexploresa larger function
space that canconnectchallenges
data to sometarget
or label. In is how
K.J. Bergen | Frontier of Understanding particularly
Earth'sInterior&Dynamics
for disco
to extract
as muchinformationas possible
in hazard-prone
We havea between
doing so, it provides
the meansareas.
to discover
relationsvariablesin usefulhigh-dimensional
space.

TtionshiBig Data Analysis is helping Earth scientists
extract more knowledge & insightsfrom larger, more complex data sets than ever before.

Questions?

karianne_bergen@brown.edu@KarianneBergen

ご清聴ありがとうございました。
K.J. Bergen | Frontier of Understanding Earth's Interior & Dynamics