WCCM-APCOM 2022
MS0808-2114

Gaussian Process Regression Surrogate Modelingwith Transfer Learning for Low Computational Cost
Structural Reliability Analysis

Taisei Saida(University of Tsukuba)

Mayuko Nishio (University of Tsukuba)1【Background】Necessity to consider uncertainties in infrastructures

2/26• Infrastructures such as bridges are designed for load and strength.

• However, structures may deteriorate and suffer damage, or collapse due
to earthquakes or other damage, during the service life of a that.
• This is due to the difference between design and reality.
There are many uncertainties in reality.
• Therefore, a reliability analysis is needed that considers uncertainties
related to loads and structural strength.
Displacement of approx. 60 cm【Deterioration and damage of bridges】【Examples of serious damage】

Steel pile pier corrosion（MLIT，Measures to prevent roads from aging，Aging Status）（MLIT，Anti-aging Initiatives）

Steel piers show
no damage

All bearings ruptured（JSCE, Steel Structure Committee）

Maximum step height ofapprox. 40 cm

【Background】Reliability Analysis Flow
TargetGeneral Reliability Analysis
AnalyticalmodelOutput
distribution

Reliability Analysis using Surrogate Model

Mass

Analyticalmodel

Analysis
DataMaximum
StressSurrogate
ModelMaximum
Stress

Outputdistribution
Probability

DoESampling
StiffnessProbabilityex)Mass of Piers

Disaster Risk
Evaluation

Maximum
Stress

MassConsidering
Uncertainties

dangerProbability

Monte Carlo
Sampling
Stiffnessex) Bridge

3/26

dangerMaximum
StressSurrogate models can reduce
computational cost of reliability analysis

4/26【Previous Research】Reduced computational cost of building surrogate models• Adaptive Sampling
Reduces computational cost byfocusing on hard-to-predict points
and points of high importancewhen sampling input parametersEchard et al., Structural Safety, 2011

(Jan et al., Archives of Computational
Methods in Engineering, 2021)• Variable fidelity surrogate model
The use of low-fidelity analysis
results with low computational cost
reduces the number of targeted
high-computational-cost analyses
Skandalos et al., Structural Safety, 2022

Problem(Tian et al., Composite Structures, 2021)

The surrogate model is valid only
for the analysis of the target

5/26

Transfer Learning Gaussian Process Regression Surrogate Model (TL-GPRSM)

When designing a bridgeMax
StressWeight

WeightSurrogate Model

Stiffness

MaxStress

Stiffness

StiffnessWeight

Analytical
Model

Analytical
Data

Transfer learning leverages
design-time data
Surrogate ModelMaxStress

WeightStiffness

Dangerex)Max Stress

When evaluation of existing bridges
DoE Sampling

WeightOutputDistribution
Probability

Analytical
DataMax
StressStiffness

WeightOutputDistribution
Probability

Analytical
Model

Stiffness

DoE Sampling

Dangerex)Max Stress

Reduced computational cost for reliability assessment of existing bridges
by building a TL-GPRSM using transfer learning

【Previous Reserch】Surrogate model with transfer learning• Application of Transfer Learning
to Variable Fidelity Surrogate
Models：Transfer learning of DNN
models trained on low-fidelity
data to high-fidelity domains(Tian et al., Composite Structures, 2021)• Surrogate models for energy
system optimization：
Using transfer learning torespond to environmental
changes such as wind and solar(Perera et al., Applied Energy, 2019)

The case of unsuccessful transfer
learning is not anticipated.

6/26

Issues in transfer learning

7/26• Negative Transfer
Transfer learning degrades
the performance ofmachine learning models.
⇩Tommasi et al., IEEE transactions on pattern analysis and machine intelligence, 2013

The cause is low similarity
between the source anddestination data.

The possibility of negative transfer should be consideredGaussian Process Regression (GPR) with ARD KernelGPR• Nonparametric• Non-linear regression

8/26

ARD Kernel Function
ARD：Automatic Relevance Determination
DDri 5 D ri ri k (r ) =σ  1 + 5 ∑ + ∑ 2  exp  − 5 ∑ li 3 i 1 li i 1 li =
=i 1=
2

Matern5/2 kernel

x : input vectory : output vector
k：kernel function
K : kernel matrix

Kernel Matrix

Length Scale（li）
Represents the contributionof each input variable to the output

ARD Kernel

Estimate the contributionof input parameters

Poisson’s ratio
Yang’s modulus
Thickness

Knm : elements of kernel matrix

Contribution (%)

Transfer Learning in Gaussian Process Regression

9/26

Φ s ( x) = 〈Φ ( x), Φ ( x), 0〉
Φ t ( x) = 〈Φ ( x), 0, Φ ( x)〉
OutputInputsCommonSourceDomainySxS

SourcexS

Target0

ARD Kernel
Estimate thecontribution ofeach part

CommonSourceTargetTargetDomainyTxT0

The greater the contribution of theCommon part, the greater theeffect of transfer learning.

xT02040Contribution（%）

60

H. Daumé III, Frustratingly Easy Domain Adaptation, ArXiv [Cs.LG]. (2009).
http://arxiv.org/abs/0907.1815.

FE model of bridge

10/26Analysis model: Standard simple I-girder bridge
LengthWidthGirder
Floor slab

Steel Wire Bearing : Solid Elements: 20000 mm
Number of elements : 104799: 10700 mm: Shell element Analysis software : Abaqus: Shell element

All bridge model
Half-digit model
G1
G2
G3
G4
G5

Move

Corrosive area

Friction considerations

FixAdjustment to match behavior
with all bridge model

400

400400Uncertainty setting11/26
At Design

FE Model Parameters (Units)

At DamageNominalCOV

Nominal

COV

#1

Dc

Density of concrete slab(kg/m3)

2400

0.0171

*

*

#2

Es

Young's modulus of steel main girders(GPa)2000.0450

*

*

#3

Ec

Young's modulus of concrete slab(GPa)250.0167

22.5

0.0333#4Eb

Young's modulus of steel bearings(GPa)2000.0450

*

*

#5

Vs

Poisson’s ratio of steel main girder

0.30.0910

*

*

#6

Vc

Poisson’s ratio of concrete slab

0.20.0167

*

*

#7

Vb

Poisson’s ratio of steel bearing

0.30.0910**#8CfFriction coefficient of steel bearing

0.20.0167

0.90.0333

#9

Tuf1

Thickness of upper flange of steel girder at near-end section (mm)0.01900.0121**#10 Tuf2

Thickness of upper flange of steel girder at mid-span section (mm)0.03000.0121**#11 Tw

Thickness of web plate of steel girder (mm)0.00900.0121**#12 Tbf1

Thickness of lower flange of steel girder at near-ends section (mm)0.02700.0121**#13 Tbf2

Thickness of lower flange of steel girder at mid-span section (mm)0.03000.0121**#14 Tstc

Thickness of stiffener of steel girder at near-ends section (mm)0.01300.0121**#15 Tstm

Thickness of stiffener of steel girder at mid-span section (mm)0.01000.0121**#16 Tstn

Thickness of stiffener of steel girder at other section (mm)0.00650.0121**Thickness of corroded area in lower flange of steel girder at near#17 Tbf-d end
section (mm)

−

−

0.025

0.0270#18 Tw-d Thickness of corroded area in web plate of steel girder (mm)−−0.0080.0162−−0.0120.0162of corroded area in stiffener of steel girder at#19 Tst-d Thickness
near-end section (mm)※Determined with reference to previous studies

Reliability Analysis Overview
Uncertain
Parameters

Probability

At design time
16 variables
At deterioration
19 variables

Parameter

FE Analysis(design live loadis applied)

12/26

Maximum Mises StressDistribution(500 data each)Half-digit modelStress at a girder end(output maximum stress)

Reliability Analysis Inputs and Outputs
Inputs
Uncertain
Parameters

FE Analysis

OutputMaximum Misesstress at girder ends

Datas for Transfer Learning

13/26

Source Data: At design
Maximum
Stress

Uncertain
ParameterySxS
Target Data: At damage

MaximumStressyTUncertain
ParameterxTTransferLearning

【Result】Accuracy of TL-GPRSM
Predict Maximum Stress（10 Trials）

RMSPE
= 100

ー TL-GPRSM
ー Without transfer learning
TL-GPRSM

05101520

Number of data

25

Number of data at damage

TL-GPRSM
Target only

RMSPE(%)
RMSPE (%)

RMSPE(%)
RMSPE (%)

Target only

Number of data at design: 30

109876543210300510152025

Number of data at damage
Number of data30

1  yi − yˆi ∑n i =1 yi n2

Numberof data at design: 100
10RMSPE(%)
RMSPE (%)

Number of data at design: 10

10
98
7
6
5
4
3
2
10

14/26

98
7
6
5
4
3
2
10

TL-GPRSM
Target only

05101520

Number of data

25

Number of data at damage• TL-GPRSM converges to the number of training data faster,regardless of the number of data at design time, in some caseswith 40% less data than the model without transfer learning• The greater the number of data at design time, the higher theaccuracy of TL-GPRSM

30

【Result】Predicted Distribution of Maximum Stress

15/26

Predicted Distribution of Maximum Stress（10 trials）
Number of data at design：30
Number of data at damage:ー TL-GPRSM（10 trials）
ー without TL（10 trials）
ー FE Analysis• TL-GPRSM converged faster on the number of training data than thesurrogate model without transfer learning• TL-GPRSM predicted a distribution shape closer to that by FE analysis thanthe SM without transfer learning for the same number of training data【Result】 parameter contribution estimation by ARD

16/26

Contribution of each part

60504030

CommonTargetSource2010005101520

Number of data

2530

Number of Target Data (at damage)Source Data (at design): 30

7060504030

CommonTargetSource2010005101520

Number of data

2530

Number of Target Data (at damage)

Estimated
Contribution(%)
ARD weight(%)

Source Data (at design): 1070

Estimated(%)
ARD Contribution
weight (%)

Estimated(%)
ARD Contribution
weight (%)

ー Common partー Target partー Source part
Source Data (at design): 10070605040

CommonTarget
Source302010005101520

Number of data

2530

Number of Target Data (at damage)• The higher the number of source data, the higher the contribution of thecommon part, and the greater the effect of transfer learning.
• The number of source data (30) and the number of source data (100)converged to roughly the same contribution.

【Result】 parameter contribution estimation by ARD

17/26

Contribution of each uncertain parameter
Number of data at design
Number of data at damage
Common part
Tstm
Tstn
Tstc
Tbf2
Tbf1
TwTuf2
Tuf1
CfVbVcVsEbEcEs
Dc

The contribution ofthe friction
coefficient of thebearing is significant.
051015Contribution (%)

20: 30: 15
Target part

Source part
Tst_d
Tw_d
Tbf_d
Tstm
Tstn
Tstc
Tbf2
Tbf1
Tw
Tuf2
Tuf1
Cf_d
Vb
Vc
Vs
Eb
Ec_d
Es
Dc

Tstm
Tstn
Tstc
Tbf2
Tbf1
Tw
Tuf2
Tuf1
CfVb
Vc
Vs
Eb
Ec
Es
Dc
05

101520Contribution (%)• ARD is able to properly estimate the contribution.

The contributions ofparameters related
to damage are large.

051015Contribution (%)

20

18/26Another story building TL-GPRSM
When designing a bridge
Surrogate ModelMax
StressMax
Stress

WeightWeightStiffness

Stiffness

Stiffness

Analytical
Model

Analytical
Data

WeightSurrogate ModelMax
StressMax
StressWeight
Stiffness

Dangerex)Max Stress

Transfer learning leverages
design-time data

When evaluation of existing bridges
DoE Sampling

WeightOutputDistribution
Probability

Analytical
Data

Stiffness

WeightOutputDistribution
Probability

Analytical
Model

Stiffness

DoE Sampling

Dangerex)Max Stress

Reduced computational cost for reliability assessment of existing bridges
by building a TL-GPRSM using transfer learningAnalytical model of isolated RC piers

19/26（Japan Road Association, 1997）

Upper Structure
Isolated Bearing

AnalysisParameters
Integration：
Bilinear•

Newmark - β•Newton Raphson

Time Increment：0.001s
TakedaRC Pier
2DOF model of
Isolated RC Pier

Rayleigh damping
Damping Ratio：•

2% for pier•

0% for bearing20/26

Uncertainty parameter setting
Parameters

NominalSuperstructure weight (Mu)

604000 kg

Primary stiffness (Kb1)
Seismic
Isolation
Bearing

RC Pier

Uncertainty

40023.2 kN/m

Secondary stiffness (Kb2) 6154.4 kN/mYield load (Qb)1117.2 kN

weight (Mrc)

346300 kg

Primary stiffness (Krc1)

110000 kN/m

Uniform
Distribution± 10 %

Secondary stiffness (Krc2) 8250 kN/mYield load (Qrc)

3399 kN（Reference: Japan Road Association, 1997）

Reliability Analysis Overview and Input/Output
Uncertain
Parameters

Seismic Response
Analysis

21/26

Maximum Displacement(Pier and Bearing)

8 variable uncertainty

Pier

Probability

Bearing
Uncertain Parameter

Output maximum
displacement

Surrogate model inputs and outputs
Inputs
Uncertainty Parameters
8 Variables

Surrogate Model

Outputs
Maximum Displacementsof Pier and Bearing

Datas for Transfer Learning
Source Data:

Level2-Type1-1-1 (200 data) andLevel2-Type2-1-1 (200 data)

Maximum
Displacement

Uncertain
ParameterySxS

Target Data: JMA-KOBE Earthquake
Maximum
Displacement

Uncertain
Parameter

yTxT

22/26

Response Spectrum

TransferLearning

【Result】Accuracy of TL-GPRSM
Predict Maximum Displacement（10 Trials）

1  yi − yˆi 
RMSPE= 100 ∑n i =1 yi ー TL-GPRSM
ー Without transfer learning
Predict for Piern2

Predict for Bearing
RMSPE (%)

RMSPE (%)

23/26

Number of data

Number of data

• In predicting the maximum displacement of the Pier, TL-GPRSM
was more accurate than the SM without transfer learning• In the prediction of the bearing, the presence or absence oftransfer learning did not affect the prediction accuracy.

24/26【Result】Predicted Distribution of Maximum Displacement
Predicted Distribution of Maximum Displacement（10 trials）
ー TL-GPRSM
ー 2DOF model
Predict for Pier

Predict for Bearing• For Pier, the maximum displacement distribution was predictable• For Bearing, the TL-GPRSM was able to roughly predict the maximum
displacement distribution, but was not able to properly predict thedistribution shape at the tail【Result】 parameter contribution estimation by ARD

25/26

Contribution of each part

Predict for Pier

Number of data

Estimated Contribution (%)

Estimated Contribution (%)

ー Common partー Target partー Source part
Predict for Bearing

Number of data

• In general, the contribution of the Common part was smaller than thesurrogate model to the analysis in the previous case, converging to about 4%or less.

Conclusion and Future work

26/26

Conclusion• A transfer learning Gaussian process regression surrogate model (TL-GPRSM) wasproposed and applied to evaluate the active load performance of a corrosiondamaged steel plate girder bridge by using design data for post-damage analysis• Looking at RMSPE, TL-GPRSM achieved a reduction in computation cost ofover 40%• The effectiveness of transfer learning was higher the greater the number ofsource data
• TL-GPRSM was used for seismic response analysis with different input seismic
motions, and the data obtained with the seismic design motion was used during theanalysis with the observed seismic motion• The accuracy of TL-GPRSM was slightly higher than without transfer learning• The contribution of the Common part, which measures the effect of transfer
learning, was generally lower than in the first case analysis

Future Work• Combined with adaptive sampling, which preferentially samples points that have asignificant impact on the performance of the surrogate model, the computational
cost could be further reduced