1
1

Title: Transfer learning Gaussian process regression surrogate model with explainability for2

structural reliability analysis under variation in uncertainties

3
4

Authors: Taisei SAIDAa* and Mayuko NISHIOb

5

aGraduate student, Department of Engineering Mechanics and Energy, University of Tsukuba

6bPh.D., Associate professor, Department of Engineering Mechanics and Energy, University of7

Tsukuba (ORCID number: 0000-0003-1079-2577)

89* Corresponding author

10

Email: saida.taisei.tj@alumni.tsukuba.ac.jp

11

Address: 1-1-1 Tennodai, Tsukuba 305-8573, Japan

12

Tel: +81-29-853-6192

13
14

Abstract:

15

In this paper, a Gaussian process regression surrogate model with transfer learning (TL-GPRSM) is

16

introduced to reduce the computational cost of structural reliability analysis by using the input–

17

output relationship of the source analysis having similarities with that of the target analysis. In

18

addition, automatic relevance determination (ARD) is introduced for providing the explainability of19

the constructed model and confidence assurance of transfer learning. Two verifications were20

conducted: (i) the surrogate modeling of the live-load performance analysis of a steel bridge with21

corrosion by applying the source analysis in the undamaged condition and (ii) seismic performance

ANN: Artificial neural network; ARD: Automatic relevance determination; COV: Coefficient of variation; DoE: Design ofexperiments; DOF: Degree of freedom; FE: finite element; GP: Gaussian process; GPR: Gaussian process regression; KS:
Kolmogorov—Smirnov; L-BFGS: Limited-memory Broyden–Fletcher–Goldfarb–Shanno; LHS: Latin hypercube sampling; MC:Monte Carlo; RMSPE: Root mean square percent error; TL-GPRSM: Gaussian process regression surrogate model with transfer
learning

2
22

analysis of a bridge pier, which required nonlinear time-history analyses for various earthquake loads.

23

The results showed that TL-GPRSM was especially effective in surrogate modeling of the24

performance analysis with linear numerical calculations. Moreover, it was shown that the25

effectiveness of transfer learning in each modeling and explainability of the constructed model could

26

be discussed based on the contributions of the model parameters estimated through ARD.

27
28

Keywords: surrogate model, Gaussian process regression, transfer learning, explainability, Monte

29

Carlo calculation, structural reliability analysis

30
31

1. Introduction

32

Structural performance evaluation requires reliability analysis wherein the limit state capacity is33statistically derived. It is useful to evaluate the performances of existing structures by considering the34

actual structure conditions to develop maintenance plans or implement reinforcement/retrofit

35

activities in disaster events. Statistical performance analysis requires the consideration of the36

uncertainties of the structural properties using Monte Carlo (MC) calculations. The MC calculations

37

require several to ten thousands of numerical calculations with a random sampling of the input

38

parameter space. In modeling an existing structure with deteriorations or damages, a detailed model

39

is required to represent the damage to the structural member; therefore, the number of model

40

parameters becomes large. Moreover, the properties to consider the uncertainties due to the41

deterioration or damage should be included in addition to the uncertainties of the nominal model

42

properties. In the case of performance evaluation under a disaster load such as an earthquake load,

43

the MC calculations of multiple input load cases should be combined to consider the input load

44

uncertainty. Therefore, high calculation effort is required to assess the structural performance.

45

The surrogate model is a regression model that alternates the numerical analysis constructed using

46

training data created through design of experiments (DoE) sampling of uncertain input parameters.

3
47

The total cost of MC calculation can be reduced if a small amount of training data can appropriately

48

represent the input–output relationship of the target numerical analysis. Numerous studies have been49

conducted on the application of the surrogate model to structural reliability analysis. The first

50

application of surrogate model for the structural reliability analysis was shown by Bucher and

51

Bourgund [1], where the performance function was approximated by the response surface method

52(RSM). The RSM has then been improved in many approaches, e.g., in studies by Kim et al. [2] and53

Zhao et al. [3]. Recently, support vector machine [4,5], polynomial chaos expansion [6,7], and54

artificial neural network (ANN) [8–10] are often adopted for surrogate modeling, and their55

effectiveness has been demonstrated. For instance, Marelli and Sudret [7] showed that the56

polynomial chaos expansion with active learning could construct surrogate models for structural

57

analysis at low computational cost. The applicability of Gaussian process regression (GPR) to

58

surrogate modeling has often been validated [11–14]. GPR [15] is a nonparametric regression

59

method and does not require the determination of the model configuration; moreover, the number of60

hyperparameters required for the estimation is small, and it can be applied to nonlinear input–output

61

relationships. For instance, Su et al. [12] conducted surrogate modeling by using GPR for the62

reliability analysis of bridges through finite element (FE) analysis. It was shown to be more efficient63

and accurate than the polynomial function-based response surface method. Avendaño et al. [14]64constructed a GPR surrogate model that effectively predicts wind turbine loads with accuracy of 4%

65

error or less. Adaptive learning method for GPR [16–22] have also been studied for efficient66

surrogate modeling of the structural reliability analysis, as surveyed by Moustapha et al. [23].

67

Adaptive learning is the method of incorporating additional learning points into the training data by

68

evaluating the learning function. Some studies, e.g., EGRA by Bichon et al. [16] and AK-MCS by

69

Echard et al. [17], successfully reduced the computational cost of surrogate modeling by implement70

adaptive sampling to the GPR. Since the AK-MCS was presented, Adaptive Kriging has been studied

71

actively, e.g., AK-IS by Echard et al. [21] and AK-SS by Huang et al. [22]. Many studies on472

surrogate modeling have been conducted, although most of them can be applied only to target

73

calculations. However, evaluations through numerical calculations to support decision-making in the74

engineering field require many similar calculations. The use of the surrogate model in such scenarios

75

will require new training data for the same modeling procedure for each calculation. In other words,

76

a surrogate model and its outputs cannot be used to construct other surrogate models.

77

Transfer learning (TL) is a machine learning technique in which the knowledge learned in a problem

78

is utilized in the target problem [24]. TL is expected to be applicable to the surrogate modeling of a79

target numerical calculation that shows any similarity to the source numerical calculation for which a

80surrogate model is already available; this approach may reduce the computational costs of81

constructing new surrogate models. Some cases require the repetition of numerical calculations for82

evaluations in the civil structural engineering field. For instance, when the target calculation is the83

performance analysis of an existing structure with damages, the surrogate model of the numerical84

calculation for the initial undamaged condition is expected to be used as the source in the TL. The

85

other is the seismic performance analysis of a structure considering various input earthquake loads.

86

Once a surrogate model for an input earthquake load is constructed, it can be used to construct the87

surrogate models for other earthquake loads with similar characteristics. If the TL works effectively

88

in these cases, the total computational cost of the structural performance evaluation is expected to be89

reduced.

90

Some previous studies have used TL in surrogate modeling. Xiong et al. applied TL to the deep91

learning model for the thermal analysis of spacecraft [25]. Kaya and Hajimirza applied ANN with TL

92

to the optimization problem of thin-film multilayer solar cells [26]. Tian et al. used TL to construct93

the variable-fidelity surrogate model with a deep neural network for the buckling analysis of a94

composite shell with seven design variables and a hierarchical reinforced shell with nine design

95

variables [27]. In all verifications, the prediction errors of the constructed surrogate models that used96

TL were significantly reduced. However, the studies [25–27] in which TL was used for the surrogate

597

model assumed that the source and target data in TL are similar from the beginning of the process. In

98

TL, the negative transfer problem is known to occur in some cases. Negative transfer refers to the99

issue wherein the performance of a machine learning model decreases owing to TL when the100

similarity between the source and target data, i.e., the input–output relationships between the source

101

and target numerical calculations, is low [24]. For the use of TL with confidence, the possibility of102

this negative transfer must be considered when it is generally unknown whether the source and target

103

numerical calculations are similar to each other.

104

In the engineering field, it is generally required to explain the results of physics numerical105

calculations for their appropriate use, i.e., provide explanations for the results from the viewpoint of

106

validity of the modeling; this requirement applies to surrogate models as well. The issue in most107

machine learning techniques, which are applicable to surrogate modeling, is that the training and108

prediction process is a black box. In the structural performance analysis using surrogate models, it is109

essential to know which uncertainties of the structural properties contribute significantly to the110

demand output to provide the explainability of the surrogate model. A previous study mentioned the111

necessity of demonstrating the explainability of the surrogate model. Golparvar et al. [28]112

constructed a surrogate model for predicting the offshore wind power using GPR with an automatic

113

relevance determination (ARD) kernel, which can estimate the contributions to the input parameters

114

by the hyperparameters in the ARD kernel. The ARD kernel [29,30] is the kernel function to115

represent the correlation function in the Gaussian process (GP) model, which can assign different

116

weights to each input dimension, and the corresponding length-scale is larger for input dimensions

117

that are irrelevant to the output. Wipf and Nagarajan [30] showed that ARD can be applied to the118

linear model estimation problem, where the inputs are sparse relative to the output, while retaining

119

the estimation accuracy. Owing to this feature, it is expected that the construction of GPR surrogate

120

models with an ARD kernel can estimate the parameter contribution to the output. However, very121

few studies have provided the explainability of the use of a machine learning technique to surrogate

6122

modeling, especially in the structural engineering field. Furthermore, no study has considered both123

TL and explainability in surrogate models.

124

This paper develops surrogate modeling using GPR with TL (TL-GPRSM). GPR is also applicable to125

the surrogate modeling of structural performance analysis, because it can deal with nonlinear input–

126

output relationships that may possibly emerge in the required structural analysis. In addition, as GPR

127

is a nonparametric method, the number of model parameters to be determined can be relatively low

128

when compared with that of other regression methods. Here, TL reduces the computational cost of129

surrogate modeling for structural performance analysis by considering the similarity in the input–

130

output relationships in the numerical analysis. We also propose the use of the ARD kernel for

131

providing the explainability of the constructed surrogate model and confidence assurance of TL. The

132

appropriately estimated ARD kernel hyperparameters are expected to be useful in evaluating the133

degree of the TL effect in each target surrogate modeling and solving the issue of negative transfer.

134

In this paper, the overviews of GPR and TL are presented, and the TL-GPRSM is formulated. Then,

135

two case studies are conducted to demonstrate the effectiveness of TL-GPRSM. The first is the live-

136

load performance evaluation analysis of a steel plate girder bridge. Here, the target domain is the137

maximum stress evaluation of the bridge with corrosion damages, and the source domain is that of138

the bridge in the initial undamaged condition. The second case study is on the seismic performance139

evaluation of a seismic isolation bridge pier that requires nonlinear time-history analysis. Here, the140target domain is the maximum displacement evaluation of the bridge pier and rubber bearing with141

the input as the recorded earthquake ground motion, and the source domain contains the inputs of142

ground motions provided in the design standard. The effectiveness of the TL-GPRSM is verified

143

based on the accuracy of the constructed surrogate models and effect of computational cost144

reductions. In addition, the contribution of TL is discussed using the similarity evaluation based on

145

ARD. Further, the contributions of uncertain parameters estimated through ARD to the output are

146

validated using structural engineering observations.

7147

2. Transfer training in GPR surrogate modeling

148

The TL-GPRSM developed in this paper is explained in this section. First, an outline of GPR with149

the ARD kernel function is presented, and the incorporation of TL into the GPR is demonstrated. In

150

addition, the application of the estimated ARD parameters to validate the constructed TL-GPRSM

151

and effect of TL is explained.

152
153

2.1 GPR with ARD kernel154

GPR [15] is a nonparametric regression method and applicable to various input–output relationships.

155First, suppose an input-and-output data matrix D with number of data N is defined as shown below.

156

𝐃𝐃 = {(𝐱𝐱1 , 𝑦𝑦1 ), (𝐱𝐱2 , 𝑦𝑦2 ), … , (𝐱𝐱𝑁𝑁 , 𝑦𝑦𝑁𝑁 )}𝑇𝑇 ,157

where x is a vector of the input parameters with length of L, and y is the output. The input–output

158

relationship is described as

(1)

159

𝑦𝑦 = 𝑓𝑓(𝐱𝐱).

160

Suppose that y is standardized to the zero-mean variable, and f is supposed to be generated from the161following GP with zero mean.

(2)

162

𝑓𝑓 ∼ GP�𝟎𝟎, 𝑘𝑘(𝐱𝐱, 𝐱𝐱′ )� .

163

Here, GP represents the Gaussian process and k is a kernel function used to calculate the kernel

164

matrix K, which has N rows and N columns.

(3)

165

𝐾𝐾𝑛𝑛𝑛𝑛 = 𝑘𝑘(𝐱𝐱𝑛𝑛 , 𝐱𝐱𝑚𝑚 ),166

where Knm is the element of K with n rows and m columns. Because f in Eq. (2) follows the GP, as

167

shown in Eq. (3), the output vector 𝐲𝐲 = (𝑦𝑦1 , 𝑦𝑦2 , … , 𝑦𝑦𝑁𝑁 )𝑇𝑇 also follows the Gaussian distribution with

168

zero mean and covariance matrix K.

(4)

169

𝐲𝐲 ∼ 𝒩𝒩(𝟎𝟎, 𝐊𝐊).

170

In this paper, we used the autoregressive relevance determination (ARD) kernel function [29]. In the171

ARD, the input parameters that contribute to the output can be determined automatically. Further, the

(5)

8
172

performance of regression in GPR depends on the selection of the kernel function. For instance, if

173

the Matern5/2 kernel is selected, ARD is applied as174

𝑘𝑘(𝑟𝑟, 𝛉𝛉) = 𝜎𝜎2 �1 +𝐿𝐿√√
(𝑥𝑥 − 𝑥𝑥 )2
55𝑟𝑟 + 𝑟𝑟2 � exp�− 5𝑟𝑟� , where 𝑟𝑟(𝐱𝐱𝑛𝑛 , 𝐱𝐱𝑚𝑚 ) = �� 𝑛𝑛𝑛𝑛 2 𝑚𝑚𝑚𝑚 .
𝑙𝑙𝑖𝑖
3
𝑖𝑖=1

(6)

175

Here, L is the number of dimensions of the input parameter vector, and θ is a vector of the176

hyperparameters σ and li. Among these hyperparameters, li is called the length-scale, which indicates177

the contribution of the input parameter xi in the ARD. The smaller the estimation of the length-scale178

li, the larger is the contribution of xi to the output y. The estimator of GPR with ARD is thus the179

hyperparameter θ, which includes the length-scale li. To verify the contribution of each parameter,

180

index ci was defined, which was the index converted from the length-scale li to the relative

181

percentage contribution in each input dimension i, as below.
𝑐𝑐𝑖𝑖 =

182

1⁄𝑙𝑙𝑖𝑖𝐿𝐿��1⁄𝑙𝑙𝑗𝑗 �

× 100(𝑖𝑖 = 1,2, ⋯ , 𝐿𝐿)

(7)

𝑗𝑗=1

183

In this study, the maximum likelihood estimation was used for estimating the hyperparameters of184

GPR model θ by adopting the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS)

185

method [31] as the optimization method. The GPR model construction was implemented using the186

Python library GPy [32].

187
188

2.2. TL in TL-GPRSM

189

TL in TL-GPRSM is implemented by the data expansion proposed by Daumé [33]. This method can

190

apply to TL in most machine learning models including GPR by expanding the matrix of input

191

parameters in training data to the common, source, and target parts. This makes possible to192

implement TL without losing advantages of GPR, applicability to nonlinear and nonparametric193

regressions. Furthermore, as the ARD kernel can be applied straightforward, the explainability of194

constructed TL-GPRSM is introduced including the effectiveness of TL.

9195

The input–output relationship of the target numerical analysis for surrogate modeling is called the196“target domain.” The “source domain” is the input–output relationship of the previously trained

197

model. For successful TL, the input–output relationship of the source domain is required to be

198

similar to that of the target domain. First, suppose that the source domain data Ds and target domain

199

data Dt are described as

200

𝑠𝑠
𝑠𝑠
𝐃𝐃𝑠𝑠 = �(𝐱𝐱1𝑠𝑠 , 𝑦𝑦1𝑠𝑠 ), (𝐱𝐱2𝑠𝑠 , 𝑦𝑦2𝑠𝑠 ), … , �𝐱𝐱𝑁𝑁, 𝑦𝑦𝑁𝑁
��,
𝑠𝑠
𝑠𝑠

(8)

201

𝑡𝑡
𝑡𝑡
𝐃𝐃𝑡𝑡 = �(𝐱𝐱1𝑡𝑡 , 𝑦𝑦1𝑡𝑡 ), (𝐱𝐱2𝑡𝑡 , 𝑦𝑦2𝑡𝑡 ), … , �𝐱𝐱𝑁𝑁, 𝑦𝑦𝑁𝑁
��,
𝑡𝑡
𝑡𝑡

(9)

202

where the vector x is the input variables vector and y is output value. The number of data in the203source domain is Ns and that in the target domain is Nt. The data expansion for TL is implemented by

204

configuring the expanded input vectors for the target domain xˆ t and source domain xˆ s using input

205

vectors xs and xt, respectively, as follows:

206
207
208

𝐱𝐱̂ 𝑠𝑠 = (𝐱𝐱𝑠𝑠 , 𝐱𝐱𝑠𝑠 , 𝟎𝟎) and𝐱𝐱̂ 𝑡𝑡 = (𝐱𝐱𝑡𝑡 , 𝟎𝟎, 𝐱𝐱𝑡𝑡 ).

(10)

The expanded input and output data DTL-GPRSM for TL-GPRSM are as follows:
𝑇𝑇

𝑠𝑠
𝑠𝑠
𝑡𝑡
𝑡𝑡
𝐃𝐃TL-GPRSM = �(𝐱𝐱̂ 1𝑠𝑠 , 𝑦𝑦1𝑠𝑠 ), (𝐱𝐱̂ 2𝑠𝑠 , 𝑦𝑦2𝑠𝑠 ), … , �𝐱𝐱̂ 𝑁𝑁, 𝑦𝑦𝑁𝑁, 𝑦𝑦𝑁𝑁�, (𝐱𝐱̂ 1𝑡𝑡 , 𝑦𝑦1𝑡𝑡 ), (𝐱𝐱̂ 2𝑡𝑡 , 𝑦𝑦2𝑡𝑡 ), … , �𝐱𝐱̂ 𝑁𝑁
�� .
𝑠𝑠
𝑠𝑠
𝑇𝑇
𝑇𝑇

(11)

209

TL can be realized by constructing a GPR model using the data matrix in Eq. (11) as training data.

210

The objective of this formulation is to expand the input vector x to three parts: “common part,”

211“source part,” and “target part,” as shown in Fig. 1. TL-GPRSM is then realized by constructing a212

GPR model using these expanded data. This means that the information of the source domain is used

213

in training the target domain model. Within the expanded input and output data, the common part214

considers both the source and target domains, the source part considers only the source domain, and215

the target part considers only the target domain. The contribution of each part to the constructed GPR

216

model indicates the utility of the TL. By adopting the ARD kernel for GPR, it becomes possible to217

derive the contribution of each part. Here, the contribution of each input dimension can also be218

derived using Eq. (7). The contribution for each part can be calculated by adding the contribution ci,

219

where i indicates the component corresponding to each part.

10

220

𝐿𝐿𝐶𝐶

𝐿𝐿𝑆𝑆

𝐿𝐿𝑇𝑇

𝑖𝑖=1

𝑖𝑖=1

𝑖𝑖=1

𝐶𝐶𝐶𝐶 = � 𝑐𝑐𝑖𝑖 , 𝐶𝐶𝑆𝑆 = � 𝑐𝑐𝑖𝑖 , and 𝐶𝐶𝑇𝑇 = � 𝑐𝑐𝑖𝑖 .

(12)

221

Here, CC, CS, and CT are the contributions of the common, source, and target parts, and LC, LS, and LT222

are the number of input dimensions of the common, source, and target parts, respectively.

223

Considering that the common part contributes to both the source and target domains, the relative

224

magnitude of CC against CT indicates the effect of TL in predicting in the target domain.

225

Input
CommonSourceOutputTargetSourceDomainTargetDomain226

Fig. 1 Data expansion for incorporating TL into GPR

227
228

2.3. Construction and validation of TL-GPRSM

229

In the construction of TL-GPRSM developed in this paper, the training and test datasets for the230source and target domains are created through numerical calculations of the structural model with the231

input parameter sets generated by Latin hypercube sampling (LHS) [34]. The hyperparameters of232

GPR are then estimated using L-BFGS optimization.

233

The performance of constructed surrogate model was evaluated by the prediction error with respect234

to the test data and the accuracy of predicted probability distribution of output, which is required for235

the structural reliability analysis. It is particularly important for the surrogate modeling for structural236

reliability analysis to be able to predict the appropriate distribution of output response. The root237

means square percent error (RMSPE) was used for the evaluation of the prediction error. The

11
238

agreement of distributions was evaluated using not only the overlay of cumulative distributions but

239

also the p-value in the Kolmogorov–Smirnov (KS) test. In each verification case, the surrogate

240

model was constructed for the ten training datasets created through ten trials of LHS sampling, and241

the variations in the accuracy and probability distributions were derived for evaluating the modeling242

stability.

243

The computational cost of surrogate modeling was evaluated by the required number of times of244

numerical calculations of the structural model (e.g., FE model) to predict the accurate distribution of

245

output response. Here, it should be noted that the computational cost of prediction using constructed

246

surrogate model is negligible compared to the computational cost of numerical analysis of the247

structural model. The TL-GPRSM is expected to realize reducing the number of training data in the248

target domain by taking advantage of data in the source domain, which has been already existed.

249

The explainability of the constructed surrogate model and confidence assurance of the TL were then250

evaluated based on the percentage contributions derived using the estimated length-scales in the251

ARD kernel. Structural reliability analysis is the procedure of considering the uncertainties of252

multiple parameters for evaluating the limit state capacity. However, the number of parameters

253

contributing to the output required for the performance evaluation is actually limited in many cases.

254

This sparsity of the contribution can be estimated by using the ARD kernel. The contribution of each

255

input dimension ci, derived using Eq. (7) with the estimated length-scale li, indicates the extent to256

which the uncertainty of the corresponding model parameter affects the demand output. The

257

constructed surrogate model can be accepted when the contributing model parameters can explain258

the structural response output behaviors from the viewpoint of the physics of the structure.

259

Furthermore, the contributions of the common, source, and target parts, which are the summed

260

contributions of the dimensions belonging to each part, calculated using Eq. (12), is used for261

evaluating the effectiveness of TL. Here, TL is considered effective when the contribution of the262

common part Cc against CT is relatively high.

12
263
264

2.4. Application example of TL-GPRSM

265

A simple application example is shown to recognize the significance of TL-GPRSM. Here, the266function models, fs and ft, are given as the source domain and the target domain, respectively.

267

𝑓𝑓𝑠𝑠 (𝐱𝐱) = 2𝑥𝑥1 2 + 𝑥𝑥2 2 + 0.00001𝑥𝑥3 + 5

(13)

268

𝑓𝑓𝑡𝑡 (𝐱𝐱) = 2𝑥𝑥1 2 − 2 (𝑥𝑥2 − 1)sin 𝑥𝑥2 + 0.00001𝑥𝑥3 + 5

(14)

269

where x1, x2, and x3 are input variables with the range of −1≤ xi ≤1 (i =1−3), and the input vector is270

configured as x=(x1, x2, x3). Seeing the similarity between those two function models, only the terms271

of x2 acts on the output differences between the models of source domain fs and target domain ft, and272

contributions of the terms of x1 and x3 and the constant terms are the same in both models. In

273

addition, the coefficient of each term indicates how much that term contributes to the output of274

function. It is obvious that coefficients on x3 are relatively small in both models.

275

First, the data sets of function input-output relationships for source domain fs and target domain ft are

276

created with the numbers of data Ns =100 and Nt =10, respectively. The samples of input variables277

space are plotted in Fig.2, where they are shown in the spaces of x1 and x2 because x3 less contributes

278

to the function output, as mentioned above. The data matrix of source domain Ds and target domain

279

Dt are represented as:

280

𝑠𝑠
𝑠𝑠
𝐃𝐃𝑠𝑠 = {�𝐱𝐱1𝑠𝑠 , 𝑓𝑓𝑠𝑠 (𝐱𝐱1𝑠𝑠 )�, �𝐱𝐱2𝑠𝑠 , 𝑓𝑓𝑠𝑠 (𝐱𝐱2𝑠𝑠 )�, … , �𝐱𝐱100, 𝑓𝑓𝑠𝑠 (𝐱𝐱100)�}𝑇𝑇 .

(15)

281

𝑡𝑡
𝑡𝑡
𝐃𝐃𝑡𝑡 = {�𝐱𝐱1𝑡𝑡 , 𝑓𝑓𝑡𝑡 (𝐱𝐱1𝑡𝑡 )�, �𝐱𝐱2𝑡𝑡 , 𝑓𝑓𝑡𝑡 (𝐱𝐱2𝑡𝑡 )�, … , �𝐱𝐱10, 𝑓𝑓𝑡𝑡 (𝐱𝐱10)�}𝑇𝑇 ,

(16)

282
283
284
285

Then, the expanded input vectors for implementing TL are configured following Eq. (10), as:𝐱𝐱̂ 𝑠𝑠 𝑖𝑖 = (𝐱𝐱𝑠𝑠 𝑖𝑖 , 𝐱𝐱𝑠𝑠 𝑖𝑖 , 𝟎𝟎𝟏𝟏×𝟑𝟑 )(𝑖𝑖 = 1 − 100) and 𝐱𝐱̂ 𝑡𝑡 𝑗𝑗 = (𝐱𝐱𝑡𝑡 𝑗𝑗 , 𝟎𝟎𝟏𝟏×𝟑𝟑 , 𝐱𝐱𝑡𝑡 𝑗𝑗 ) (𝑗𝑗 = 1 − 10).

(17)

The data matrix for TL-GPRSM is given as follows:
𝐃𝐃𝑇𝑇𝑇𝑇−𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺 = {�𝐱𝐱̂ 𝑠𝑠 1 , 𝑓𝑓𝑠𝑠 (𝐱𝐱1𝑠𝑠 )�, �𝐱𝐱̂ 𝑠𝑠 2 , 𝑓𝑓𝑠𝑠 (𝐱𝐱2𝑠𝑠 )�, … , �𝐱𝐱̂ 𝑠𝑠 100 , 𝑓𝑓𝑠𝑠 (𝐱𝐱𝑠𝑠100 )�,�𝐱𝐱̂ 𝑡𝑡 1 , 𝑓𝑓𝑡𝑡 (𝐱𝐱1𝑡𝑡 )�, �𝐱𝐱̂ 𝑡𝑡 2 , 𝑓𝑓𝑡𝑡 (𝐱𝐱𝑡𝑡2 )�, … , �𝐱𝐱̂ 𝑡𝑡 10 , 𝑓𝑓𝑡𝑡 (𝐱𝐱𝑡𝑡10 )�}𝑇𝑇

(18)

286

The obtained surrogate model for target domain ft is then evaluated by predicting the distribution of287

function outputs to 1000 samples generated from the space of input variables xi with range of −1≤ xi

13
288

≤1 (i =1−3). For comparison, a GPR surrogate model without TL is also constructed only using the289

10 data from the target function model ft. The predicted distributions are compared in Fig.3 (a). Here,290

the distribution of function outputs created by assigning the same 1000 samples of input variables to291

the target function model ft is also shown as “True” case. The TL-GPRSM can predict the292

distribution closer to the true distribution than the one predicted from the GPR model without TL.

293

The RMSPE accuracies are 0.13% and 15.2% in TL-GPRSM and GPR without TL, respectively.

294

Also, the function field of ft predicted by the TL-GPRSM in Fig. 3(b) clearly shows good agreement295

with the true function field, which is shown in Fig. 2(b), compared to the function field predicted by296

the GPR without TL in Fig. 3(c).

297

The contribution of each input variable in each of "Common," "Source," and "Target" parts is then298

evaluated by calculating cj using Eq. (7) as:
𝑐𝑐𝑗𝑗 =

299

1/𝑙𝑙𝑗𝑗× 100
9∑𝑗𝑗=1 �1/𝑙𝑙𝑗𝑗 �(𝑗𝑗 = 1, ⋯ ,9),

(19)

300

where lj is length-scale in ARD kernel, and subscript j indicates the order of each input parameter in

301

expansion data, i.e., j=1−3 indicates three input variables x1-x3 in “Common” part, j=4−6 indicates302

those in “Source” part, and j=7−9 are those in “Target” part. Those parts correspond to the303“Common”, “Source”, and “Target” parts presented in Fig.1. Figure 4 shows calculated contribution

304

index cj. In the “Common” part, only input variable x1 has non-zero value. This indicates that the305

terms of x1 contribute similarly to both function outputs of the source domain fs and the target

306

domain ft. On the other hand, there is no contribution in x2 in “Common” part, and the term of x2

307

shows significant contribution in each of “Source” and “Target” parts. These points are308

understandable because the terms of x1 are the same in the two function models, fs and ft, and the309

terms of x2 are different between them, as shown in Eqs. (13) and (14). In addition, almost zero

310

values in x3 in all parts indicate consistency with the point that the terms of x3 have small coefficients311

in both model functions.

14
312

The effect of TL can be determined by the summation of contribution cj in each part, which are CC,

313

CS, and CT expressed in Eq. (12). The calculated values of CC, CS, and CT are 39.5, 27.1 and 33.4,

314

respectively. It can be said that the TL worked effectively for the appropriate surrogate modeling315

from the point that the relative value of common part contribution CC against target part contribution

316

CT was large in this application example.

317

318
319

(a) Source domain (data samples NS =100)(b) Target domain (data samples NT =10)
Fig. 2 Function field of source and target domain models and created training data (described in “×”)

320

321
322

(a) Distributions of function outputs

15

323
324
325

(b) Predicted function field (TL-GPRSM)(c) Predicted function field (GPR without TL)
Fig. 3 Prediction performance of constructed surrogate models for target function model ft

326

327
328

Fig. 4 Estimated contributions of model parameter uncertainties(Left: Common part, Center: Source part, Right: Target part)

329
330

3. Live-load capacity evaluation of plate girder bridge

331

TL-GPRSM is expected to be applicable to the case wherein the number of input parameters and332

their uncertainties change in the source and target domains of the TL. In existing structures, the333number of uncertain parameters and their degrees of uncertainties may change from those at the time334

of new construction owing to damage or deterioration. In this case, if TL-GPRSM is effective, the335

computational cost of the reliability analysis can be reduced by using the data saved in the design336phase. In this section, we use the analytical model of a steel plate girder bridge with damaged

337

sections. The surrogate model was constructed by applying TL-GPRSM to the data before and after338

the damage, and its effectiveness in reducing the computational cost was verified. The effectiveness339

of TL and adaptation to the sparsity of input parameters using ARD contribution estimation were also

16
340

verified.

341
342

3.1 FE modeling and evaluation analysis

343

The numerical analysis as the target of surrogate modeling in this paper was the structural reliability

344

analysis of a simply supported steel plate girder bridge considering the damage of corrosion at the345

end of the main girder, which was conducted in a previous study by the authors [35]. The bridge is a346

testbed structure for the numerical study designed based on the Japanese design standard [36]. The

347

bridge consists of five main I-shape steel girders, a reinforced concrete (RC) slab, and steel bearings

348

with a span length of 20.0 m and width of 10.7 m. The FE model of the bridge was constructed using349

the general-purpose FE analysis software Abaqus 6.14. Figure 5(a) shows the overall view of the350

constructed whole-bridge FE model. Here, the five main girders and concrete slab were modeled

351

using the shell element, and the members of the steel bearings were described using the solid element.

352

The output response for the performance evaluation was the maximum Mises stress at the region

353

near the ends of each main girder under static loading of the designed live load introduced in the354Japanese design standard [36]. The convergence of the mesh size to the output response was verified

355

to determine the mesh size configuration for the whole-bridge model; however, the determined

356

number of elements became excessively large. Therefore, the half-girder model was prepared to357

reduce the computational cost, as shown in Fig. 5(a). Here, the previously determined mesh358

configuration was retained, and the continuity condition at the mid-span cross-section was applied359

with symmetry preservation. In addition, the input live load was adjusted to obtain the stress360

distribution at the main girder as the output, which was consistent with that obtained in the whole-361

bridge model analysis. The total number of elements in the half-girder model was 104,799. Most

362

parameters of the initial model without damages were determined based on the nominal properties of363

steel and RC used in the bridge design. The friction coefficient of the contact surface between the364upper and lower parts of the steel bearing was set to 0.2, which is the generally used value of the17coefficient for metal contact surfaces. In the constructed initial FE model, the static deflection under366

the designed live load satisfied the requirement for the limit state in the design standard [36]. Then,367

the FE model for the performance evaluation under the damaged condition was constructed. The

368

corrosion at the main girders and bearings was represented by setting the individual areas in the main369

girders, as shown in Fig. 5(b), to reduce the thickness at the corroded areas and by increasing the370

friction coefficients at the corroded bearings, respectively. The effect of cracks at the concrete slab371

was considered by reducing the Young’s modulus of the elements of the slab. The input–output

372

relationship of the initial FE model is the source domain, and that of the FE model with damages is373

the target domain in the TL-GPRSM.

400

365

374
375
376

(a) Full bridge model and its half-girder model(b) Corroded area
Fig. 5 Testbed bridge FE model for verification

377
378

3.2 Uncertainties of model parameters

379

In the TL-GPRSM, the increase in the uncertainties in the structural properties due to damages is

380

considered through TL. The parameter uncertainties were determined both in the initial FE model as381

the source model and in the damaged-condition FE model as the target model. In the initial FE model,

382

sixteen parameters #1–#16 were considered, and their uncertainties were represented by the uniform

383

distributions with nominal mean and coefficient of variation (COV), as shown in Table 1. Here, the384mean and COV values were determined based on the statistical properties reported in previous

18
385

research papers and surveys, as summarized in the authors’ previous study [35]. For the damaged-386

condition FE model, three parameters #17–#19 were added to represent the effect of corrosion, that is,387

the thickness reduction at the corroded areas. In addition, the probability distribution of the friction388

coefficient #8 (Cf) was changed to represent the decrease in the moving function due to the corrosion,389

and the distribution of the Young’s modulus of the concrete slab #3 (Ec) was changed to consider the390

effect of cracks.

391

Figure 6 shows the distributions of the maximum Mises stress near the ends of the girders for the392

performance evaluation using MC calculations with 500 samples from the space of uncertain model

393

parameters in the initial- and damaged-condition FE models. Here, the 500 samples were generated

394

through LHS. The maximum Mises stress in the damaged-condition FE model is distributed in the395

range of higher stress values when compared with the distribution in the initial FE model. It can be396

said that the limit state capacity of yield stress is reduced owing to the damages. The distribution of397

the maximum Mises stress in the damaged-condition FE model is the target output of the surrogate

398

modeling using the TL-GPRSM. The surrogate model for the initial FE model with sixteen uncertain

399

model parameters as the inputs is the source domain, and that for the damaged-condition FE model

400

with nineteen parameters as the inputs is the target domain. Although the number of input parameters401

is different between the source and target domains, the TL-GPRSM is applied by data expansion with402

16 dimensions for the common part, 16 dimensions for the source part, and 19 dimensions for the403

target part.

404
405
406
407
408
409

19
410
411

Table 1 Uncertainties of FE model parameters (−: N/A, *: the same as in the initial FE model)
FE model parameter (Unit)

412

Initial (Source)

Damaged (Target)

NominalCOV

Nominal

COV

#1

Dc

Density of concrete slab (kg/m3)

2400

0.0171

*

*

#2

Es

Young's modulus of steel main girders
(GPa)

200

0.0450

*

*

#3

Ec

Young's modulus of concrete slab (GPa)250.016722.5

0.0333#4Eb

Young's modulus of steel bearings (GPa)200

0.0450

*

*

#5

Vs

Poisson’s ratio of steel main girder

0.3

0.0910

*

*

#6

Vc

Poisson’s ratio of concrete slab

0.2

0.0167

*

*

#7

Vb

Poisson’s ratio of steel bearing

0.3

0.0910

*

*

#8

Cf

Friction coefficient of steel bearing

0.2

0.01670.9

0.0333#9Tuf1

0.01900.0121

*

*

#10

Tuf2

0.03000.0121

*

*

#11

Tw

0.00900.0121

*

*

#12

Tbf1

0.02700.0121

*

*

#13

Tbf2

0.03000.0121

*

*

#14

Tstc

0.01300.0121

*

*

#15

Tstm

0.01000.0121

*

*

#16

Tstn

0.00650.0121

*

*

#17

Tbf-d

−

−

0.025

0.0270

#18

Tw-d

−

−

0.008

0.0162

#19

Tst-d

−

−

0.012

0.0162Thickness of upper flange of steel girder atnear-end section (m)Thickness of upper flange of steel girder at
mid-span section (m)Thickness of web plate of steel girder (m)Thickness of lower flange of steel girder atnear-end section (m)Thickness of lower flange of steel girder at
mid-span section (m)Thickness of stiffener of steel girder atnear-end section (m)Thickness of stiffener of steel girder at
mid-span section (m)Thickness of stiffener of steel girder atother sections (m)Thickness of corroded area in lower flangeof steel girder at near-end section (m)Thickness of corroded area in web plate ofsteel girder (m)Thickness of corroded area in stiffener ofsteel girder at near-end section (m)

20

413
414
415

Fig. 6 Distributions of maximum von Mises stress in the steel girder, derived using Monte Carlocalculation of FE models

416
417

3.3 TL for FE model analysis in damaged condition

418

The training dataset in the source domain was configured using the sampled 16-dimension uncertain

419

model parameter sets and their output responses, i.e., the maximum Mises stress near the end of the420

girder, in the initial FE model. The dataset in the target domain comprised the inputs sampled from421

the space of 19-dimension uncertain parameters and their output responses in the damaged-condition

422

FE model. The TL-GPRSM was constructed using the input-output data from the FE analysis in the423

initial condition as the source domain and the input-output data from the FE analysis in the damaged424

condition as the target domain.

425

In the verification, a fundamental dataset that consists of 500 input–output relationships was

426

prepared by conducting FE analysis for 500 input model parameter samples to configure the training

427

and test datasets in the target domain. For N number of training data, N input–output relationships

428

were randomly selected from the fundamental dataset, and the remaining were used as the test data.

429

The accuracy of the surrogate model was verified using the RMSPE. The p-value of the KS test was

430

also derived to evaluate the agreement between the maximum Mises stress distributions from the431

surrogate model and those from the MC calculation using the FE model. In each case, surrogate

432

modeling was performed for ten training datasets for evaluating the modeling stability.

21
433

The number of training data in the source domain, i.e., the initial FE model, was varied to evaluate434

how it affected the accuracy of the TL-GPRSM by setting the cases of 10, 30, and 100 source

435

domain data. The average of RMSPEs for ten GPR surrogate models for the initial FE model in each436

case was 3.1 in the case of 10 data, 0.10 for 30 data, and 0.0096 for 100 data. It is known that the437

accuracy of regression increases with the amount of data. The target surrogate model constructed438

using the TL-GPRSM was then verified. The RMSPE of the constructed surrogate model for the439

damaged-condition FE model is shown in Fig. 7. Here, the horizontal axis is the number of training440

data in the target domain. The red line is the accuracy obtained by the TL-GPRSM, and blue line is441

the result obtained using only the data from the damaged-state FE model. The average and variance442

of ten surrogate models constructed by ten times samplings were plotted. For each number of source443

domain data, the RMSPE in the TL-GPRSM converged to a low value faster than that of the444

surrogate model constructed using only the data from the damage-condition FE model. This indicates

445

that the use of TL-GPRSM can reduce the number of numerical calculations of the damage-condition

446

FE model if there are numerical results from the initial FE model. As the number of source data447

increases, the RMSPE value and its variation become smaller, i.e., the accuracy and stability of448

surrogate modeling improve. For all numbers of source domain training data, a surrogate model with449

accuracy of less than 1% can be obtained by using 15 training data in the target domain. For instance,

450

considering the total computational cost, the accuracy of the surrogate model with 1% RMSPE can451

be achieved by using 10 source data and 15 target domain data in the TL-GPRSM. Although this452

accuracy can be obtained by the target-only surrogate model using 25 training data from the453

damaged-condition FE model, TL-GPRSM ensures higher stability.

454

The distributions of the maximum Mises stress derived from the constructed surrogate models are455

depicted in Fig. 8(a). The red lines denote the cumulative distributions of the maximum Mises stress456

predicted from the surrogate models constructed by the TL-GPRSM using 30 source data and 15

457

target data. The distributions obtained by ten TL-GPRSMs show good agreement with the

22
458

distribution obtained by the MC calculation of the FE model, which is shown as a black solid line.

459

The blue lines are the distributions derived through GPR using only the 15 target domain data, which460

do not agree with the distribution obtained with the FE model MC calculation. Hence, the prediction

461

stability of TL-GPRSM is much higher than that in the case of GPR modeling using only 15 target

462

domain data. In addition, the average of R2 index, which is to evaluate regression performance, over463

ten trials, was 0.98 in TL-GPRSM; on the other hand, that in GPR surrogate model without TL was464

0.57. The R2 index also indicates that the TL-GPRSM realized more accurate surrogate modeling.

465

The plots of the p-values in the KS test are also shown in Fig. 8(b). The high p-value indicates that466

the similarity between the two distributions corresponding to the surrogate model and FE model is467

high. As the number of target domain training data increases, the p-value in the TL-GPRSM reaches

468

a high value faster than that in the case of the GPR model using only the target domain data. It can be469

concluded that high accuracy and high stability can be obtained using the surrogate model

470

constructed with the TL-GPRSM.

471

Figure 9 shows plots of CC, CS, and CT expressed in Eq. (12) against the number of target domain

472

training data to evaluate the effect of TL. In each plot, the red line shows the relative value of the473

length-scale sum of the common part, and the blue and green lines show those of the target part and474

source part, respectively. Each plot represents the ten-time average for each number of target data. In

475

these plots, the higher the value of the common part when compared with that of the target part, the476

greater is the effect of the TL. In Fig. 9 (a)-(c), the contribution values of all parts are almost

477

converged for the number of target domain data of 10–15 in all cases of the source domain data.

478

Moreover, the effect of TL increases as the number of source domain data increases. The significance479

of TL is enhanced when a larger number of training data is used in the source domain.

480

Figure 10 shows the contribution of each uncertain model parameter in the constructed surrogate

481

model, that is, the relative value of the inverse length-scale estimated in the TL-GPRSM with 30

482

source domain data and 15 target domain data. The results are shown separately for the expanded

23
483

data vector's common, source, and target parts. First, the friction coefficient at the bearings, Cf,

484

shows the highest contribution in the common part. This is interpreted as the similarity in the485

contribution of parameter Cf to the output in the target and source domains. It can also be seen that486

the contributions of these parameters are sparse, i.e., some parameters show little contribution;

487

therefore, the significant parameters in the input–output relationship of the target surrogate model488

can be extracted from this figure. In the initial state analysis of the source domain, the parameters489

contributed significantly to the overall bending behavior. Regarding parameters with high490

contributions, the contribution of the steel girder web thickness Tw, which is related to the overall

491

bending stiffness of the girder, was significant. The parameters Tbf1, Tstc, and Eb representing the492characteristics near the girder ends, which are correlated with the boundary conditions, also had an493

effect. This result is reasonable based on structural engineering findings. However, in the source494

domain, which is the damaged-state analysis, the contribution was large for parameters related to495

localized deterioration damage, such as lower flange thickness Tbf-d and web thickness Tw-d at the496

girder end corrosion area. These estimated contributions are consistent with the findings in structural

497

engineering that the stress at the girder end increases as the corrosion loss progresses. The validity of498

these estimated contributions supports the explanatory power of the TL-GPRSM.

499

500
501
502

(a) Source domain data: 10

24

503
504

(b) Source domain data: 30

505

506
507
508

(c) Source domain data: 100
Fig. 7 Accuracy and stability of surrogate model against the number of target domain data

Cumulative probability

509

510
511
512
513

Max Mises stress (N/m 2)(a) Comparison of cumulative distributions (TL-GPRSM: 30 source domain data and 15 target
domain data, Target only: 15 target domain data)

25

514
515

(b) Plot of p-values in KS test (Source domain data: 30)

516

Fig. 8 Agreement of predicted maximum Mises stress distributions

517

518
519

(a) Source domain data: 10

520

521
522
523

(b) Source domain data: 30

26

524
525
526

(c) Source domain data: 100
Fig. 9 Contributions of common, target, and source parts in TL

527

528
529
530
531

Fig. 10 Estimated contributions of model parameter uncertainties to the maximum Mises stress in thesteel girder (number of source data: 30, number of target data: 15) (Left: Common part, Center:
Source part, Right: Target part)

532
533

534

4. Performance evaluation of seismic isolated bridge pier535

Another application of the structural reliability analysis is performance evaluation against disaster536

loads such as earthquakes. Here, the structural response analysis requires a nonlinear calculation, and537

must be performed for various external loads that are probabilistically determined. In this section, the538

TL-GPRSM was verified using the earthquake response analysis for the seismic performance539

evaluation of an isolated RC bridge pier. In this section, TL-GPRSM is constructed with the source

27
540

domain as the input–output of seismic response analysis with the designed earthquake loads, and the541

target domain as the input–output of seismic response analysis for a certain input observed

542

earthquake load. We verified whether the computational cost of the seismic performance analysis of543

the structure could be reduced by using the prepared data created based on numerical analysis with544

the designed earthquake loads.

545
546

4.1 Modeling and parameter uncertainties

547

The numerical model of a seismic isolation RC pier used in this verification corresponds to the548

dynamic seismic design shown in the design standard of road bridges in Japan [37]. Figure 11(a)549shows the overall view of the bridge, and the target is an RC pier with seismic rubber bearing,550

indicated as P1. This pier was modeled using the two degree-of-freedom (DOF) lumped-mass model

551

shown in Fig. 11(b). The masses of the superstructure and RC pier were assigned to the upper and552

lower lumped-masses of the 2DOF system, respectively, and the seismic isolation rubber bearing was

553

modeled as a horizontal spring with nonlinear characteristics described using the bilinear model. The

554

nonlinear stiffness of the RC pier was described using the Takeda model [38]. The nominal

555

parameters were determined based on the values of the bridge properties introduced in the design

556

standard [37], as shown in Table 2. The uncertainty of each model parameter was represented as a557

uniform distribution with upper and lower limits of ±10% from the nominal value.

558

559
560

(a)

Overall view of the target bridge [37]

28

561
562

(b) 2DOF lumped-mass model of an isolated RC pier563

Fig. 11 Illustrations of target structure

564
565

4.2 Input earthquake loads and nonlinear time-history analysis

566

The input earthquake loads for the target domain in the TL-GPRSM were the designed ground567

motion in the design standard [39] and two observed earthquake ground motions recorded as JMA

568

Kobe for the Kobe earthquake in 1995 and KAIHOKUBASHI for the Tohoku earthquake in 2011, as

569

shown in Table 3. As the source domain, the designed earthquake ground motion, called Level-2

570

ground motion, was adopted, which was used for evaluation based on the time-history analysis. Two

571

types of Level-2 ground motions were considered, namely, Type-1 for the plate boundary type572

earthquake and Type-2 for the inland earthquake. In the design standard [39], three ground motions

573

were prepared for each of Type-1, Type-2, and three ground classifications. Here, the seismic

574

isolation bridge was allowed to be constructed on hard ground; hence, the earthquake ground575

motions for the corresponding ground classification were used: Type1-1-1/2/3 and Type2-1-1/2/3.

576

For the source domain in all cases, the first ground motions of the two types, Type1-1-1 and Type2-

577

1-1, were adopted. As shown in Table 3, the second ground motion of Type-2, Type2-1-2, was set for578

the target domain in Case #1. One of the purposes of this verification was to investigate how the579

similarity in the input earthquake ground motion affected the performance of the TL-GPRSM. It

580

should be noted that JMA KOBE was classified as an inland earthquake (Type-2) and581

KAIHOKUBASHI as a plate boundary type earthquake (Type-1). The acceleration response spectra582

for all ground motions for the target and source domains are shown in Fig. 12. In Type-1 ground

29
583

motion, high response was observed in the low-range period of less than 0.3 s, and the dominant584

period in Type-2 ranged from approximately 0.3 to 0.8 s. For the nonlinear time-history analysis, the585time increment was set to 0.001 s, and the Newmark-β method (γ = 0.5 m, β = 0.25) was adopted for586

the numerical integration. The structural damping was assumed to be Rayleigh damping with the587

component damping coefficient of 0% for the seismic isolation bearing and 2% for the RC pier.

588

Figure 13(a) and (b) show the response hysteresis of the RC pier and seismic rubber bearing obtained

589

from the results of the time-history analysis for the input cases of JMA KOBE and590

KAIHOKUBASHI, respectively. Although both earthquake inputs had the same intensity level as the591

designed input earthquake for each earthquake type, and both showed the maximum displacements in592

the pier and bearing, the occurrences of nonlinearity were different. In Fig. 13 (a) for JMA KOBE,593

not only the rubber bearing but also the RC pier showed plasticized responses. However, the RC pier594

response was in the elastic range by utilizing the rubber bearing in Fig. 13(b) corresponding to595

KAIHOKUBASHI. It is well known that the nonlinear response of each member strongly depends

596

on the input ground motion. The TL is expected to effectively consider the input–output relationship597

of each earthquake type trained in the source domain, in the target domain training.

598
599

Table 2 Uncertain parameters of the seismic isolation bridge pier modelParameter
Superstructure

Seismic isolation
bearing

RC Pier

600

NominalMass (Mu)

604000 kg

Primary stiffness (Kb1)

40023.2 kN/m

Secondary stiffness (Kb2)6154.4 kN/mYield load (Qb)1117.2 kN

Mass (Mrc)

346300 kg

Primary stiffness (Krc1)

110000 kN/m

Secondary stiffness (Krc2)

8250 kN/mYield load (Qrc)

3399 kN

Uncertainty± 10 %

30
601

602

Table 3 Target and source domain settings for verification
Case #

Target domain

Source domain

Case #1

Level2 Type2-1-2

Level2 Type1-1-1 (200 data)
Level2 Type2-1-1 (200 data)

Case #2

JMA KOBE (Type-2)

Level2 Type1-1-1 (200 data)
Level2 Type2-1-1 (200 data)

Case #3

KAIHOKUBASHI (Type-1)

Level2 Type1-1-1 (200 data)
Level2 Type2-1-1 (200 data)

603

Type1-1-1
Type2-1-1
Type2-1-2
JMA KOBEKAIHOKUBASHI

Max acceleration (m/s 2)

25
20
15
10
5
0

604
605

0

0.5

1

Period (s)1.5

Fig. 12 Response spectra of earthquake ground motions

606

607
608

2

(a) JMA KOBE (left: pier, right: bearing)

31

609
610

(b) KAIHOKUBASHI (left: pier, right: bearing)

611

Fig. 13 Response hysteresis with input earthquake ground motions in target domain in Cases #2 and
#3

612
613
614

4.3 Surrogate model construction using TL-GPRSM

615

Figures 14−16 show the accuracies and effects of TL in the constructed surrogate models in Cases

616#1−#3. In this verification, all surrogate models were constructed using 200 Type1-1-1 data and 200

617

Type2-1-1 data, with the data having a higher TL effect as the source domain data, as shown in Table

618

3. For setting the training and testing datasets in the target domain, a fundamental dataset was

619

prepared by creating 10,000 input–output relationships of the nonlinear time-history analysis of the620

2DOF lumped-mass model. In each figure, plot (a) depicts the comparison of the accuracies based on

621

RMSPE between the surrogate models constructed using the TL-GPRSM and those constructed622

using GPR with only the target domain data, plotted against the number of target domain training

623

data. Plot (b) shows the relative contributions of the common, source, and target parts, calculated624

from the estimated length-scales of ARD, and plot (c) depicts the comparison of the contributions of625

the common part for each of the two source domains: Type1-1-1 and Type2-1-1.

626

In Case #1 with the target domain of Type2-1-2, the RMSPE of the TL-GPRSM is slightly lower

627

than that of the surrogate model constructed using GPR in the range of number of target domain data

628

smaller than 50; further, the accuracies in the cases with number of training data over 50 show little

629

difference, as shown in Fig. 14(a). This means that the TL was not useful in ensuring effective

630

surrogate modeling. From the plots in Fig. 14(b), the contribution of the common part to the target

32
631

part was almost constant, ranging from 10 to 15%, beyond the number of training data of 10. Figure632

14(c) shows that the contribution of Type2-1-1 was higher than that of Type1-1-1. This means that633

the output of the maximum displacement of the RC pier mainly considers the source domain of634

Type2-1-1, which is the same type of input ground motion as that of the target domain. However, the635

output of the rubber bearing mainly considers Type1-1-1.

636

In Case #2 with the target domain of JMA KOBE, the surrogate models by TL-GPRSM show

637

slightly higher accuracy for both the RC pier and isolation bearing, especially when approximately

638

30 target domain data are used, as shown in Fig. 15(a). Figure 15(b) shows that the relationship639

between the contributions of the common and target parts is constant, as seen in Case #1, and the640ratios of those two contributions are similar. In Fig. 15(c), the contribution of the common part for

641

each of the two source domains shows the same trend as that in Fig. 15(c), both for the RC pier and642

seismic isolation bearing; however, the difference between the contributions of Type1-1-1 and643

Type2-1-1 decreases.

644

In the case of KAIHOKUBASHI in Fig. 16(a), the accuracy of the TL-GPRSM for the RC pier645

almost overlaps that of the surrogate model constructed using only the target domain training data.

646

However, the accuracy of the TL-GPRSM for the bearing improves, especially when the number of

647

training data is less than 100. Considering the range in Fig. 16(b), the contribution of the common

648

part to the target part is relatively high. This indicates that the TL was effective in the surrogate

649

model construction for the RC pier in this case. However, the RMSPE of the bearing surrogate model650

was approximately three times higher than those of all the other surrogate models, including those of651

the two previous cases. Figure 16(c) does not show a clear trend in which source domain data were652

selected in the TL for the surrogate model of the RC pier; however, for the bearing, there was a653

tendency to select the source domain of Type2-1-1, although the KAIHOKUBASHI ground motion654

is related to the plate boundary.

655

Figure 17 shows the estimated contributions of each model parameter uncertainty to the demand

33
656

output for discussing the explainability of the constructed surrogate models. Here, the contributions

657

to the maximum displacement of the RC pier are shown in Case #2 and Case #3 in Fig. 17(a) and (b),658respectively. Notice that highly nonlinear behavior occurred in Case #2 (JMA KOBE) but did not659

occur in Case #3 (KAIHOKUBASHI), as explained in Fig. 13. The plot of red bars shows the660

contributions of the model parameters in the common part, the one with green bars denotes those in661

the source part, and the plot of blue bars denotes the contributions in the target part. It can be first662

understood that no parameter significantly contributed to the output in the common part in both cases.

663

This is considered the reason why the TL was not effective in these cases. The plot of the target part664

indicates how each model parameter is considered in each target surrogate model. It can be seen that665

the contributions of the parameters that are related to the nonlinear behavior, such as Kb2, Qb, and666

Qrc, are high in Case #2 (JRA KOBE), whereas the contribution of those parameters is much lower

667

in Case #3 (KAHOKUBASHI). These show the consistencies in the structural responses, i.e.,

668

occurrences of nonlinearity, which depend on the input earthquake waveform. Even though the effect669

of TL in computational cost reduction was not achieved by the TL-GPRSM in the cases considered

670

here, the validity of the constructed surrogate models could be assured by observing the estimated671

contributions. Further considerations of the surrogate modeling of nonlinear dynamic systems with672

effective TL are required in a future study.

673

However, the constructed surrogate models can provide appropriate distributions of the maximum

674

displacements for seismic evaluation. Figure 18 shows the cumulative distributions of the maximum

675

displacements of the RC pier and seismic isolation rubber bearing in Case #2. The surrogate model676

was constructed using TL-GPRSM with the number of target domain data as 50. Ten red lines denote677

the distributions from the ten surrogate models constructed by ten-time DoE samplings of the data.

678

The black line denotes the distributions derived using the 2DOF model MC calculations with 10,000

679

samples. Especially in the RC pier, the distribution from the surrogate model shows good agreement680

with those from the MC calculation along with high stability. In the bearing, the both-side tails of the

34
681

distributions from the surrogate models do not show high accuracy and stability.

682

683
684

(a) Accuracies of constructed surrogate models (left: pier, right: bearing)

685

686
687

(b) Contributions of common, source, and target domains (left: pier, right: bearing)

688

689
690
691
692
693

(c) Contribution of common part for each source domain (left: pier, right: bearing)
Fig. 14 Results of surrogate model construction in target domain of Type2-1-2 (Case #1)

35

694
695

(a) Accuracies of constructed surrogate models (left: pier, right: bearing)

696

697
698

(b) Contributions of common, source, and target domains (left: pier, right: bearing)

699

700
701
702
703
704
705

(c) Contribution of common part for each source domain (left: pier, right: bearing)
Fig. 15 Results of surrogate model construction in target domain of JMA KOBE (Case #2)

36

706
707

(a) Accuracies of constructed surrogate models (left: pier, right: bearing)

708

709
710

(b) Contributions of common, source, and target domains (left: pier, right: bearing)

711

712
713
714
715
716
717

(c) Contribution of common part for each source domain (left: pier, right: bearing)
Fig. 16 Results of surrogate model construction in target domain of KAIHOKUBASHI (Case #3)

37

718
719

(a) Case #2 JMA KOBE (number of target domain training data: 50)

720

721
722
723
724
725
726
727

(b) Case #3 KAIHOKUBASHI (number of target domain training data: 50)
Fig. 17 Estimated contributions of model parameter uncertainties to the maximum displacement of
the RC pier (Left: Common part, Center: Source part, Right: Target part)Cumulative probability

Cumulative probability

38

728
729
730

(a) RC pier(b) Rubber bearing

Fig. 18 Prediction of cumulative distribution of output values by TL-GPRSM in Case #2

731
732
733

5. Conclusions

734

The GPR surrogate model with TL (TL-GPRSM) was proposed in this paper. The computational cost735

for constructing a surrogate model of the target analysis was reduced by using the data of the input–

736

output relationships of the source analysis with any similarity to those of the target analysis. The use

737

of the ARD kernel in GPR was suggested to evaluate the effectiveness of TL and explainability of738

the constructed surrogate model. Two case studies were conducted to verify the significance of TL-

739

GPRSM. The conclusions are summarized as follows:

740- TL-GPRSM was applied to the surrogate modeling of the live-load performance evaluation of a741

steel plate girder bridge with corrosion damage by applying the source analysis of the undamaged742

condition. The prediction error was less than 1% RMSPE when 15 target domain data were used.

743

This is equivalent to the accuracy obtained with 25 target data for the surrogate model without TL,

744

regardless of the number of source data.

745- The predicted cumulative distribution of the maximum stress in the TL-GPRSM had less746

uncertainty and the shape was closer to that obtained with the numerical results than the surrogate

747

model without TL using the same number of target data.

748- The TL effect could be determined from the contribution of the common part calculated from the

39
749

ARD kernel. Transfer learning was determined to be more effective when the number of source data

750

with lower RMSPE was large. Furthermore, ARD made it possible to know the contribution of the751

individual parameters, and the contribution of parameters related to degradation damage was high.

752- The seismic performance evaluation of a seismic isolation bridge pier considering the variation in753

the input earthquake ground motions was the second application considered for verification. This

754

involved the surrogate modeling of nonlinear time-history analysis. It was shown that the755

effectiveness of TL was not as high as in the linear structural analysis in the first case; however, the756

TL-GPRSM was able to predict the distributions of the maximum displacements with slightly

757

higher accuracy than the surrogate model without TL in some input cases.

758- In nonlinear analysis, it was possible to determine the effectiveness of TL based on the magnitude759

of the contribution of the common part as estimated by the ARD kernel. The contribution of each760

parameter to the output estimated by the ARD kernel was reasonable from the viewpoint of

761

nonlinear structural dynamics.

762

A future topic of study is the further consideration of surrogate modeling of nonlinear dynamic

763

systems with effective TL. Further, the advanced DoE sampling that takes advantage of the estimated764

contributions in ARD is expected to reduce the computational cost of creating training data for765

effective TL. The selection of the kernel function in GPR with ARD is also worth considering, as the766

computational cost might be reduced by capturing the characteristics of the input–output relationship767of the target numerical analysis appropriately. However, the results of this paper showed that the TL-

768

GPRSM was effective in constructing the surrogate model of linear numerical calculations by

769

reducing the computational cost of the structural performance analysis with linear numerical770

calculation under variation in uncertainties, such as the analysis of the damaged condition based on

771

that of the undamaged initial condition. Moreover, it was shown that the effectiveness of TL in each772

surrogate modeling and the explainability of the constructed model could be discussed by deriving

773

the contributions of each parameter by using the ARD kernel. This is significant in ensuring the

40
774

acceptability of the constructed surrogate models in the structural performance evaluation for any

775

decision-making.

776
777

Funding

778

This study was supported by the JST FOREST Program, Japan [grant number JPMJFR205T].

779
780

References781[1] Bucher CG, Bourgund U. A fast and efficient response surface approach for structural reliability
problems. Struct Saf 1990;7:57–66. https://doi.org/10.1016/0167-4730(90)90012-E.

782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803

[2] Kim S-H, Na S-W. Response surface method using vector projected sampling points. Struct Saf
1997;19:3–19. https://doi.org/10.1016/S0167-4730(96)00037-9.
[3] Zhao W, Qiu Z. An efficient response surface method and its application to structural reliability
and reliability-based optimization. Finite Elem Anal Des 2013;67:34–42.
https://doi.org/10.1016/j.finel.2012.12.004.
[4] Rocco CM, Moreno JA. Fast Monte Carlo reliability evaluation using support vector machine.
Reliab Eng Syst Saf 2002;76:237–43. https://doi.org/10.1016/S0951-8320(02)00015-7.
[5] Roy A, Manna R, Chakraborty S. Support vector regression based metamodeling for structuralreliability analysis. Probab Eng Mech 2019;55:78–89.
https://doi.org/10.1016/j.probengmech.2018.11.001.
[6] Hawchar L, El Soueidy C-P, Schoefs F. Principal component analysis and polynomial chaos
expansion for time-variant reliability problems. Reliab Eng Syst Saf 2017;167:406–16.
https://doi.org/10.1016/j.ress.2017.06.024.
[7] Marelli S, Sudret B. An active-learning algorithm that combines sparse polynomial chaos
expansions and bootstrap for structural reliability analysis. Struct Saf 2018;75:67–74.
https://doi.org/10.1016/j.strusafe.2018.06.003.
[8] Le V, Caracoglia L. A neural network surrogate model for the performance assessment of avertical structure subjected to non-stationary, tornadic wind loads. Comput Struct
2020;231:106208. https://doi.org/10.1016/j.compstruc.2020.106208.
[9] Chojaczyk AA, Teixeira AP, Neves LC, Cardoso JB, Guedes Soares C. Review and applicationof Artificial Neural Networks models in reliability analysis of steel structures. Struct Saf

41
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836

2015;52:78–89. https://doi.org/10.1016/j.strusafe.2014.09.002.
[10] Deng J, Gu D, Li X, Yue ZQ. Structural reliability analysis for implicit performance functions
using artificial neural network. Struct Saf 2005;27:25–48.
https://doi.org/10.1016/j.strusafe.2004.03.004.
[11] Zhou T, Peng Y. Kernel principal component analysis-based Gaussian process regression
modelling for high-dimensional reliability analysis. Comput Struct 2020;241:106358.
https://doi.org/10.1016/j.compstruc.2020.106358.
[12] Su G, Peng L, Hu L. A Gaussian process-based dynamic surrogate model for complex
engineering structural reliability analysis. Struct Saf 2017;68:97–109.
https://doi.org/10.1016/j.strusafe.2017.06.003.
[13] Gaspar B, Teixeira AP, Soares CG. Assessment of the efficiency of Kriging surrogate models forstructural reliability analysis. Probab Eng Mech 2014;37:24–34.
https://doi.org/10.1016/j.probengmech.2014.03.011.
[14] Avendaño-Valencia LD, Abdallah I, Chatzi E. Virtual fatigue diagnostics of wake-affected windturbine via Gaussian Process Regression. Renewable Energy 2021;170:539–61.
https://doi.org/10.1016/j.renene.2021.02.003.
[15] Rasmussen CE. Gaussian Processes in Machine Learning. In: Bousquet O, von Luxburg U,
Rätsch G, editors. Advanced Lectures on Machine Learning: ML Summer Schools 2003,
Canberra, Australia, February 2 - 14, 2003, Tübingen, Germany, August 4 - 16, 2003, Revised
Lectures, Berlin, Heidelberg: Springer Berlin Heidelberg; 2004, p. 63–71.
https://doi.org/10.1007/978-3-540-28650-9_4.
[16] Bichon BJ, Eldred MS, Swiler LP, Mahadevan S, McFarland JM. Efficient Global Reliability
Analysis for Nonlinear Implicit Performance Functions. AIAA Journal 2008;46:2459–68.
https://doi.org/10.2514/1.34321.
[17] Echard B, Gayton N, Lemaire M. AK-MCS: An active learning reliability method combining
Kriging and Monte Carlo Simulation. Struct Saf 2011;33:145–54.
https://doi.org/10.1016/j.strusafe.2011.01.002.
[18] Gaspar B, Teixeira AP, Guedes Soares C. Adaptive surrogate model with active refinement
combining Kriging and a trust region method. Reliab Eng Syst Saf 2017;165:277–91.
https://doi.org/10.1016/j.ress.2017.03.035.
[19] Xiao N-C, Zuo MJ, Guo W. Efficient reliability analysis based on adaptive sequential sampling
design and cross-validation. Appl Math Model 2018;58:404–20.
https://doi.org/10.1016/j.apm.2018.02.012.

42
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866

[20] Zhou T, Marelli S, Sudret B, Peng Y. AK-PDEMi: A failure-informed enrichment algorithm for
improving the AK-PDEM in reliability analysis. Mech Syst Signal Process 2022.
[21] Echard B, Gayton N, Lemaire M, Relun N. A combined Importance Sampling and Kriging
reliability method for small failure probabilities with time-demanding numerical models. Reliab
Eng Syst Saf 2013;111:232–40. https://doi.org/10.1016/j.ress.2012.10.008.
[22] Huang X, Chen J, Zhu H. Assessing small failure probabilities by AK–SS: An active learning
method combining Kriging and Subset Simulation. Struct Saf 2016;59:86–95.
https://doi.org/10.1016/j.strusafe.2015.12.003.
[23] Moustapha M, Marelli S, Sudret B. Active learning for structural reliability: Survey, generalframework and benchmark. Struct Saf 2022;96:102174.
https://doi.org/10.1016/j.strusafe.2021.102174.
[24] Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. Journal of Big Data
2016;3:1–40. https://doi.org/10.1186/s40537-016-0043-6.
[25] Xiong Y, Guo L, Zhang Y, Xu M, Tian D, Li M. Surrogate modeling for spacecraft
thermophysical models using deep learning. Neural Comput Appl 2022.
https://doi.org/10.1007/s00521-022-07257-7.
[26] Kaya M, Hajimirza S. Using a Novel Transfer Learning Method for Designing Thin Film Solar
Cells with Enhanced Quantum Efficiencies. Sci Rep 2019;9:5034.
https://doi.org/10.1038/s41598-019-41316-9.
[27] Tian K, Li Z, Zhang J, Huang L, Wang B. Transfer learning based variable-fidelity surrogate
model for shell buckling prediction. Compos Struct 2021;273:114285.
https://doi.org/10.1016/j.compstruct.2021.114285.
[28] Golparvar B, Papadopoulos P, Ezzat AA, Wang R-Q. A surrogate-model-based approach forestimating the first and second-order moments of offshore wind power. Appl Energy
2021;299:117286. https://doi.org/10.1016/j.apenergy.2021.117286.
[29] Williams C, Rasmussen C. Gaussian processes for regression. Advances in Neural Information
Processing Systems 1995.
[30] Wipf D, Nagarajan S. A new view of automatic relevance determination. Advances in Neural
Information Processing Systems 2007;20.

867[31] Liu DC, Nocedal J. On the limited memory BFGS method for large scale optimization. Math
Program 1989;45:503–28. https://doi.org/10.1007/BF01589116.

868[32] GPy. GPy: A Gaussian process framework in python since 2012.

43
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885

http://github.com/SheffieldML/GPy.
[33] Daumé H III. Frustratingly Easy Domain Adaptation. ArXiv [CsLG] 2009.
https://doi.org/10.48550/arXiv.0907.1815.
[34] McKay MD, Beckman RJ, Conover WJ. Comparison of Three Methods for Selecting Values of
Input Variables in the Analysis of Output from a Computer Code. Technometrics 1979;21:239–
45. https://doi.org/10.1080/00401706.1979.10489755.
[35] Nishio M, Miura M, Shuku T. Sparse Surrogate Modeling for Structural Reliability Analysis of
Existing Bridges. Journal of Japan Society of Civil Engineers, Ser A2 (Applied Mechanics(AM)) 2018;74:I_125-I_136. https://doi.org/10.2208/jscejam.74.I_125. (in Japanese)[36] Japan Road Association. SPECIFICATION FOR HIGHWAY BRIDGES PART II STEELBRIDGES. Japan Road Association; 2012. (in Japanese)[37] Japan Road Association. Handout on Seismic Design of Road Bridges [Translated fromJapanese]. Japan Road Association; 1997. (in Japanese)[38] Takeda T, Sozen MA, Nielsen NN. Reinforced Concrete Response to Simulated Earthquakes.
Journal of the Structural Division 1970;96:2557–73. https://doi.org/10.1061/JSDEAG.0002765.
[39] Japan Road Association. SPECIFICATIONS FOR HIGHWAY BRIDGES Part V SEISMICDESIGN. Japan Road Association; 2012. (in Japanese)