# Inverse Reliability Task: Artificial Neural Networks and Reliability-Based Optimization Approaches

David Lehký, Ondřej Slowik, Drahomír Novák

► To cite this version:

David Lehký, Ondřej Slowik, Drahomír Novák. Inverse Reliability Task: Artificial Neural Networks and Reliability-Based Optimization Approaches. 10th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Sep 2014, Rhodes, Greece. pp.344-353, 10.1007/978-3-662-44654-6_34. hal-01391333

HAL Id: hal-01391333

https://inria.hal.science/hal-01391333

Submitted on 3 Nov 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Inverse Reliability Task: Artificial Neural Networks and Reliability-based Optimization Approaches

David Lehký, Ondřej Slowik, Drahomír Novák

Brno University of Technology, Brno, Czech Republic
{lehky.d,slowik.o,novak.d}@fce.vutbr.cz

**Abstract.** The paper presents two alternative approaches to solve inverse reliability task – to determine the design parameters to achieve desired target reliabilities. The first approach is based on utilization of artificial neural networks and small-sample simulation Latin hypercube sampling. The second approach considers inverse reliability task as reliability-based optimization task using double-loop method and also small-sample simulation. Efficiency of both approaches is presented in numerical example, advantages and disadvantages are discussed.

**Keywords:** Inverse reliability, artificial neural network, reliability-based optimization, double-loop optimization, uncertainties, Latin hypercube sampling.

## 1 Introduction

To achieve desired level of reliability in limit state design is generally not an easy task. Uncertainties are involved in every part of structural system (e.g. material properties, geometrical imperfections, dead load, live load, wind, snow, corrosion rate, etc.). When performing either reliability assessment or advanced engineering design, it is certainly essential to take uncertainties into account using a probabilistic analysis. Reliability assessment requires forward reliability methods for estimating the reliability (usually theoretical failure probability and/or reliability index are determined). On the other hand, the engineering design requires an inverse reliability approach to determine the design parameters to achieve desired target reliabilities.

Some sophisticated approaches to determine design parameters (material properties, geometry, etc.) related to particular limit states have been proposed under the name “inverse reliability methods”, e.g. a reliability contour method [1] and [2], iterative algorithm based on the modified Hasofer-Lind-Rackwitz-Fiessler scheme used in reliability analysis [3], Newton-Raphson iterative algorithm to find multiple design parameters [4] and [5], decomposition technique [6] or various implementation of artificial neural network (ANN) with other soft-computing techniques [7], [8] and [9].

The two methods proposed in this paper attempts to overcome the shortcomings of existing inverse reliability methods. The first one utilizes ANN too, but in a different way: Computational time is reduced by using a small-sample simulation technique
called Latin hypercube sampling (LHS) in ANN based inverse problem proposed by Novák and Lehký in [10] and [11] first.

The second one is double-loop reliability based optimization (RBO) approach. Classical optimization usually leads to solutions that lie at the boundary of the admissible domain, and that are consequently rather sensitive to uncertainty in the design parameters. In contrast, RBO aims at designing the system in a robust way by minimizing some objective function under reliability constraints. It provides the means for determining the optimal solution of a certain objective function, while ensuring a predefined small probability that a structure fails. Thus RBO methods have to mix optimization algorithms together with reliability calculations. The approach known as “double-loop” consists in nesting the computation of the failure probability with respect to the current design within the optimization loop (e.g. [12]). FORM-based double-loop approach has been proposed by Dubourg in [13, 14]. The authors developed a double-loop reliability-based optimization approach based on small-sample simulation and FORM [15, 16].

## 2 Inverse Reliability Task

The aim of classical (forward) reliability analysis is the estimation of unreliability using a probability measure called the theoretical failure probability, defined as:

$$p_f = P(Z \le 0), \qquad (1)$$

where Z is a function of basic random variables X = X₁, X₂, ..., Xₙ called safety margin. This failure probability is calculated as a probabilistic integral:

$$p_f = \int_{D_f} f_X(\mathbf{X}) d\mathbf{X} \qquad (2)$$

where the domain of integration of the joint probability distribution function (PDF) above is limited to the failure domain D_f where g(X) ≤ 0. The function g(X), a computational model, is a function of random vector X (and also of other, deterministic quantities). Random vector X follows a joint PDF f_X(X) and, in general, its marginal variables can be statistically correlated. The explicit calculation of integral in (2) is generally impossible. Therefore a large number of efficient stochastic analysis methods have been developed during the last seven decades.

The inverse reliability task is the task to find design parameters corresponding to specified reliability levels expressed by reliability index or by theoretical failure probability. In general, an inverse problem involves finding either a single design parameter to achieve a given single reliability constraint or multiple design parameters to meet specified multiple reliability constraints. The design parameters can be deterministic or they can be associated with random variables described by statistical mo-
ments (mean value, standard deviation) and PDF. In case of mean value one need to
choose if either standard deviation or coefficient of variation will be fixed.

## 2.1 Solution Based on Artificial Neural Networks

An efficient general approach of inverse reliability analysis is proposed to obtain
design parameters of a computational model in order to achieve the prescribed relia-
bility level. The inverse analysis is based on the coupling of a stochastic simulation of
Monte Carlo type and an ANN. The design parameters (e.g. mean values or standard
deviations of basic random variables) play the role of basic random variables with a
scatter reflecting the physical range of design values. A novelty of the approach is the
utilization of the efficient small-sample simulation method LHS used for the stochas-
tic preparation of the training set utilized in training the ANN. The calculation of
reliability is performed using the first order reliability method (FORM). Once the
ANN has been trained, it represents an approximation consequently utilized in an
opposite way: To provide the best possible set of design parameters corresponding to
prescribed reliability.

The procedure of ANN based inverse reliability method is illustrated by a simple
flow chart as shown in Figure 1 and is implemented as follows:

1. The design parameters are considered as random variables with selected (physical-ly reasonable) appropriate scatter and probability distribution. Rectangular distri-bution is often used.

2. Random samples of design parameters (possibly correlated) are generated using LHS simulation method.

3. Stochastic model of analyzed problem is prepared including generated samples of design parameters.

4. Reliability analyses are performed repeatedly for individual samples of design pa-rameters and set of reliability measures like failure probabilities or reliability indi-ces are calculated.

5. Reliability measures obtained from simulations together with set of random design parameters serve as training set for ANN training. During training an error between simulated and desired outputs of ANN (here in form of MSE) is minimized using appropriate optimization technique (e.g. back propagation methods, evolutionary algorithms).

6. Desired reliability measures are used as an input signal which is distributed through ANN structure to its output where optimal design parameters are obtained.

7. Verification of the results by calculation of failure probabilities related to limit state functions using the optimal parameters is carried out. A comparison with tar-get failure probabilities will show the extent to which the inverse analysis was suc-cessful.

In the case of inverse reliability analysis a double stochastic analysis is needed for
the training set preparation for ANN (steps 2 and 4 of the procedure). In the outer
loop random realizations of design parameters are generated using the LHS simula-tion technique. The inner loop represents the reliability calculation for one particular
realization of design parameters. Here, the FORM approximation method is recom-
mended due to computational demands. The number of simulations in outer loop is
driven by ANN and only tens of simulations are usually needed.

<figure><img src="image_2.png" /><figcaption>Fig. 1. A flow chart of proposed inverse reliability method.</figcaption></figure>

2.2 Solution by Small-sample Double-loop Reliability-based Optimization

Typically, reliability-based optimization is formulated as:

$$
\begin{equation}
\begin{array}{ll}
\text{find } \mathbf{x} & \\
\min f(\mathbf{x}) & (3) \\
\text{subject to: } & P_f[g(\mathbf{x},\mathbf{z}) \le 0] \le P_0, \quad \mathbf{l} \le \mathbf{x} \le \mathbf{u} \\
\end{array}
\end{equation}
$$

with $P_f$ the probability of constraint satisfaction. The limit state $g = 0$ separates the region of failure ($g \le 0$) and safe region ($g > 0$) and is a function of the design variables $\mathbf{x}$ (and $\mathbf{l}$ and $\mathbf{u}$ are lower and upper bounds) and the uncertain variables $\mathbf{z}$. $P_0$ is the reliability level or performance requirement. The above inequality can be expressed by a failure probability multi-dimensional integral with the joint probability density function of probabilistic variables $\mathbf{z}$. Formulation based on reliability index instead of failure probability is popular especially in the context of FORM approximation.
From the theoretical point of view, RBO has been a well-established concept. However, computing failure probabilities appears as a technically involved problem analytically tractable for very simple cases only. This is because it is often a multi-dimensional integral equation for which the joint probability density function and/or limit state function *g* is unknown in explicit form, like FEM computational model. The same difficulty is with objective function *f(x)* – it can be computationally demanding FEM analysis and the use of classical optimization technique is problematic or even impossible. Then an application of RBO for real-world problem is difficult.

Computational demands of reliability-based optimization are obvious from the formulation above. For the purposes of stochastic optimization it is necessary to repeatedly generate random realizations within the design space. It is also necessary for each of these realizations to calculate the probability of failure in the general case by computationally demanding (mostly numerical) integration of the equation (2). Therefore we suggest here an original small-sample double-loop RBO methodology where lower computational burden exists in case of both outer loop – minimization of objective function and inner loop – calculation of failure probability (or reliability index). A practical solution to the above-defined optimization problem is performed using the so-called double-loop approach. The algorithm is composed of two basic loops:

*   **The outer loop** represents the optimization part of the process based on small-sample simulation Latin hypercube sampling. The simulation within the design space is performed in this cycle. For obtained design vectors of *n*-dimensional space $x_i=(x_1, x_2, ..., x_n)$ objective function values are calculated. The best realization is then selected based on these values and utilized optimization method. Consequently the best realization of random vector $x_{i,best}$ is compared with optimization constraints. These constraints may be formulated by any deterministic function which functional value can be compared with a defined interval of allowed values. Constraints are also possible to formulate as allowed interval of reliability index $\beta$ for any limit state function (within design space of given problem). Calculations of reliability index of each generated random vectors $x_i$ takes place in the inner loop. Note that it is necessary to use some of advanced meta-heuristic optimization techniques (e.g. simulated annealing or genetic algorithms) to avoid local minima.

*   **The inner loop** is used to calculate reliability index (FORM-based) either for the need of checking of generated solutions – if they satisfy constraints, or to calculate the actual value of the objective function, if the target reliability index is set as goal of optimization process.

## 3 Numerical Example

Selected application originates from the civil engineering field of structural mechanics. The aim is to design the dimensions of a rectangular cross-section with width *b* and height *h* of a simply-supported beam made of timber (Figure 2). Both dimensions are considered as random variables with a variation of 5 %. The mean values of *b* and *h* are design parameters in the inverse reliability problem.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Scheme of a simply-supported beam with a rectangular cross-section.</figcaption></figure>

The design is performed fully according to Eurocode 5. The ultimate limit state (ULS) as well as the serviceability limit state (SLS) is taken into account. Target reliability indices are $\beta_1 = 3.8$, $\beta_2 = 1.5$. The limit states are described by the following limit state functions $g_1$ and $g_2$:

$$ 
\begin{aligned}
g_1 &= M_R - M_E \\
g_2 &= u_{\lim, fin} - u_{net, fin}
\end{aligned}
\qquad (4) $$

where $M_R$ is the bending moment of resistance, $M_E$ is the bending moment of load action, $u_{\lim, fin}$ is the final limit deflection and $u_{net, fin}$ is the final deflection caused by load action. Bending moments $M_R$ and $M_E$ are calculated as:

$$ 
\begin{aligned}
M_R &= \theta_R \frac{1}{6}bh^2k_{\text{mod}}f_m \\
M_E &= \theta_E \frac{1}{8}(g+q)l^2
\end{aligned}
\qquad (5) $$

where $b$ and $h$ are the width and height of rectangular cross-section, $l$ is the length of the beam, $f_m$ is flexural strength, $k_{\text{mod}}$ is the modification factor taking into account the effect on the strength parameters of the duration of the load and the moisture content in the structure (value $k_{\text{mod}} = 0.8$ was considered), $g$ is permanent load, $q$ is variable load and $\theta_R$ and $\theta_E$ are the model uncertainties of resistance and load action. Deflections in the second limit state function $g_2$ are calculated as:

$$ 
\begin{aligned}
u_{\lim, fin} &= \frac{l}{200} \\
u_{net, fin} &= \theta_E (u_{1, fin} + u_{2, fin}) \\
u_{1, fin} &= \frac{5}{384} \frac{gl^4}{E \frac{1}{12} bh^3} (1 + k_{1, def}) \\
u_{2, fin} &= \frac{5}{384} \frac{ql^4}{E \frac{1}{12} bh^3} (1 + k_{2, def})
\end{aligned}
\qquad (6) $$

where $u_{1, fin}$ and $u_{2, fin}$ are the final deflections caused by the permanent load and variable load, $E$ is the modulus of elasticity of timber, $k_{1, def}$ is a factor which takes into
account the increase in deflection with time due to the combined effect of creep and moisture and it belongs to permanent load and $k_{2,def}$ is the same factor but for variable load (values of $k_{1,def}$ = 0.8 and $k_{2,def}$ = 0.25 were used). Table 1 summarizes all random variables and their randomization. The values of the material parameters correspond to spruce timber. Randomization was carried out according to the recommendations of JCSS probabilistic model code [17]. Reliability analysis was carried out using the FORM method; the starting values were means; the tolerance for convergence was $10^{-4}$. For both a training set preparation and purpose of optimization the design parameters were considered as random variables with rectangular distribution, see Table 2.

<table><caption>**Table 1. Random variables and design parameters**</caption><thead><tr><th>Variable</th><th>Distribution</th><th>Mean</th><th>Std</th><th>COV</th></tr></thead><tbody><tr><td>l [m]</td><td>Normal</td><td>3.5</td><td>0.175</td><td>0.05</td></tr><tr><td>b [m]</td><td>Normal</td><td>?</td><td>--</td><td>0.05</td></tr><tr><td>h [m]</td><td>Normal</td><td>?</td><td>--</td><td>0.05</td></tr><tr><td>E [GPa]</td><td>Lognormal (2 par)</td><td>10</td><td>1.3</td><td>0.13</td></tr><tr><td>f<sub>m</sub> [MPa]</td><td>Lognormal (2 par)</td><td>34</td><td>8.5</td><td>0.25</td></tr><tr><td>g [kN/m]</td><td>Gumbel max EV 1</td><td>1.686</td><td>0.169</td><td>0.10</td></tr><tr><td>q [kN/m]</td><td>Gumbel max EV 1</td><td>2.565</td><td>0.770</td><td>0.30</td></tr><tr><td>θ<sub>R</sub> [-]</td><td>Lognormal (2 par)</td><td>1</td><td>0.1</td><td>0.10</td></tr><tr><td>θ<sub>R</sub> [-]</td><td>Lognormal (2 par)</td><td>1</td><td>0.1</td><td>0.10</td></tr></tbody></table>

<table><caption>**Table 2. Randomization of design parameters for training set preparation and purpose of optimization**</caption><thead><tr><th>Variable</th><th>Distribution</th><th>Mean</th><th>Std</th><th>a</th><th>b</th></tr></thead><tbody><tr><td>mean(b)</td><td>Rectangular</td><td>0.125</td><td>0.0144</td><td>0.10</td><td>0.15</td></tr><tr><td>mean(h)</td><td>Rectangular</td><td>0.225</td><td>0.0144</td><td>0.20</td><td>0.25</td></tr></tbody></table>

## 3.1 ANN Inverse Analysis

First, ANN inverse reliability analysis was carried out. The ANN (see Figure 3) consisted of one hidden layer having four nonlinear neurons (hyperbolic tangent transfer function) and an output layer having two output neurons (linear transfer function) which correspond to two design parameters – the mean values of width *b* and height *h*. The ANN has two inputs which correspond to two specified reliability indices, $\beta_1$ and $\beta_2$. For preparation of training set one hundred random samples of design parameters were generated using LHS method according to stochastic model in Table 2 and stochastic analyses were carried out to obtain corresponding reliability indices.
The resulting design parameter values are given in Table 3. To check their accuracy these values were used in equations (4) to (6) and reliability indices were calculated; see the comparison with the target reliability indices in Table 3. In the case of practical design the dimensions of cross-section should be selected from available set of dimensions. In our example, the resulting width and height would be $b = 140$ mm and $h = 220$ mm which gives the final reliability indices $\beta_{1,fin} = 4.068$ and $\beta_{2,fin} = 1.912$.

<figure><img src="image_4.png" /><figcaption>Fig. 3. A scheme of artificial neural network</figcaption></figure>

## 3.2 Double-Loop Optimization Approach

If we define optimization problem according to definition at section 2.2 then inverse reliability problem can be also solved using double-loop reliability-based optimization approach. During the solution of the problem an option to determine a target value of reliability index for the selected limit state function was utilized. Therefore target reliability index for the limit state function $g_2$ was defined as $\beta_2 = 1.5$. As a constraint an interval (3.75, 3.85) of allowable values of reliability index $\beta_1$ for the limit state function $g_1$ was set.

During solution of the task using aimed multilevel sampling (AMS) optimization algorithm [15] the total number of 300 simulations was used. The result solution of the task is displayed in Table 3 and in Figure 4. Final solution corresponds well to the values obtained using ANN inverse analysis. The resulting cross-sectional area is $2.8302 \times 10^{-2} \text{ m}^2$ compared to $2.8385 \times 10^{-2} \text{ m}^2$ obtained from ANN inverse analysis. The graph in Figure 4 shows the gradual convergence of generated solutions toward the required values of reliability indices.

<table><caption>Table 3. Resulting values of design parameters and reliability indices $\beta$ obtained by double-loop optimization approach</caption><thead><tr><th>Approach</th><th>mean(b)</th><th>mean(h)</th><th>β<sub>1</sub></th><th>β<sub>2</sub></th><th>β<sub>1 target</sub></th><th>β<sub>2 target</sub></th></tr></thead><tbody><tr><td>ANN inverse analysis</td><td>0.13244</td><td>0.21432</td><td>3.8001</td><td>1.5001</td><td></td><td></td></tr><tr><td>Double-loop optimization</td><td>0.1311555</td><td>0.215135</td><td>3.793</td><td>1.50009</td><td>3.8</td><td>1.5</td></tr></tbody></table>
<figure><img src="image_5.png" /><figcaption>Fig. 4. Evolution of values of reliability indices during optimization</figcaption></figure>

## 4 Conclusion

The paper presents two alternative approaches to solve inverse analysis task. Both approaches provide very good results, as is indicated in numerical example. Some advantages and disadvantages of the methods can be highlighted:

ANN inverse analysis will be probably more accurate and capable to solve also highly nonlinear problems. But more simulations are generally needed for good training of ANN (in case of very small numbers – training cannot be simply done at all). The step “training of ANN” requires deeper involvement of a user, which makes the usage of the approach difficult.

On the other hand the double-loop reliability-based optimization approach can solve problem satisfactorily using small number of simulations, but the lower accuracy can be expected. In case of highly nonlinear problems less efficiency can be expected comparing to ANN inverse analysis approach. The advantage of double-loop optimization approach is a transparency of solution and better understanding by general engineering practice.

The above mentioned summary is formulated based on testing approaches using limited number of numerical examples. The more systematic verification and testing are needed. Presents results indicate that both approaches have very good potential to solve inverse reliability task using small-sample simulation.

## Acknowledgements

The authors give thanks for the support of the research project No. TA01011019 of Technology Agency of the Czech Republic (TACR) and the European Union's "Operational Programme Research and Development for Innovations", No. CZ.1.05/2.1.00/03.0097, as an activity of the regional Centre AdMaS "Advanced Materials, Structures and Technologies". In this undertaking, theoretical results
gained in the project of the specific university research at Brno University of Tech-
nology, registered under the number FAST-J-14-2425 were exploited.

References

1. Winterstein S.R., Ude T.C. & Cornell C.A.: Environ-mental Parameters for Extreme Re-sponse: Inverse Form with Omission Factors. In Schueller, Shinozuka and Yao (eds), Pro-ceedings of ICOSSAR'93: Structural Safety and reliability, Innsbruck, Austria, 551-557, Rotterdam: Balkema, (1994)

2. Maes, M.A. & Huyse, L.: Developing Structural Design Criteria with Specifies Response Reliability. Canadian Journal of Civil Engineering, 24(2): 201-210, (1997)

3. Der Kiureghian A., Zhang Y. & Li C.C.: Inverse Reliability Problem. Journal of Engineering mechanics, 120(5): 1154-1159, (1994)

4. Li, H. & Foschi, R.O.: An Inverse Reliability Method and its Application. Structural Safety, 20: 257-270, (1998)

5. Sadovský, Z.: Discussion on: An Inverse Reliability Method and its Application. Structural Safety, 22(1): 97-102, (2000)

6. Mínguez, R., Castillo, E. & Hadi, A.S.: Solving the Inverse Reliability Problem Using Decomposition Techniques. Structural Safety, 27: 1-23, (2005)

7. Shayanfar, M.A., Massah, S.R. & Rahami, H.: An Inverse Reliability Method Using Networks and Genetic Algorithms. World Applied Sciences Journal 2(6): 594-601, (2007)

8. Cheng, J. & Li, Q.S.: Application of the Response Surface Methods to Solve Inverse Reliability Problems with Implicit Response functions. Computational Mechanics, 43(4): 451-459, (2009)

9. António, C. & Hoffbauer, L.: Uncertainty Propagation in Inverse Reliability-based Design of Composite Structures. International Journal of Mechanics and Materials in Design, 6(1): 89-102, (2010)

10. Novák, D. & Lehký, D.: ANN Inverse Analysis Based on Stochastic Small-Sample Training Set Simulation. J. of Eng. Application of Artificial Intelligence, 19: 731-740, (2006)

11. Lehký, D., & Novák, D.: Solving Inverse Structural Reliability Problem Using Artificial Neural Networks and Small-Sample Simulation. Advances in Structural Engineering, 15(11): 1911-1920 (2012)

12. Tsompanakis, Y., N. Lagaros, & M. Papadrakis (Eds.) (2008). Structural Design Optimization Considering Uncertainties. Taylor & Francis.

13. Dubourg, V., Noirfalise, C., Bourinet, J.-M.: Reliability-Based Design Optimization: An Application to the Buckling of Imperfect Shells. In: 4th ASRANet colloquium, Athens, Greece, (2008)

14. Dubourg, V., Bourinet, J.-M., Sudret, B.: A Hierarchical Surrogate-based Strategy for Reliability-based Design Optimization. In: D. Straub, L. Esteva, and M. Faber (Eds.), Proc. 15th IFIP WG7.5 Conference on Reliability and Optimization of Structural Systems, Munich, Germany, 53-60. Taylor & Francis, (2010)

15. Slowik, O.: Reliability-based Structural Optimization. Master's thesis. Brno University of Technology. Supervisor: prof. Ing. Drahomír Novák, DrSc, Brno, Czech Republic, (2014)

16. Slowik, O. & Novák, D.: Algorithmization of Reliability-based Optimization. Transactions of the VŠB - Technical University of Ostrava, Civil Engineering Series, ISSN: 1804-4824, (2014)

17. JCSS.: JCSS Probabilistic Model Code, Joint Committee on Structural Safety, Technical University of Denmark, Kongens Lyngby, Denmark, (2007)