# Shake-table testing of a brick masonry groin vault: overview of blind predictions and post dictions and comparison with experimental results

C. Calderini, N. Bianchini, P.B. Lourenço, N. Mendes, P.X. Candeias, O.

Alshawa, C. Chácara, M. M. Chávez, G. de Felice, A. Ferrante, et al.

► To cite this version:

C. Calderini, N. Bianchini, P.B. Lourenço, N. Mendes, P.X. Candeias, et al. Shake-table testing of a brick masonry groin vault: overview of blind predictions and post dictions and comparison with experimental results. International Journal of Architectural Heritage, 2024, Seismic Response of Masonry Cross Vaults: Shaking table testing and numerical validation, 18 (12), pp.1825-1851. 10.1080/15583058.2024.2419545. hal-05012438

HAL Id: hal-05012438

https://cnrs.hal.science/hal-05012438v1

Submitted on 31 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
1 Journal: International Journal of Architectural Heritage - Special Issue

2 Manuscript #:

3 Title of Paper: Shake-table testing of a brick masonry groin vault: overview of blind

4 pre-dictions and post-dictions and comparison with experimental

5 results.

6 Authors: C. Calderini¹*, N. Bianchini², P.B. Lourenço², N. Mendes², P.X.

7 Candeias³, O. AlShawa⁴, C. Chácara⁵, M. M. Chávez⁶, G. de Felice⁷, A.

8 Ferrante⁸, P. Fici⁹, R. Gagliardo¹⁰, P. Kesavan¹¹, G.P. Lignola¹⁰, M. Malena⁷, D.

9 Malomo¹², D. Pellegrini¹³, F. Peña Mondragón⁶, M. Petracca¹⁴, B. Pintucchi¹⁵,

10 R. F. Portioli¹⁰, B. Pulatsu¹⁶, G. Ramaglia¹⁰, D. Rapicavoli¹⁷, F. Trovatelli¹⁵

11 *corresponding author: chiara.calderini@unige.it

12

13 Affiliations:

14 ¹Department of Civil, Chemical and Environmental Engineering (DICCA), University of

15 Genoa Via Montallegro 1, 16145 Genoa, Italy

16 ²Department of Civil Engineering, ISISE, ARISE, University of Minho, Guimarães, Portugal

17 ³National Laboratory for Civil Engineering (LNEC), Earthquake Engineering and Structural

18 Dynamics Unit (NESDE), Department of Structures, Structures Department, Avenida do

19 Brasil, 101, 1700-066, Lisbon, Portugal

20 ⁴University of Rome “La Sapienza”

21 ⁵Pontificia Universidad Católica del Perú

22 ⁶Universidad Nacional Autónoma de México

23 ⁷University of Rome “Roma Tre”

24 ⁸LMGC, CNRS, Univ. Montpellier, Montpellier, France

25 ⁹Arup

26 ¹⁰University of Naples “Federico II”
27 $^{11}$IIT Madras

28 $^{12}$McGill University

29 $^{13}$CNR Pisa

30 $^{14}$ASDEAsoft, Italy

31 $^{15}$University of Florence

32 $^{16}$Carleton University

33 $^{17}$University of Catania
# Shake-table testing of a brick masonry groin vault: overview of blind predictions and post dictions and comparison with experimental results.

## Abstract.

Masonry cross vaults are often victim of severe damage and collapses when dynamic and horizontal loads act on them. To advance the state of the art on the seismic response of these pleasant and valued architectural element, the European project from The Seismology and Earthquake Engineering Research Infrastructure Alliance for Europe (SERA) provides shake-table test data of an unstrengthened full-scale cross vault with mortar joints and solid bricks, (and then strengthened with textile reinforced mortar), subjected to a monotonic horizontal component of dynamic excitation, recorded during the L'Aquila earthquake in 2009 (Italy). To test modelling approaches and assumptions and to learn about the extent of uncertainty in modelling for such masonry elements, a blind prediction was organised with participants from academia and industry.

The participants were provided with the full set of material and geometrical data, construction details and original seismic input and asked to predict the expected seismic response in terms of damage mechanisms, maximum absolute displacements, and accelerations, as well as the peak ground acceleration before the numerical collapse. Participants employ modelling approaches that vary significantly in terms of detail and underlying assumptions. This paper offers an overview of the modelling approaches utilised, along with their corresponding predictions and post dictions, following the provision of modal parameters to the participants. Additionally, the paper delves into the spectrum of assumptions inherent in modelling the masonry shell, boundary conditions, and connections. Its objective is to explore how common queries concerning the modelling of masonry in general, and masonry vaults specifically, impact the outcomes. The findings are assessed based on both the damage mechanisms observed and the predicted values for displacements and accelerations in both directions. These results are then compared with the experimental findings. Modelling approaches utilizing the Finite Element Method (FEM) yielded the most accurate predictions regarding displacements and crack patterns. Conversely, a submission employing a discrete element model provided the most accurate prediction of damage mechanisms. Nonetheless, the significant discrepancies in predicted displacements, along with the general underestimation compared to experimental results, underscore the necessity for continued efforts to establish consensus on appropriate modelling assumptions for masonry vaults.

Keywords: masonry vaults, shaking table test, seismic retrofitting, masonry models, blind prediction-postdiction.
67 **1 Introduction**

68
This paper presents the results of the blind test competition carried out within the scope of the
69
European project SERA.ta “Seismic Response of Masonry Cross Vaults: Shaking table tests
70
and numerical validations”. In the competition, the participants were invited to predict the
71
response of a full-scale mortared masonry cross vault tested on the shaking table of the
72
Laboratório Nacional de Engenharia Civil in Lisbon. Several experts were invited to predict the
73
in-plane seismic capacity prior to the disclosure of the test results, in what is commonly known
74
as a *blind prediction*. The primary objective of the exercise is to stimulate discussion regarding
75
the challenges associated with current methods for assessing masonry vaults, ultimately paving
76
the way for identifying research gaps and limitations.

77
The specimen, made of regular solid clay bricks radially arranged, represented the vault of a
78
central bay of a lateral nave of a generic three-nave church (Figure 1). It had asymmetric
79
boundary conditions: on the external side it rested on a stiff lateral wall, while on the interior
80
side, it rested on a colonnade, more flexible under seismic actions (Figure 2). This particular
81
boundary condition configuration likely causes in-plane shear crack on the shell of the vault,
82
which are commonly observed in post-earthquake surveys.

<figure><img src="image_2.png" /><figcaption>83 Figure 1. Virtual location of the specimen in a real church.</figcaption></figure>
A unidirectional seismic action along the longitudinal axis of the bay, for an increasing intensity
earthquake testing protocol based on a pre-processed strong ground motion component of the
L'Aquila (Italy) April 6, 2009 earthquake (Dipartimento della Protezione Civile 2022) that was
applied at the base of the shaking table. Then the specimen was properly repaired and
strengthened with textile reinforced mortar (TRM) solution and tested again. A detailed
description of the setup, material characterisation tests and conclusions on the displacements,
damage and collapse mechanisms can be found in Bianchini *et al.* (2024).

## 2 Brief description of the experimental campaign

The geometry of the tested specimen is presented in Figure 2.

<figure><img src="image_3.png" /><figcaption>Figure 2. Geometrical model of the full-scale specimen. (Dimensions in m).</figcaption></figure>

The main dimensions were namely the square plan equal to 3.5 × 3.5 m², the net span
was 2.9 m, the rise was 1.20 m and constant thickness was 0.12 m. The total height of the
specimen was equal to 1.80 m and the lengths of the three couples of steel cables with Φ32 mm
diameter (red lines in Figure 2) were equal to 2.40 m along the eastern edge and equal to 2.45
m along the North and South edges. Steel cables' purpose was to avoid torsional effects of the
piers and ensure the in-plane shear mechanism during the dynamic action. Two movable piers,
located along the eastern side of the vault, were left free to slide. The other two masonry piers
on the western side were fixed to a reinforced concrete slab placed on the shaking table. The
weight of the roof and the weight of the columns, which hosted the supports of the vault, were also considered through equivalent masses: four steel masses, with dimensions 0.84m × 0.84m × 0.27 m and 1.2 tons each, were placed below the movable supports. An overlying infill was built in correspondence with the vault's four corners up to 50% of the rise of the shell of the vault (Gaetani, 2016). Steel elements were included along the height of the supports and along the boundaries of the infill to avoid local failures at the corners. Once the first set of shaking table tests were performed on the unstrengthened configuration (UNS), the specimen was then repaired from the previous damage with grout injections and then strengthened (SM) with textile reinforced mortar (TRM). The seismic tests on the UNS specimen were performed until it attained a level of damage, which did not compromise its structural equilibrium so that it was still possible to be repaired.

Before and after each sequence of shaking table tests, Dynamic Identification Tests (DIT) were conducted to assess the reduction of frequencies and the increase of damage as the seismic tests progressed. Beginning at 25% of the input and incrementing by 25%, the seismic test on the UNS specimen reached a maximum level of 75% of the target input. Conversely, for the SM specimen, the test achieved a level of 150% of the target amplitude.

## 3 Blind pre-diction of the experimental results

The competition was launched in February 2021, after the completion of the vault construction in December 2020. The experimental campaign was carried out between September and November 2021: in particular, the UNS specimen was tested on September 21st while the SM specimen on November 10th. The deadline for the submission of the blind prediction was at the end of July 2022. The results of the competition were presented on December 2022. At the same time, the post-diction competition was launched, being completed on May 2023.
In total, 13 research groups, comprising nearly 50 researchers from 17 countries across 5 continents, participated in the competition. Notably, while the majority of researchers were affiliated with universities (35), some were associated with research institutes, such as the Italian CNR or the French CNRS (3), or private companies like Hopper Engineering and Consultancy and ARUP (6).

## 3.1 Objectives and data shared with the participants

The objectives of the blind prediction were:

* **Objective 1:** prediction of the seismic response before the repair of the unstrengthened specimen and the type of damage at the given level of input, considering, if applicable, its accumulation.

* **Objective 2:** prediction of the seismic response of the strengthened specimen and the type of damage at the given level of input, considering, if applicable, its accumulation.

Prediction of the seismic amplitude (PGA) that causes severe damage/near collapse in the strengthened specimen.

To launch the call and share all the data needed to participate to the competition, a website was created: https://sera-crossvault.wixsite.com/blindprediction and also ZENODO links. The following data were shared with the participants in the blind prediction phase:

* *Geometry, masses distributions and boundary conditions.* Very detailed 2D drawings of the specimen were provided. Moreover, a 3D Autocad model has been made available.

To allow the participants to correctly define the masses and the boundary conditions of the model, a further document was shared, containing specific information on the different components of the model.
149 • *Material properties.* The results of tests on masonry components (bricks, mortar, and brick-mortar interface) and on masonry wallets were provided (Bianchini, Calderini, et al. 2023a). In particular, for each set of tests and each parameter, the mean value (μ), the standard deviation (σ) and the co-variance (CoV) of the results were given to the participants. Since during the competition some participants asked for the full force-displacement curves of the tests on masonry wallets, they were shared with all the participants in a data sheet. A Poisson's ratio of between 0.2-0.25 was suggested. All relevant and given data regarding material properties, and valid for the UNS and SM models, are retrieved from Table 1 to Table 5.

<table><caption>Table 1. Properties of the mortar from tripoint bending and compressive tests results (standard EN 1015-11: 2019).</caption><thead><tr><th>Material</th><th>Dimensions [mm]</th><th>Density [kg/m3]</th><th>Elastic modulus* [MPa]</th><th>Flex. strength* [MPa]</th><th>Compress. strength* [MPa]</th></tr></thead><tbody><tr><td rowspan="3">Hardened mortar</td><td rowspan="3">160x40x40 &amp; 80x40x40</td><td><i>m</i></td><td>1746.84</td><td>370.05</td><td>0.66</td><td>1.68</td></tr><tr><td><i>s</i></td><td>12.60</td><td>55.72</td><td>0.11</td><td>0.17</td></tr><tr><td><i>CoV</i></td><td>0.72%</td><td>15.64%</td><td>16.46%</td><td>10.14%</td></tr><tr><td colspan="2">N° of samples</td><td>20</td><td>9</td><td>59</td><td>110</td></tr></tbody></table>

<table><caption>Table 2. Properties of the bricks from compressive tests results (standard EN 772-1).</caption><thead><tr><th>Material</th><th>Dimensions [mm]</th><th>Density [kg/m3]</th><th>Elastic modulus [MPa]</th><th>Compress. strength [MPa]</th></tr></thead><tbody><tr><td rowspan="3">Prismatic bricks</td><td rowspan="3">133x40x40</td><td><i>m</i></td><td>2204.2</td><td>6200.0</td><td>25.00</td></tr><tr><td><i>s</i></td><td>33.7</td><td>1072.5</td><td>0.84</td></tr><tr><td><i>CoV</i></td><td>1.5%</td><td>10.3%</td><td>3.34%</td></tr><tr><td colspan="2">N° of samples: 6</td><td></td><td></td><td></td><td></td></tr></tbody></table>

<caption>Table 3. Masonry properties from tests results on triplets (average values after 8 months of curing – standard EN1052-3:2002).</caption>

<footnote>*The parameter was determined at the same aging of curing as the test specimen (190 days). The modulus of elasticity was determined from the average slope of the secant line through the points at 50% and 80% of the compressive strength.</footnote>
<table><caption>Table 4. Masonry properties from axial compression tests results (average values after 8 months of curing – standard EN 1052-1:1998).</caption><thead><tr><th>Material</th><th>Dimensions [mm]</th><th>Shear strength [MPa]</th><th>Cohesion [MPa]</th><th>Internal friction coefficient [rad]</th></tr></thead><tbody><tr><td rowspan="3">Masonry triplets</td><td><i>m</i></td><td>173×223×118</td><td>0.46</td><td>0.031</td><td>0.785</td></tr><tr><td><i>s</i></td><td>1.0×1.20×0.95</td><td>0.21</td><td>-</td><td>-</td></tr><tr><td><i>CoV</i></td><td>0.60%×0.53%×0.80%</td><td>16.60%</td><td>-</td><td>-</td></tr><tr><td colspan="5">N° of samples: 12</td></tr></tbody></table>

<table><thead><tr><th>Material</th><th>Dimensions [mm]</th><th>Max force [N]</th><th>Density [kg/m³]</th><th>Elastic modulus* [MPa]</th><th>Compress. strength [MPa]</th></tr></thead><tbody><tr><td rowspan="3">Masonry wallets</td><td><i>m</i></td><td>619×468×118</td><td>5.0e+05</td><td>2255.2</td><td>2223.0</td><td>9.1</td></tr><tr><td><i>s</i></td><td>9.1×0.6×0.5</td><td>1.4e+04</td><td>20.5</td><td>309.6</td><td>0.2</td></tr><tr><td><i>CoV</i></td><td>1.5%×0.1%×0.4%</td><td>3.0%</td><td>1.0%</td><td>14.0%</td><td>2.0%</td></tr><tr><td colspan="6">N° of samples: 4</td></tr></tbody></table>

<table><caption>Table 5. Masonry properties from diagonal compression tests results (average values after 8 months of curing - standard ASTM E519/E519M).</caption><thead><tr><th>Material</th><th></th><th>Cross section area [mm²]</th><th>Tensile strength [MPa]</th><th>Shear modulus [MPa]</th></tr></thead><tbody><tr><td rowspan="3">Masonry wallets</td><td><i>m</i></td><td>118287.00</td><td>0.31</td><td>762.0</td></tr><tr><td><i>s</i></td><td>843.00</td><td>0.04</td><td>519.0</td></tr><tr><td><i>CoV</i></td><td>0.70%</td><td>14.3%</td><td>68.0%</td></tr><tr><td colspan="5">N° of samples: 2</td></tr></tbody></table>

* *Properties of the TRM system.* Since the results of the characterization tests were not yet available when the tests on the vault were carried out, for the pre-diction phase only the technical parameters provided by the producer were provided to the participants. In particular, the technical leaflet of the strengthening system proposed by KERAKOLL, based on a natural mortar (GEOCALCE® F ANTISISMICO) reinforced by a basalt/steel grid (GEOSTEEL GRID 200), was shared. Synthetically, the mortar was reported to have a compressive strength of at least 15 MPa (being classified as M15), a tensile strength of at least 5 MPa and an elastic modulus of 9 GPa, while the grid was reported to have a

<footnote>*The elastic modulus was obtained through the average of the longitudinal extensions of the four measurement positions, corresponding to a stress equal to one-third of the maximum stress.</footnote>
175

conventional tensile strength on bricks support of 945 MPa and an overall elastic modulus of 62 GPa.

176

177

* *Input sequences for the UNS and SM specimen.* The selected input, for both UNS and SM specimens, was the time series, named “AQA”, recorded during the L’Aquila earthquake, along the North-South component, which occurred on the 6th of April 2009 (Figure 3).

178

179

The maximum absolute acceleration, velocity and displacement of the target input are equal to 4.25 m/s², 26.33 cm/s and 3.70 cm, respectively. For the blind prediction phase, participants were provided with effective accelerations recorded at the shake table level for both configurations (UNS and SM) in a datasheet. The input signals had already been filtered, and baseline correction had been applied. Therefore, participants did not require any signal processing.

180

181

182

183

184

185

<figure><img src="image_4.png" /></figure>

186

Figure 3. Effective seismic input recorded at the base of the shaking table given to the participants.

187

188

189

* *Measurement system.* Detailed setup schemes were provided to precisely illustrate the positioning of all installed instruments (see Figure 4) and to indicate the specific locations where outputs were requested. In particular, the accelerations have been measured by 32

190

191
piezometer accelerometers (Accᵢ), the displacements by means of transducers 8 (LVDTᵢ) and 4 optical cameras (OCᵢ) and the forces in the steel cables were measured by 6 load cells (LCᵢ). Same devices were used for both seismic and dynamic identification tests. The evolution of damage occurring in the test specimen was recorded by five video cameras (VCᵢ) located in front of each elevation to draw a complete set of crack maps. Figure 4 presents the setup both for the unstrengthened specimen (UNS) and the strengthened specimen (SM).

<figure><img src="image_5.png" /></figure>

Fixed edge

<caption>Figure 4. Setup of the unstrengthened and strengthened specimen.</caption>

199

## 3.2 Data required from participants and evaluation criteria

Given all the previous data, the participant were asked to compile a “report template” where the following information/data were required:

201

202
203 • Description of the numerical model: geometry, discretization and tools; material
204 behaviours and type of analysis; main characteristics and size of the model (e.g. type of
205 elements, number of nodes, number of DOF, etc.); description of the modelling strategy
206 adopted to simulate the strengthening technique (TRM at the extrados), including the
207 adopted element types and material behaviours; boundaries conditions; results of
208 eigenvalue analysis, damping criteria.

209 • Brief description of failure mechanism obtained from numerical modelling, including
210 pictures of the deformed shape.

211 • Predicted crack pattern (based on total strains, crack widths, etc...).

212 • Maximum total displacement (in mm) at the following points during the applied time-
213 history of 75% for UNS and 150% for SM: OC1-y, OC2-x, OC2-y, OC4-x, OC4-y.

214 • Maximum total acceleration (in m/s²) at the following points during the applied time-
215 history of 75% for UNS and 150% for SM: Acc2-x, Acc6-x, Acc12-x, Acc14-x, Acc18-x,
216 Acc2-y, Acc6-y, Acc12-y, Acc14-y, Acc18-y.

217 • PGA (in m/s²) leading the strengthened specimen to severe damage or near collapse.

218 To facilitate the comparison of the pre-diction results between the various numerical models,
219 the evaluation grid shown in Table 6 was established. Four criteria for the UNS (C<sub>A</sub> - C<sub>D</sub>)
220 specimen and five criteria for the strengthened specimen (C<sub>E</sub> - C<sub>I</sub>) were defined. This grid
221 encompasses both qualitative and quantitative criteria, with varying weights assigned to each
222 criterion based on its relevance. In particular, C<sub>A</sub> - C<sub>B</sub> and C<sub>E</sub> - C<sub>F</sub> qualitative, as their scores are
223 determined by organizers through qualitative analysis of the correspondence between observed
224 failure mechanisms and crack patterns in experimental tests and those predicted by the models.
225 In contrast, C<sub>C</sub> - C<sub>D</sub> and C<sub>G</sub> - C<sub>I</sub> are quantitative, as their scores are determined by calculating the
226 error between experimental and numerical results on various quantitative parameters
227 (displacements and accelerations).
<table><caption>Table 6. Evaluation criteria considered for the blind prediction.</caption><thead><tr><th colspan="2">Objective 1 - Unstrengthened specimen - L'Aquila 75%</th><th>Weights</th></tr></thead><tbody><tr><td>C<sub>A</sub> - Simulation of the experimental failure mechanism</td><td>Not matching: 0% Matching: 100%</td><td>30</td></tr><tr><td>C<sub>B</sub> - Crack pattern at the end of the specified level of input, when reparation is needed/recommended</td><td>Not good: 0% Fair: 50%<br>Good: 75% Very good: 100%</td><td>30</td></tr><tr><td>C<sub>C</sub> - Maximum total displacement at the following points during the applied time-history (see test setup):<br>OC<sub>1-y</sub> - OC<sub>2-x</sub> - OC<sub>2-y</sub> - OC<sub>4-x</sub> - OC<sub>4-y</sub></td><td>The score is inversely proportional (linear function) to the average error of all values:<br>Average error: 0%---50%---&ge;100%<br>Score: 100%---50%---0%</td><td>20</td></tr><tr><td>C<sub>D</sub> - Maximum total acceleration at the following points during the applied time-history (see test setup):<br>Acc<sub>2-x</sub> - Acc<sub>6-x</sub> - Acc<sub>12-x</sub> - Acc<sub>14-x</sub> - Acc<sub>18-x</sub><br>Acc<sub>2-y</sub> - Acc<sub>6-y</sub> - Acc<sub>12-y</sub> - Acc<sub>14-y</sub> - Acc<sub>18-y</sub></td><td>The score is inversely proportional (linear function) to the average error of all values:<br>Average error: 0%---50%---&ge;100%<br>Score: 100%---50%---0%</td><td>20</td></tr><tr><th colspan="2">Objective 2 - Strengthened specimen - L'Aquila 150%</th><th>Weights</th></tr><tr><td>C<sub>E</sub> - Simulation of the experimental failure mechanism</td><td>Not matching: 0% Matching: 100%</td><td>30</td></tr><tr><td>C<sub>F</sub> - Crack pattern at the end of the specified level of input</td><td>Not good: 0% Fair: 50%<br>Good: 75% Very good: 100%</td><td>30</td></tr><tr><td>C<sub>G</sub> - Maximum total displacement at the following points during the applied time-history (see test setup):<br>OC<sub>1-y</sub> - OC<sub>2-x</sub> - OC<sub>2-y</sub> - OC<sub>4-x</sub> - OC<sub>4-y</sub></td><td>The score is inversely proportional (linear function) to the average error of all values:<br>Average error: 0%---50%---&ge;100%<br>Score: 100%---50%---0%</td><td>15</td></tr><tr><td>C<sub>H</sub> - Maximum total acceleration at the following points during the applied time-history (see test setup):<br>Acc<sub>2-x</sub> - Acc<sub>6-x</sub> - Acc<sub>12-x</sub> - Acc<sub>14-x</sub> - Acc<sub>18-x</sub><br>Acc<sub>2-y</sub> - Acc<sub>6-y</sub> - Acc<sub>12-y</sub> - Acc<sub>14-y</sub> - Acc<sub>18-y</sub></td><td>The score is inversely proportional (linear function) to the average error of all values:<br>Average error: 0%---50%---&ge;100%<br>Score: 100%---50%---0%</td><td>15</td></tr><tr><td>C<sub>I</sub> - PGA for severe damage/near collapse, when you think it is not possible to increase the amplitude<br>The score is inversely proportional (linear function) to the error of the PGA:<br>Error: 0%---50%---&ge;100%<br>Score: 100%---50%---0%</td><td></td><td>10</td></tr></tbody></table>

The final score $C_{tot}$ is derived from the (Equation 1:

$$C_{tot} = 0.5 (C_{A} + C_{B} + C_{C} + C_{D}) + 0.5 (C_{E} + C_{F} + C_{G} + C_{H} + C_{I}) \quad (\text{Equation 1})$$

In the event of a tie, the organising team was tasked to decide the winner based on the closest
crack pattern to the experimental results (criteria $C_B$ and $C_F$).

## 3.3 Participants and pre-diction models

Table 7 lists all the groups, their provenance and each of them is identified with a group number “Gi”. Same group numbering remains consistent throughout the paper. To differentiate between multiple contributions from the same group, subsequent submissions are indicated with subscript letters (a, b).
<table><caption>239 Table 7. List of the participants.</caption><thead><tr><th>Group</th><th>Participants</th><th>Institutions</th><th>Country</th><th>Reference if available</th></tr></thead><tbody><tr><td>G1</td><td>D. Pellegrini</td><td>Institute of Information Science and Technologies "Alessandro Faedo" – ISTI/CNR Pisa</td><td>Italy</td><td>(Pellegrini 2023)</td></tr><tr><td rowspan="3">G2a,b</td><td>A. Ferrante</td><td>LMGC, CNRS, Université de. Montpellier</td><td rowspan="3">France</td><td rowspan="3"></td></tr><tr><td>F. Dubois</td><td>Université de Toulouse, UPS, INSA, LMDC, Toulouse</td></tr><tr><td>P. Morenon</td><td></td></tr><tr><td>G3</td><td>A. Marasca<br>M. Moretti</td><td>Hopper Engineering and Consultancy</td><td>Netherland</td><td></td></tr><tr><td rowspan="3">G4</td><td>F. Portioli</td><td rowspan="3">University of Naples "Federico II"</td><td rowspan="3">Italy</td><td rowspan="3">(Gagliardo, Cascini, and Portioli 2023)</td></tr><tr><td>L. Cascini</td></tr><tr><td>R. Gagliardo</td></tr><tr><td rowspan="3">G5</td><td>P. Fici</td><td rowspan="3">ARUP</td><td rowspan="3">UK</td><td rowspan="3"></td></tr><tr><td>A. Zanichelli</td></tr><tr><td>F. I. Garzon</td></tr><tr><td rowspan="4">G6</td><td>L. Sorrentino</td><td rowspan="4">"Sapienza" University of Rome</td><td rowspan="4">Italy</td><td rowspan="4">(AlShawa, Liberatore, and Sorrentino 2023)</td></tr><tr><td>AlShawa O.</td></tr><tr><td>D. Liberatore</td></tr><tr><td></td></tr><tr><td rowspan="6">G7</td><td>D. Malomo</td><td>McGill University</td><td>Canada</td><td rowspan="6">(Oktiovan et al. 2023)</td></tr><tr><td>A. Mehrotra</td><td>TU Delft</td><td>Netherland</td></tr><tr><td>A. Dell'Endice</td><td>ETH</td><td>Switzerland</td></tr><tr><td>B. Pulatsu</td><td>Carleton University</td><td rowspan="2">Canada</td></tr><tr><td>L. Davis</td><td>McGill University</td></tr><tr><td>F. Santamaria</td><td>Instituto de Ingenieria UV</td><td></td></tr><tr><td rowspan="4">G8</td><td>F. Peña Mondragon</td><td rowspan="4">Instituto de Ingenieria UNAM</td><td rowspan="4">Mexico</td><td rowspan="4"></td></tr><tr><td>M. Chavez Cano</td></tr><tr><td>D. Durán Sánchez</td></tr><tr><td></td></tr><tr><td rowspan="4">G9</td><td>G.P. Lignola</td><td rowspan="4">University of Naples "Federico II"</td><td rowspan="4">Italy</td><td rowspan="4"></td></tr><tr><td>G. Ramaglia</td></tr><tr><td>A. Prota</td></tr><tr><td></td></tr><tr><td rowspan="6">G10a,b</td><td>V. Cardinali</td><td rowspan="6">University of Florence</td><td rowspan="6">Italy</td><td rowspan="6">(Pintucchi, Trovatelli, and Zani 2024)</td></tr><tr><td>N. Zani</td></tr><tr><td>B. Pintucchi</td></tr><tr><td>M. Tanganelli</td></tr><tr><td>F. Trovatelli</td></tr><tr><td>M. Malena</td></tr><tr><td rowspan="3">G11</td><td>M. Lorello</td><td>Roma Tre University</td><td>Italy</td><td></td></tr><tr><td>I. Caliò</td><td>University of Catania</td><td>Italy</td><td rowspan="3">(Chácara et al. 2023)</td></tr><tr><td>C. Chácara</td><td>Pontificia Universidad Católica del Perú</td><td>Peru</td></tr><tr><td>G12</td><td>F. Cannizzaro</td><td>University of Catania</td><td>Italy</td></tr><tr><td></td><td>B. Pantò</td><td>University Of Durham</td><td>UK</td><td></td></tr><tr><td></td><td>D. Rapicavoli</td><td>University of Catania</td><td>Italy</td><td></td></tr><tr><td rowspan="3">G13</td><td>P. Kesavan</td><td>IIT Madras</td><td>India</td><td></td></tr><tr><td>M. Petracca</td><td>ASDEAsoft</td><td>Italy</td><td></td></tr><tr><td>K. Sethumadhavan</td><td>IIT Madras</td><td>India</td><td></td></tr><tr><td></td><td>A. Menon</td><td>University "Gabriele D'Annunzio" Pescara and Chieti</td><td>Italy</td><td></td></tr><tr><td></td><td>G. Camata</td><td></td><td></td><td></td></tr></tbody></table>

240 Many different modelling approaches were proposed: 10 out of the 15 models are based on the Finite Element Method (FEM); 3 models are based on the Discrete Elements Method (DEM); 1 model is based on the Applied Element Method (AEM), mixing FEM and DEM; 1 model is based on Rigid Block Limit Analysis using mathematical programming. In the following
244

subsection, a description of prediction model strategies and assumptions is provided to facilitate
comparison and highlight key findings.

245

246

Masonry description

247

In this subsection, an outline of how various numerical models describe homogenised masonry,
the discretization of the bricks along the shell of the vault, in case of micromodelling, as well
as the material constitutive law, is provided. Each model offers unique approaches and
assumptions, which contribute to the diverse range of predictions observed.

248

249

250

251

For what concerns the FEM models, which constitute the majority of the predictions, eight out
of the ten models describe the masonry as a continuum material (G1, G2a, G5, G8, G9, G10a,
G10b, G11), adopting a macromodelling approach, while two of them describe it as an
assemblage of solid (G6) or shell (G13) elements with non-linear interfaces, adopting a
micromodelling approach where each solid/shell element corresponds to one brick (or to a so
called "extended" brick, to take into account the thickness of the mortar joint).

252

253

254

255

256

257

**G12** proposes a discrete macromodelling approach (DMEM method), while two of them adopt
a micro modelling approach (G2a and G7), where each element corresponds to one brick. A
similar approach is used also for the model based on AEM (G3) and for the model using Rigid
Block Limit Analysis (G4).

258

259

260

261

Within the FEM models adopting a macromodelling approach, half utilise solid elements (G1,
G2b, G8, G10a) and while the other half employ shell elements (G5, G9, G10b, G11). The
constitutive equations are various. G1 has a nonlinear elastic material model with zero tensile
strength and bounded compressive strength. G2b uses a non-linear material model with
smeared cracking, with a Rankine criterion for tensile cracking/re-closure and Drucker-Prager
criteria for shear-compression. G8 adopts a non-linear material model with smeared cracking
based on Willam-Warnke model. G10a uses a non-linear isotropic damaged-plasticity material

262

263

264

265

266

267
model (Concrete Damage Plasticity). **G5** has a non-linear material model with smeared
cracking, **G9** uses a elastic-perfectly plastic material in compression, with a limited
strength/strain in tension and compression. **G10b** adopts a non-linear elastic model with
bounded tensile and compressive strength based on the infinitesimal strain theory. Lastly, **G11**
adopts an elastic perfectly-plastic plate model, whose strength domain is derived by a
homogenization procedure on a thin periodic heterogeneous plate made of 3D infinitely
resistant blocks connected by Mohr-Coulomb interfaces.

Within the FEM models adopting a micromodelling approach, in **G6** elastic solid elements are connected with non-linear interfaces with tension cut-off, cohesion and Coulomb's behaviour in shear, while in **G13** both shell elements and interface elements are modelled with a damage-plasticity model.

Regarding DEM models, based on a micromodelling approach, **G2a** have either rigid blocks
with non-smooth frictional cohesive contacts, while **G7** adopts linear elastic blocks connected
through zero-thickness interface springs with the Pulatsu *et al.* (2023) softening contact model.
DEM models based on macromechanical approach (named Discrete Macro Element Method –
**G12**), masonry is discretised with panels connected by means of 2D zero-thickness interface
elements. In the model based on Applied Element Method (**G3**) the masonry is modelled
through rigid solid element with non-linear interfaces. Finally, in the model using Rigid Block
Limit Analysis (**G4**), masonry is described by means of rigid blocks connected with no-tension
frictional interfaces with zero cohesion, infinite compressive strength and sliding failure
governed by a Coulomb friction criterion.

289 *Accessory structures and boundary conditions*

The following subsection outlines the description of boundary conditions, namely: steel cables,
frame, masses and profiles, masonry piers, reinforcement and reinforced concrete slab above
292 the shake table where the specimen stand on. All these elements are here named as "accessory structures" within various numerical models. Each model adopts distinct approaches and assumptions to incorporate these factors into the structural analysis.
293
294

295 Regarding the steel cables, **G1**, **G3**, **G9**, **G12**, **G13** groups have modelled them as elastic truss elements, while **G8** adopted inelastic truss elements. **G5** and **G10b** groups have considered them as linear elastic beams, **G2b** and **G10a** have modelled them with linear elastic solid elements. **G2a** and **G6** have used linear elastic discrete springs. Finally, **G4** and **G11** have represented them as infinitely rigid internal constraints.
296
297
298
299

300 The majority of groups have opted for simplified representations of accessory structures such as piers and steel elements. Specifically, only the masonry piers were physically modelled, with steel profiles introduced solely as boundary conditions (**G2a**, **G2b**, **G6**, **G8**, **G9**, **G11**). Only in **G4** and **G10b**, the piers were not modelled either, reducing them (along with all the other accessory structures) to restraints given by external boundary conditions.
301
302
303
304

305 **G1**, **G3**, **G10a**, **G12**, **G13** have meticulously modelled the accessory structures, delineating the masonry piers, infill, steel masses and cables. In general, all the accessory structures were modelled with linear elastic materials as part for **G10a** and **G13** where the masonry piers were considered as inelastic.
306
307
308

309 Only **G5** and **G7** have modelled all the accessory structures, including the shaking table. In group **G5** the masonry piers stand on steel wheels (modelled by 1D dummy elements with friction) laying on the shaking table plane. In group **G7**, the piers were free to move in the X and Y directions on the shaking table plane in order to simulate the roller supports.
310
311
312

313 Nine out of the thirteen groups have included in their models the reinforcement at the extrados, namely **G1**, **G3**, **G5**, **G6**, **G7**, **G8**, **G9**, **G10a,b**, **G13**, while none of them considered the contribution of the grout injections on the strengthened configuration. In all the models, the
314
315
316 TRM was introduced by adding a further layer (either solid or bidimensional) on the shell of
317 the vault. G1, G3, G8, G9, G10a,b considered the TRM system as an additional layer, made
318 by an equivalent homogeneous non-linear material, while in other cases, a micromodelling
319 approach was used. G5 added a new layer of shell elements connected to the curved masonry
320 vault through of a 100x100mm grid of dummy beams representing the potential debonding of
321 TRM. In G6, an additional 10-mm-thick layer of solid finite elements was connected to the
322 masonry vault with interface elements with reduced tensile and shear strengths. In G7, TRM
323 was idealised as a discontinuous assembly of 1 cm-thick linear elastic solid elements with the
324 same cross section of the bricks. Only the fibers of the TRM are considered in G13, by
325 introducing a layer of non-linear shell elements with a thickness of 0.032 mm and limited tensile
326 strength.

327 Concerning the type of analysis, most of the groups performed a nonlinear dynamic analysis.
328 G3 and G6 have performed it incrementally, in step of 25% of amplitude, as it was done in the
329 experimental tests. G4 performed a limit analysis, thus providing only the mode of collapse and
330 the value of the PGA at severe limit state. Table 8 and Figure 5 synthetise the different
331 modelling approaches adopted, summarising also the number of DOF of each model.

<table><caption>332 Table 8. Group identification and adopted modelling approaches.</caption><thead><tr><th>Group</th><th>Masonry vault</th><th>Acc. structures</th><th>Strength.</th><th>DOF<sup>1</sup></th><th>Software</th><th>Type of analysis</th></tr></thead><tbody><tr><td>G1</td><td>FEM<br>Macromodelling</td><td>Complete</td><td>Yes</td><td>88056</td><td>NOSA-ITACA</td><td>Non linear dynamic</td></tr><tr><td>G2a</td><td>DEM<br>Micromodelling</td><td>Simplified</td><td>No</td><td>7452</td><td>LMGC90</td><td>Non linear dynamic</td></tr><tr><td>G2b</td><td>FEM<br>Macromodelling</td><td>Simplified</td><td>No</td><td>143390</td><td>LG3, MGC90</td><td>Non linear dynamic</td></tr><tr><td>G3</td><td>AEM<br>Micromodelling</td><td>Complete</td><td>Yes</td><td>143658</td><td>ELS</td><td>Non linear dynamic (incremental)</td></tr><tr><td>G4</td><td>RBLA<br>Micromodelling</td><td>Simplified</td><td>No</td><td>6781</td><td>LiABlock_3D</td><td>Limit equilibrium</td></tr></tbody></table>
<table><tr><td>G5</td><td>FEM<br>Macromodellin<br>g</td><td>Complete<br>(with shaking table)</td><td>Yes</td><td>≈30000</td><td>LS-DYNA</td><td>Non linear<br>dynamic</td></tr><tr><td>G6</td><td>FEM<br>Micromodellin<br>g</td><td>Simplified</td><td>Yes</td><td>≈15000</td><td>LS-DYNA</td><td>Non linear<br>dynamic<br>(incremental)</td></tr><tr><td>G7</td><td>DEM<br>Micromodellin<br>g</td><td>Complete<br>(with shaking table)</td><td>Yes</td><td>36864</td><td>3DEC</td><td>Non-linear<br>dynamic</td></tr><tr><td>G8</td><td>FEM<br>Macromodellin<br>g</td><td>Simplified</td><td>Yes</td><td>359328</td><td>ANSYS</td><td>Non linear<br>dynamic</td></tr><tr><td>G9</td><td>FEM<br>Macromodellin<br>g</td><td>Simplified</td><td>Yes</td><td>19684</td><td>SAP 2000</td><td>Non linear<br>dynamic</td></tr><tr><td>G10a</td><td>FEM<br>Macromodellin<br>g</td><td>Complete</td><td>Yes</td><td>60165</td><td>ABAQUS</td><td>Non linear<br>dynamic</td></tr><tr><td>G10b</td><td>FEM<br>Macromodellin<br>g</td><td>Simplified</td><td>Yes</td><td>12540</td><td>MADY</td><td>Non linear<br>dynamic</td></tr><tr><td>G11</td><td>FEM<br>Macromodellin<br>g</td><td>Simplified</td><td>No</td><td>≈120000</td><td>ABAQUS</td><td>Non-linear<br>dynamic</td></tr><tr><td>G12</td><td>DEM<br>Macromodellin<br>g</td><td>Complete</td><td>No</td><td></td><td>HiStrA</td><td></td></tr><tr><td>G13</td><td>FEM<br>Micromodellin<br>g</td><td>Complete</td><td>Yes</td><td>66403</td><td>STKO +<br>OpenSees</td><td>Non-linear<br>dynamic</td></tr></table>

Notes:

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

G1

G2a

G2b

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

G3

G4

G5

<footnote n="1">¹ DOF of the unstrengthened model</footnote>

<footnote n="2">² Rigid Block Limit Analysis</footnote>
<figure><img src="image_12.png" /><figcaption>Figure 5. Models proposed by the different groups.</figcaption></figure>

## 3.4 Experimental results

To enhance comprehension of the experimental outcomes in comparison with the numerical model results, a concise summary is here provided. For more detailed insights, please refer to the specific publication by Bianchini *et al.* (2024).

### 3.4.1 Qualitative data needed for the evaluation (criteria A-B-E-F)

In the UNS specimen, no significant damage was depicted after the first two seismic tests (10%, 25% amplitude). However, diagonal cracks, indicative of in-plane shear behaviour, started to appear from 50% of amplitude. After 75%, the two major diagonal cracks crossed both the
342

extrados and intrados of the shell with a crack width larger than 1 cm. The formation of hinges
was notable in correspondence between the Northern and Western arches.

343

344

The SM specimen did not exhibit any significant damage until 75% of amplitude; afterwards,
light in-plane shear diagonal cracks developed at the base of the groins (NE-NW-SW corners),
affecting the extrados with an average crack width of less than 0.5 mm. The intrados remained
almost undamaged until the end of the seismic tests. From 100% of amplitude, a complex
mechanism was observed, which was characterised by uplifts, rocking and rotations of the steel
masses, the impost of the vault and the fixed piers. During the last two seismic tests, 125% and
150%), severe cracks occurred at the SW corner, damaging the fixed pier and the southern arch,
with detachments of some bricks. Overall, the TRM system remained securely attached to the
masonry extrados throughout the duration of the seismic tests, showing no signs of debonding.

345

346

347

348

349

350

351

352

353

Although the damage increased, the strengthened specimen presented severe damage only at
the end of the final seismic test: the movable edge at the opposite side showed a complete
separation of the bed joints between the supports and the shell of the vault, where the impost of
the vault begins. Figure 6 summarises the crack pattern and damage observed after the last
seismic tests of each configuration while Table 9 gives a brief description of it. The evidence
of the damage pattern was compared with the numerical results provided by the groups as per
criteria CA – CB – CE – CF of the evaluation form.

354

355

356

357

358

359

<figure><img src="image_13.png" /><figcaption>Extrados</figcaption></figure>

<figure><img src="image_14.png" /></figure>
<figure><img src="image_15.png" /><figcaption>Figure 6. Comparison of crack patterns for unstrengthened and strengthened configurations:</figcaption></figure>

(a) UNS-75% (b) SM-150%.

<table><caption>Table 9. Brief damage description after each seismic test.</caption><thead><tr><td>Denomination</td><td>Description of damage</td></tr></thead><tbody><tr><td>UNS-10%</td><td>No damage</td></tr><tr><td>UNS-25%</td><td>No damage</td></tr><tr><td>UNS-50%</td><td>Light diagonal cracks at the extrados</td></tr><tr><td>UNS-75%</td><td>Severe diagonal cracks passing the thickness of the vault</td></tr><tr><td>SM-25%</td><td>No damage</td></tr><tr><td>SM-50%</td><td>No damage</td></tr><tr><td>SM-75%</td><td>No damage</td></tr><tr><td>SM-100%</td><td>Light diagonal cracks at the extrados, rocking starts to appear</td></tr><tr><td>SM-125%</td><td>Light diagonal cracks at the extrados, detachment between the supports of the vault and the steel masses, and detachment between fixed piers and the base</td></tr><tr><td>SM-150%</td><td>Medium diagonal cracks at the extrados, severe damage of the fixed piers, detachment between the supports of the vault and the steel masses, and detachment between fixed piers and the base</td></tr></tbody></table>

360

361

362

363
### 3.4.2 Quantitative data needed for the evaluation (criteria C-D-G-H-I)

The quantitative results asked to the participants and evaluated by criteria C<sub>C</sub> – C<sub>D</sub> – C<sub>G</sub> – C<sub>H</sub> – C<sub>I</sub> were: maximum absolute total displacement, maximum total acceleration at the 75% of the UNS specimen and at the 150% of the SM specimen of the specific measured points (Figure 4 and Table 6) and the value of the highest PGA (in m/s²) applied on the SM specimen to obtain severe damage/near-miss collapse. Those experimental outcomes are summarised in Table 10.

<table><caption>Table 10. Experimental displacements and acceleration measured.</caption><thead><tr><th rowspan="2">Test sequence</th><th rowspan="2">PGD [mm]</th><th colspan="2">OC<sub>1</sub> [mm]</th><th colspan="2">OC<sub>2</sub> [mm]</th><th colspan="2">OC<sub>3</sub> [mm]</th><th colspan="2">OC<sub>4</sub> [mm]</th></tr><tr><th>YN-S</th><th>Vertical*</th><th>X W-E</th><th>Y N-S</th><th>X W-E</th><th>Y N-S</th><th>X W-E</th><th>Y N-S</th></tr></thead><tbody><tr><td>UNS-75%</td><td></td><td>30.74</td><td>10.64</td><td>4.4</td><td>52.51</td><td>5.08</td><td>32.80</td><td>1.25</td><td>32.05</td></tr><tr><td>SM-150%</td><td>54.15</td><td>56.03</td><td>12.51*</td><td>42.81</td><td>63.48</td><td>15.79</td><td>64.68</td><td>12.77</td><td>62.17</td></tr></tbody></table><table><thead><tr><th rowspan="2">Test sequence</th><th rowspan="2">PGA [m/s<sup>2</sup>]</th><th colspan="2">ACC<sub>2</sub> [m/s<sup>2</sup>]</th><th colspan="2">ACC<sub>6</sub> [m/s<sup>2</sup>]</th><th colspan="2">ACC<sub>12</sub> [m/s<sup>2</sup>]</th><th colspan="2">ACC<sub>14</sub> [m/s<sup>2</sup>]</th><th colspan="2">ACC<sub>18</sub> [m/s<sup>2</sup>]</th></tr><tr><th>W-E</th><th>N-S</th><th>W-E</th><th>N-S</th><th>W-E</th><th>N-S</th><th>W-E</th><th>N-S</th><th>W-E</th><th>N-S</th></tr></thead><tbody><tr><td>UNS-75%</td><td>3.62</td><td>4.41</td><td>4.54</td><td>3.97</td><td>4.52</td><td>3.43</td><td>4.91</td><td>5.22</td><td>4.46</td><td>3.77</td><td>3.46</td></tr><tr><td>SM-150%</td><td>7.27</td><td>10.03</td><td>9.41</td><td>11.73</td><td>9.06</td><td>9.47</td><td>8.37</td><td>17.33</td><td>10.56</td><td>8.17</td><td>7.67</td></tr></tbody></table>

## 3.5 Numerical results and discussion

### 3.5.1 Criteria related to the mechanism and the damage pattern

The results of the qualitative criteria related to the mechanism (C<sub>A</sub> – C<sub>E</sub>) and the crack pattern (C<sub>B</sub> – C<sub>F</sub>) for the UNS specimen, and the SM specimen will be firstly discussed. In Figure 7 the unweighted scores assigned to the related criteria are shown. They were assigned according to this logic:

* Criterion C<sub>A</sub> – C<sub>E</sub>: verbal description of the failure mechanism together with pictures of the deformed shapes (from the seismic analysis) and/or modal shapes (from the modal analysis).

* Criterion C<sub>B</sub> – C<sub>F</sub>: crack pattern based on plastic strains, crack widths, etc...

It can be observed in that almost all the models (11 out of 15, considering a threshold of 75%) were able to describe well the failure mechanism of the UNS specimen (criterion C<sub>A</sub>). However,
only a sub-set of the models that predicted the mechanism well were able to predict the crack pattern (criterion C<sub>B</sub>) with the same accuracy (7 out of 15). In some cases, even if the model was able to catch the in-plane shear mechanism of the vault, then it failed in recognizing the prevalence of the cracks along the diagonals, concentrating the damage on the supports (see **G6, G10a and G10b** in Figure 8).

<figure><img src="image_16.png" /><figcaption>Figure 7. Absolute scores in the qualitative criteria (damage mechanism/crack pattern).</figcaption></figure>

<figure><img src="image_17.png" /></figure>
<figure><img src="image_18.png" /></figure>

G6

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

G7

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

G8

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

G9

<figure><img src="image_25.png" /></figure>

G10a

<figure><img src="image_26.png" /></figure>

G10b

G11

G12

G13

<caption>Figure 8. Classification of predicted crack pattern for the UNS models from top view (damage mechanism/crack pattern).</caption>

For the SM specimen, 6 out of 10 models offered a highly satisfactory description of the mechanism (criterion C<sub>E</sub>) as possible to observe from Figure 9. Remarkably, all these groups are part of the subset that accurately predicted the behaviour of the UNS specimen.

<figure><img src="image_27.png" /></figure>
<figure><img src="image_28.png" /><figcaption>Figure 9. Classification of predicted crack pattern for the SM models from relevant view</figcaption></figure>

396

397

(damage mechanism/crack pattern).
### 3.5.2 Criteria related to the displacements and acceleration of specific points

In Figure 10, the unweighted scores assigned to the quantitative criteria related to displacements and acceleration for the UNS specimen (criteria C and D) and the SM specimen (criteria G and H) are presented. It is worth noting that for some models it was not possible to provide the displacements or the acceleration because of the type of analysis performed (see models G4 and G11). From Figure 10, it can be observed that the scores obtained by the participants are in general lower than those obtained for the qualitative criteria (most of the groups have obtained a score lower than 60).

<figure><img src="image_29.png" /><figcaption>Figure 10. Absolute scores in the quantitative criteria (displacements and accelerations).</figcaption></figure>

The scores in Figure 10 are inversely proportional to the mean of the displacement/accelerations for the required acquisition points. In Figure 11 the mean errors on the displacements for each group, for the UNS and SM specimen, are represented (white column corresponds to those cases in which the groups did not provide a forecast of displacement and a 100% error was automatically assigned). The mean error of the groups that provided the results is 56% (with a standard deviation of 17%) for the UNS specimen and 60% for the SM specimen (with a standard deviation of 30%).
<figure><img src="image_30.png" /><figcaption>Figure 11. Mean errors in the evaluation of the displacements (UNS and SM specimen).</figcaption></figure>

Figure 12 demonstrates that for the UNS specimen, the majority of groups have underestimated the displacements, indicating that the models are stiffer than the experimental specimen. It is also worth noting that the errors on the displacements along the x direction are in general larger than those on the y direction. In fact, the mean error on the y direction is 36% for both UNS and SM specimen, against 87% and 97% respectively on the x direction. This scatter can be attributed to two different effects: to the different extent of displacements (the displacements along x are one order of magnitude smaller than those along y), leading to larger experimental errors; to the boundary conditions, whose effect is particularly relevant in the x direction.

A similar result was found for the SM specimen (Figure 13): the errors on the estimation of y are generally lower than those of the UNS specimen (in 6 models lower than 25%) and not strictly negative.
<figure><img src="image_31.png" /><figcaption>Figure 12. Absolute errors in the evaluation of the displacements (UNS specimen).</figcaption></figure>

<figure><img src="image_32.png" /><figcaption>Figure 13. Absolute errors in the evaluation of the displacements (SM specimen).</figcaption></figure>

Regarding the accelerations, Figure 14 displays the mean errors on the accelerations for each group of acquisition point, concerning both the UNS and SM specimens. The white columns represent cases in which groups did not provide displacement forecasts, resulting in an automatic assignment of a 100% error. In this case, the mean error of the groups that provided the results is 144% (with a standard deviation of 177%) for the UNS specimen and 111% for the SM specimen (with a standard deviation of 119%) demonstrating again a wide range of results.
<figure><img src="image_33.png" /><figcaption>Figure 14. Mean errors in the evaluation of the accelerations (UNS and SM specimen).</figcaption></figure>

By analysing the absolute errors obtained for the accelerations (Figure 15 and Figure 16, for the UNS and SM specimen, respectively), it can be observed that in this case the errors are not strictly negative or positive.

<figure><img src="image_34.png" /><figcaption>Figure 15. Absolute errors in the evaluation of the accelerations (UNS specimen)</figcaption></figure>
<figure><img src="image_35.png" /><figcaption>Figure 16. Absolute errors in the evaluation of the accelerations (SM specimen)</figcaption></figure>

Finally, regarding the criterion C₁, related to the PGA reached at collapse by the SM specimen, the results were satisfying. The mean value of the numerical PGA submitted by the participants was equal to 9.6 m/s², that compared with the experimental PGA equal to 7.2 m/s², indicates a good agreement between the numerical and tests results. In addition, it's important to highlight that this value is influenced by the individual judgment of each participant, inf act, each group had to decide when their own models could no longer withstand additional seismic action.

Figure 17 shows the results obtained by the different groups; the mean absolute error, in this case, is 56%, with a standard deviation of 39%.
<figure><img src="image_36.png" /><figcaption>Figure 17. Absolute errors in the evaluation of the PGA in the SM specimen</figcaption></figure>

<figure><img src="image_37.png" /></figure>

G1

<figure><img src="image_38.png" /></figure>

G2a

<figure><img src="image_39.png" /></figure>

G2b

<figure><img src="image_40.png" /></figure>

G3

<figure><img src="image_41.png" /></figure>

G4

<figure><img src="image_42.png" /></figure>

G5

<figure><img src="image_43.png" /></figure>

G6

<figure><img src="image_44.png" /></figure>

G7

<figure><img src="image_45.png" /></figure>

G8

In Figure 18, the unweighted grades obtained by each group for each criterion are represented.

It can be observed that there are three groups that have the most large and symmetric area (G1, G5, and G13), meaning that provided more reliable results.
<figure><img src="image_46.png" /><figcaption>461 Figure 18. Grades per criterion.</figcaption></figure>

462 Finally, in Figure 19 the final grades obtained by the different groups are reported. Groups **G1**, **G13** and **G5** are the more performant groups whose numerical models better replicate the experimental activities.
463
464

<figure><img src="image_47.png" /><figcaption>465
466 Figure 19. Partial and total scores obtained from the different groups.</figcaption></figure>

467 **4 Post-dictions**

468 The second part of this paper deals with the post-diction analyses where participants updated
469 their initial pre-diction models based on the main outcomes of the experimental results and
470 additional data. The participants in particular were asked to:
471

* to calibrate the numerical models based on the dynamic properties;

472

* to enhance the capability of different modelling/analysis approaches and provide better justification for numerical assumptions.

473

474

The objectives of this phase to achieve a wise understanding of the limitations of modelling when different level of knowledge is available to the professionals.

475

476

## 4.1 New data provided to the participants

477

Once the blind prediction has been completed, a new dataset was provided to the research groups for conducting post-diction simulations (Bianchini, Calderini, et al. 2023b). This dataset included:

478

479

480

* Crack patterns observed at the conclusion of tests for both the UNS and SM specimens (Figure 6);

481

482

* maximum absolute total displacement and total acceleration at the 75% of the UNS specimen and at the 150% of the SM specimen, and highest PGA (in m/s²) applied on the SM specimen to obtain severe damage/near-miss collapse (Table 10);

483

484

485

* complete time histories of absolute displacements (in mm) and accelerations (in m/s²) for each seismic tests of all the acquisition points (Figure 4, 32 time-histories for each run);

486

487

* Results from dynamic identification tests, including experimental frequencies and mode shapes of the first relevant modes, for both the UNS and SM test specimens.

488

489

For the sake of completeness in this publication, the results of the dynamic tests are briefly summarized here. Specifically, three main global natural frequencies and modal shapes for the UNS specimen (Figure 20) and two for the SM specimen (Figure 21) were given to the participants.

490

491

492
493

The first global mode of UNS vault had a frequency that ranged from 6.15 Hz (undamaged configuration, period of T₁ = 0.16 s) to 5.57 Hz (most damaged configuration) and corresponded to the pure in-plane shear behaviour of the system along the North-South direction. The second global mode involved a global behaviour of the shell of the vault in the transverse direction, with an out-of-phase behaviour of the eastern arch (associated to $Acc_{16}$).

494

495

496

497

498

The third global mode was only depicted along x direction and its frequencies ranged from 19.39 Hz (undamaged configuration) to 19.30 Hz (after 50% of amplitude). It entailed a global behaviour of the shell acting in the vertical direction.

499

500

<figure><img src="image_48.png" /></figure>

501

Figure 20. Natural frequencies and mode shapes of the UNS specimen: (a) first global

502

longitudinal mode; (b) second global transverse mode; (c) third global vertical mode obtained.

503

504

505

506

507

508

For what concerns the SM specimen, the first natural frequency had been increased to 6.15 Hz after the repair of the shell injecting new grout. This value was equal to the first natural frequency of the undamaged UNS specimen. The first global mode had a frequency that ranged from 6.15 Hz (undamaged configuration) to 4.5 Hz (most damaged configuration) after the 150% of the seismic tests. This mode corresponded to a pure shear behaviour of the system along the North-South. The second global mode presented a complex global behaviour of the
shell of the specimen with torsion, which also involved the fixed edge. No other global modes were found.

<figure><img src="image_49.png" /><figcaption>Figure 21. Natural frequencies and mode shapes of the SM specimen: (a) first global longitudinal mode, (b) second global transverse mode.</figcaption></figure>

## 4.2 Updated models

The following groups participated to the post-diction phase: **G1, G4, G6, G7, G9, G10, G12, G13**. In resubmitting the results, due to the discrepancies between the pre-diction model results and the experimental data, it was necessary to reconsider the assumptions and modelling choices (Table 11). All groups, except **G4** which changed the boundary conditions, varied the mechanical parameters of the material (Young Modulus and/or friction coefficient). **G7** and **G12** also varied the Rayleigh coefficient.

<table><caption>Table 11. Summary of the modelling changes per each group.</caption><thead><tr><th>Group</th><th>Variations</th><th>Pre-diction</th><th>Post-diction</th></tr></thead><tbody><tr><td>G1</td><td><i>E</i> of the entire vault<br>Rayleigh coefficient</td><td>2.22 GPa<br>2.5%</td><td>0.84 GPa<br>3.5%</td></tr><tr><td>G4</td><td>Boundary conditions<br>New constraints at the supports of the vault.</td><td>Simply supported vault</td><td>New lateral constraints on the four supports of the vault.</td></tr><tr><td>G6</td><td><i>E</i> of the blocks</td><td>2.22 GPa</td><td>16.50 GPa</td></tr><tr><td>G7</td><td>Rayleigh coefficient</td><td>6.0%</td><td>4.0%</td></tr><tr><td>G9</td><td>Unknown</td><td>Unknown</td><td>Unknown</td></tr></tbody></table>
<table><tr><td>G10a</td><td>E of the entire vault</td><td>2.22 GPa</td><td>1.45 GPa</td></tr><tr><td rowspan="2">G10b</td><td>E of the entire vault</td><td>2.22 GPa</td><td>1.20 GPa</td></tr><tr><td>Rayleigh coefficient</td><td>2%</td><td>5%</td></tr><tr><td>G12</td><td>E of the entire vault</td><td>2.22 GPa</td><td>1.22 GPa</td></tr><tr><td>G13</td><td>Rayleigh coefficient</td><td>2.0%</td><td>2.5%</td></tr></table>

The change in stiffness of the models can be observed by Figure 22, in which the first mode frequency of the models in the pre-diction and post-diction phase are reported and compared with the experimental one. The plot confirms that in the pre-diction phase most of the groups have modelled a too stiff structure. The only exceptions are **G6** and **G9**, which underestimated the stiffness of the vault. In the case of **G6**, this was due to the use of a discrete model and to the attribution of the overall modulus of elasticity of the masonry to individual blocks. **G4**, due to the type of model used, has not provided the frequencies. In the case of **G9**, the reason is not clear, but it is probable that it is related to the material properties adopted, who where explicitly declared.

<figure><img src="image_50.png" /><figcaption>Figure 22. First mode frequencies of the UNS specimen obtained by the groups in the pre-diction and post-diction phase, compared to the experimental first mode frequency.</figcaption></figure>

## 4.3 Numerical results and discussion

The results are reported in the same format of the pre-diction. In presenting the results, however, the authors preferred to show immediately the plot of the PGA of the SM specimen (Figure 23),
where it can be seen that most of the groups that participated in the post-diction chose to calibrate the model, in addition to the frequency of the first mode, on this very parameter. In fact, five out of nine groups (G4, G10a, G10b, G12, G13) achieved a zero error on the PGA in the post-diction. Anyway, the mean absolute error is 32% with a standard deviation of 43%, only slightly lower than that of the pre-diction phase.

<figure><img src="image_51.png" /><figcaption>Figure 23. Absolute errors in the evaluation of the PGA in the SM specimen in the post-diction phase.</figcaption></figure>

From the analysis of the mean errors on the displacements (Figure 24), it can be observed that **G10a, G10b, G12 and G13** improved the prediction. However, on average, the errors on the displacements are even higher that those of the pre-diction phase (92% for the UNS specimen and 52% for the SM specimen, respectively).
<figure><img src="image_52.png" /><figcaption>Figure 24. Mean errors in the evaluation of the displacements (UNS and SM specimen) in the post-diction phase.</figcaption></figure>

Even if most of the groups have updated the stiffness of the model, the absolute errors on the displacements are still large in most of the case (Figure 25), with larger errors on the x direction.

<figure><img src="image_53.png" /><figcaption>Figure 25. Absolute errors in the evaluation of the displacements in the post-diction-phase for the UNS specimen (a), for the SM specimen (b).</figcaption></figure>

Regarding the accelerations, the prediction improves the results in most of the cases (Figure 26). Also in this case, however, the mean error continues to be rather high for both the UNS and SM specimen (66% and 158% respectively). The absolute errors confirm the trend (Figure 27).
<figure><img src="image_54.png" /><figcaption>Figure 26. Mean errors in the evaluation of the accelerations (UNS and SM specimen) in the post-diction phase.</figcaption></figure>

<figure><img src="image_55.png" /><figcaption>Figure 27. Absolute errors in the evaluation of the accelerations in the post-diction-phase the UNS specimen (a), for the SM specimen (b).</figcaption></figure>

## 5 Conclusions

The assessment of the seismic behaviour of masonry vault is still a challenge. To address this, an extensive experimental campaign was conducted involving a full-scale masonry cross vault tested on a shaking table. The aim was to capture the in-plane dynamic response of the vault shell. Initially, the specimen was tested in an unstrengthened configuration, followed by repair, and retrofitting with grout injections and a textile-reinforced mortar solution. Several experts
on masonry structures presented blind pre-dictions and post-diction on the response of this
specimen and their numerical results were evaluated and compared with respect to the
experimental results obtained from the shaking table tests. Several types of analysis, numerical
modelling and approaches for assessment were adopted by the experts. In general, the
predictions were carried out using limit analysis based on the kinematic approach, Finite and
Discrete Element Methods, and Applied element method.

Good predictions were obtained from the participants in terms of qualitative findings
like the collapse mechanism and the crack pattern. However, when dealing with the quantitative
parameters like the PGA at collapse, errors in the evaluation increase in their entity due to stiffer
models compared with the experimental specimen. FEM models generally provided a more
accurate response compared to other methods, averaging better results in terms of
displacements, accelerations, damage, and PGA at collapse. However, this improvement
sometimes came at the expense of increased computational costs. The prediction having the
least quantitative error was performed by using finite element method, while the collapse
mechanism was better defined based on both FEM and DEM models. On the other hand, the
PGA at collapse for the pre-dictions models presented an average error equal to 56%, with a
minimum error of about 10%. Most predictions showcased a PGA at collapse closely
resembling the respective PGA obtained from the shaking table tests, indicating a strong
adherence of the participants' results to the expected outcomes.

Despite conducting post-diction analyses, during which adjustments were made to
deformability (E values) and damping to enhance the response, particularly focusing on
improving accelerations, the level of errors in the quantitative parameters did not decrease
significantly. This underscores limitations in the strategy and modelling approaches,
particularly when dealing with curved structures. Such structures exhibit varying levels of
compression along the curvature, a characteristic not fully captured by material characterization
596

tests. In conclusion, further efforts are warranted in the study of seismic behaviour of masonry vaults to enhance our understanding of their behaviour. This would enable the development of methodologies that yield results closely aligned with the actual behaviour of masonry vaults.

597

598

599

600

**Acknowledgements.** This work was supported by the SERA.TA project "Seismic Response of Masonry Cross Vaults: Shaking table tests and numerical validations". This work is financed by national funds through FCT - Foundation for Science and Technology, under grant agreement [SFRH/BD/136831/2018] attributed to the XX author.

601

602

603

604

605

**Credit roles.**

606

607

**Declaration of Competing Interest.** The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest, or non-financial interest in the subject matter or materials discussed in this manuscript.

608

609

610

**References**

611

AlShawa, Omar, Domenico Liberatore, and Luigi Sorrentino. 2023. "Combined Finite-Discrete Element Model Simulations of Shake-Table Tests on a Full-Scale Masonry Cross Vault." *International Journal of Architectural Heritage* 00 (under submission): 1-15. https://doi.org/10.1080/15583058.2024.2317455.

612

613

614

615

Bianchini, Nicoletta, Chiara Calderini, Nuno Mendes, Paulo Xavier Candeias, and Paulo B Lourenço. 2023a. "Blind Prediction Competition - Sera.Ta - Seismic Response of Masonry Cross Vaults: Shaking Table Tests and Numerical Validations." *Zenodo*. 2023. https://doi.org/10.5281/zenodo.7624666.

616

617

618

619

—. 2023b. "Postdiction Competition - Sera.Ta - Seismic Response of Masonry Cross Vaults: Shaking Table Tests and Numerical Validations." *Zenodo*. 2023. https://doi.org/doi.org/10.5281/zenodo.7624791.

620

621

622

Bianchini, Nicoletta, Nuno Mendes, Chiara Calderini, Paulo Candeias, and Paulo B Lourenço. 2023. "Shaking Table Testing of an Unstrengthened and Strengthened with Textile Reinforced Mortar (TRM) Full-Scale Masonry Cross Vault." *International Journal of Architectural Heritage* under revi (00): 1-26. https://doi.org/10.1080/15583058.2023.2295900.

623

624

625

626

627

—. 2024. "Shaking Table Testing of an Unstrengthened and Strengthened with Textile
628 Reinforced Mortar ( TRM ) Full-Scale Masonry Cross Vault." *International Journal of Architectural Heritage*.

630 Chácara, César, Bartolomeo Pantò, Francesco Cannizzaro, Davide Rapicavoli, Ivo Caliò, César
631 Chácara, Bartolomeo Pantò, et al. 2023. "Numerical Simulation of the Response of an
632 Unreinforced Brick-Masonry Cross Vault Subjected to Seismic Loading." *International Journal of Architectural Heritage* 00 (00): 1-21.
633 https://doi.org/10.1080/15583058.2023.2290037.
634

635 Dipartimento della Protezione Civile. 2022. “Rete Accelerometrica Nazionale - RAN.” 2022.
636 http://ran.protezionecivile.it/IT/.

637 Gagliardo, R., L. Cascini, and F. P.A. Portioli. 2023. "3D Rigid Block Limit Analysis of a
638 Masonry Cross Vault Subjected to Shaking Table Tests." *International Journal of
639 Architectural Heritage* 00 (00): 1-14. https://doi.org/10.1080/15583058.2023.2287149.

640 Oktiovan, Y, Lucy Davis, R Wilson, Anjali Mehrotra, Bora Pulatsu, and D Malomo. 2023.
641 "Simplified Micro-Modelling of a Masonry Cross Vault for Seismic Assessment Using
642 the Distinct Element Method." *International Journal of Architectural Heritage* 00 (00):
643 1-34. https://doi.org/10.1080/15583058.2023.2277328.

644 Pellegrini, Daniele. 2023. "Pre- and Post-Diction Simulation of the Seismic Response of a
645 Masonry Cross Vault Tested on a Shaking Table." *International Journal of Architectural
646 Heritage* 00 (00): 1-21. https://doi.org/10.1080/15583058.2023.2242812.

647 Pintucchi, B., F. Trovatelli, and N. Zani. 2024. "Modeling Shake-Table Tests of a Masonry
648 Cross Vault Through the MADY Code." *International Journal of Architectural Heritage*
649 00 (00): 1-19. https://doi.org/10.1080/15583058.2024.2305200.

650 Pulatsu, Bora. 2023. "Coupled Elasto-Softening Contact Models in DEM to Predict the in-Plane
651 Response of Masonry Walls." *Computational Particle Mechanics*.
652 https://doi.org/10.1007/s40571-023-00586-x.