# A Quantitative Evaluation Framework for the Benefit of Building Information Modeling for Small and Medium Enterprises Leveraging Risk Management Concepts

Christoph Paul Schimanski, Giada Malacarne, Gabriele Pasetti Monizza,

Dominik T. Matt

► To cite this version:

Christoph Paul Schimanski, Giada Malacarne, Gabriele Pasetti Monizza, Dominik T. Matt. A Quantitative Evaluation Framework for the Benefit of Building Information Modeling for Small and Medium Enterprises Leveraging Risk Management Concepts. 17th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2020, Rapperswil, Switzerland. pp.711-723, 10.1007/978-3-030-62807-9_56. hal-03753130

HAL Id: hal-03753130

https://inria.hal.science/hal-03753130v1

Submitted on 17 Aug 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# A Quantitative Evaluation Framework for the Benefit of Building Information Modeling for Small and Medium Enterprises Leveraging Risk Management Concepts

Christoph Paul Schimanski<sup>1,2[0000-0002-0747-7173]*</sup>, Giada Malacarne<sup>2[0000-0002-5633-6721]</sup>, Gabriele Pasetti Monizza<sup>2[0000-0003-2197-6267]</sup>, Dominik T. Matt<sup>1,2[0000-0002-2365-7529]</sup>

**Abstract.** Building Information Modeling plays an increasingly important role in today's construction sector due to its growing prevalence among practitioners and the increasing legal obligation to use it in public projects. The general and most obvious benefits such as improved communication/coordination or rapid generation of various planning alternatives are widely accepted. However, there is a lack of a generic framework to quantify these benefits. This is especially important from the perspective of small and medium sized enterprises because larger investments - as they may occur through software and training when implementing BIM - should be balanced against a traceable return-on-investment. With a few exceptions, other research focuses on qualitative or non-monetary assessment parameters to evaluate these benefits. Such an approach may be too vague in the context of small and medium-sized enterprises to justify the necessary strategic decisions on a BIM changeover. In this study we propose a generic framework for quantitative evaluation of the BIM benefit. This evaluation is based on the BIM-impact on contingency estimation in construction risk management relying on Monte Carlo simulations. Semi-structured interviews and questionnaires with practitioners are used to evaluate how risk factors and subsequently contingencies can differ from each other in BIM-projects and Non-BIM projects.

**Keywords:** BIM benefits, SMEs, Risk factors, Monte-Carlo simulation

<footnote n="1"><sup>1</sup> Free University of Bozen-Bolzano, Faculty of Science and Technology, Piazza Università 5, 39100 Bolzano, Italy</footnote>

<footnote n="2"><sup>2</sup> Fraunhofer Italia Research, Via A.-Volta 13a, 39100 Bolzano, Italy
christophpaul.schimanski@natec.unibz.it*</footnote>

<footnote>*Corresponding Author</footnote>
# 1 Introduction

## 1.1 BIM and the Assessment of its Benefits

Building Information Modeling (BIM) is a process for creating and managing information throughout the lifecycle of a building or an infrastructure [1]. This process results in a Building Information Model, a digital representation of the built asset, developed collaboratively by designers, construction firms and clients. Collaboration is the key aspect of the BIM methodology since it enhances decision making, increases productivity resulting in a greater whole life value for the asset [2]. For this reason, BIM plays an increasingly important role in today's AEC sector and its adoption is growing faster, year after year. E.g., in the United Kingdom, the number of practitioners using BIM has increased from 13% in 2011 to 69% in 2019 [3]. This exponential trend addresses also the incremental demand for BIM from the client's side and especially from public organizations. In fact, since the introduction of BIM in the European Directive on Public Procurement (24/14/EU) in 2014, several governments across Europe are pushing the mandatory use of BIM in public works with the aim of reducing waste, delays and costs, while increasing the quality of their built environment [4].

Beyond strategic expectations from the AEC community on BIM, its use has already evidenced many benefits also at both project and company level, such as: better communication, coordination and rapid generation of design alternatives [5, 6]. However, even if these benefits are widely accepted, previous research in this field still consists in observations from singular case studies [5] and, as stated by [7], no unified nor global approach has been proposed so far. Since implementing BIM in organizations is a strategic investment, the lack of a framework to quantitatively measure its benefits is critical especially from the perspective of small and medium sized enterprises (SMEs) [6], where investments have to be balanced against a clear return-on-investments (ROI).

Measuring the ROI of BIM has different finalities: (i) understanding the impact on the organization performance; (ii) measuring benefits; (iii) considering the risks associated to the benefits; (iv) collecting data useful for the IT investment analysis [8]. While previous research investigated the relationship between benefits and how these effects ROI, to our best knowledge there are no scientific studies that are concerned with the relationship between potential risk factors in a construction project and the mitigation improvements achievable using BIM. This interplay might affect ROI considerations when thinking of BIM implementations.

## 1.2 Problem Statement

As being outlined above the benefit of the BIM approach for implementing companies – especially for SMEs – is not reliably measurable in a direct way. One reason for this is that there is little long-term experience with the use of BIM, and it is not sufficiently widespread within the construction industry. Secondly, the BIM impact on the performance of a construction project is difficult to relate to a reference or benchmark of a traditional approach. This is due to the unique characteristics of buildings: Two
identical buildings with identical boundary conditions (same subsoil conditions, neighboring buildings, weather, team of project participants, contractor and subcontractors, etc.) would have to be built in order to compare the two approaches. However, even in this case, no direct comparability would be guaranteed, as the project team would probably learn from the first realization, and the second one would therefore run better and faster in any case, which would dilute the assessment of the BIM influence.

Therefore, we follow an indirect way to potentially assess the impact of BIM by considering different BIM functionalities and their impact on risk management in construction projects.

More precisely, we use methods of probabilistic risk management. In the field of probabilistic risk management there are numerous known risk factors (such as poor communication and coordination. Their influence on the monetary contingency to be made available in case of risk occurrence is sufficiently described in the literature. However, what is not described in the literature is the potential influence of BIM on these risk factors. This paper aims at contributing to fill this gap.

We further hypothesize that several BIM functionalities have a positive influence on certain risk factors, both in terms of the likelihood of its occurrence and financial impact. Consequently, the contingency to be held available would be reduced compared to non-BIM projects. Using this indirect way, we attempt to describe the BIM benefit quantitatively. Accordingly, this paper investigates two aspects of research, namely (i) the quantitative evaluation of the BIM approach and (ii) the BIM influence on known risk factors in construction.

## 2 State of the Art: Risk Management in Construction

According to the PMBOK® Guide [9], risk management plays a crucial role in project management and is defined therein as “systematic process of identifying, analyzing, and responding to project risks”.

The extent of a risk is determined by two factors: The "likelihood of occurrence" and the "impact" of a risk. Both determinants have different probabilities. The first specifies the likelihood of the risk occurring, while the second quantifies the probable magnitude of the financial impact in case of occurrence [10]. Two approaches are used here: Qualitative and quantitative risk assessment strategies.

Within the existing frameworks for qualitative risk assessment, the respective risk impact and the likelihood of occurrence are classified merely trough few levels (high, medium, low) and within this classification, percentage supplements in terms of budget are added on the estimated production costs [11].

Quantitative risk assessment on the other hand requires a more detailed consideration from an economic and financial perspective. The individual risks are assigned a potential impact range in monetary units accompanied by probability distributions as well as binomial distributed likelihoods of occurrence. A resulting quantitative measure of the overall risk impact of a project is the *value at risk* as a loss-oriented risk measure.
This value is estimated with the aid of numerical simulation methods such as the Monte Carlo simulation (MCS) method.

The MCS method is used as a computer-supported, mathematical instrument for solving complex tasks. In general, the Monte Carlo method is understood as the use of random numbers for the solution of stochastically describable problems, even if these problems are not stochastic in their origin [12, 13].

It is possible to distinguish two fields in which MCS is typically used: The first field contains deterministic problems, where also exact analytical solutions exist. MCS is used here when simulation results, as an approximate solution, can describe the actual solution with enough accuracy. The loss of accuracy that is associated with MCS is consciously accepted and part of a trade-off with simplicity and velocity within the process of finding a solution. This trade-off yields frequently to the selection for MCS, as analytical solutions, in these fields, are often very computationally and resource intensive. This use of MCS has a high significance within the natural sciences and mathematics. Application examples for deterministic problems that can be solved with MCS are the calculation of integrals and the solution of ordinary and partial differential equations [13].

The second problem area comprises stochastic problems. These are characterized by the fact that the input parameters and the resulting target variables are to a certain extent subject to chance. In stochastic problems, MCS can be used to investigate situations where either no analytical solution exists or input parameters comprise random variables [14]. In the area of risk analysis, MCS can be used to aggregate individual risks to form an overall risk, multi-scenario simulations as well as for decision optimization.

# 3 Research Strategy

The research strategy followed in this paper is a combined approach of (i) elaborating a probabilistic risk model containing risk factors in accordance with [15]. The calibration of the risk model is performed by analyzing literature, results from a questionnaire as well as findings of semi-structured interviews with construction experts. Secondly (ii), the research strategy consists of performing the MCS to assess the impact of BIM on contingency estimation in construction from the risk management perspective. By doing so, we aim to measure the added value of BIM in an indirect way. While the use of MCS for contingency estimation in construction is widely described in research [15–17], the analysis of the BIM influence within such a probabilistic system is new to our best knowledge.

This evaluation is based on the framework for benefit evaluation of Target Value Design (TVD) on construction costs by [18]. TVD is a method from Lean Construction management to create more value for the client. One dimension of value in Lean Construction consists in staying on budget. In the mentioned evaluation framework, a theoretical cost-breakdown model was postulated, which divides the individual cost components for construction projects into cost of work, contingency and profit.

Their findings on the statistical analysis of TVD and non-TVD projects impose a reduction of contingency in TVD projects. The authors denominate the drivers for
contingency as “forces” that drive up total construction costs. These forces are identi-
fied by the authors as e.g. poor communication and coordination, lack of trust, change
orders etc. They argued further that in TVD projects, these negative drivers can be re-
duced and hence the required contingency is likely to be reduced, too.

Drawing a line back to this research, these mentioned “forces” can be also consid-
ered risk factors in construction. The implementation of BIM might affect some of these
risk factor positively. However, it remains unclear to which extent BIM affects these
risk factors, and what effects this has on contingency considerations.

Therefore, the proposed framework for quantitative BIM benefit evaluation is based
on a probabilistic risk model to determine the effect of BIM on contingency estimation.
The underlying assumption, that BIM can actually contribute to contingency reduction
through the mitigation of risk factors is shown in **Fig. 1** as adaptation of TVD projects
by [18].

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Potential contingency reduction through BIM, graphic adapted from [18]</figcaption></figure>

For the purpose of BIM benefit assessment, we attempt to determine a so-called BIM-
based contingency delta (BIM-C-Δ), which might be caused by certain risk factors.
Thus, the BIM-C-Δ does not represent the potential overall BIM benefit in that regard,
since we are not going to consider all factors that might have an impact on contingency
estimation, but only the top risk factors which might be affected through the use of
BIM. This means, the potential overall BIM-benefit could be even higher if all risk
factors were examined.

Consequently, this study aims to examine the following hypotheses:

1. Null-Hypothesis $H_0: \mu_d \le 0$

The implementation of BIM has no positive impact on the required contin-
gency in cost estimation.

2. Alternative-Hypothesis $H_A: \mu_d > 0$

The implementation of BIM has a positive impact on the required contin-
gency in cost estimation.
For assessing these hypotheses, we elaborate a risk register that contains a total five risks factors which might be affected by BIM. The elaboration of this risk register is described in detail in section 4.1. The risks contained therein are transferred into a questionnaire that is handed out to selected local construction experts to evaluate the extent to which the likelihood of occurrence varies under BIM application. In this study, we analyze a total of 22 responses to this questionnaire.

To test the presented hypotheses, we conducted a paired t-test on the questionnaire results. The test results are discussed in section 4.2.

Presuming a statistically significant difference of risk perception in the two scenarios (rejection of $H_0$), we apply MCS to simulate the needed contingency for a defined project scenario (residential construction in the order of magnitude up to 10 Mio.€ contract volume), taking into account the questionnaire's results for configuration of the probabilistic risk model in terms of likelihoods, impacts and dispersion.

# 4 Proposal for Quantitative BIM Benefit Evaluation

## 4.1 Elaborating the Risk Register

The risk register used in this study is based on the overall top 10 risks identified in a literature review of the most common risks in construction [19]. These are the risks that occur most frequently according to that study. We contrast these risks with the 10 risks that are most significant according to [20]. The assessment criteria for significance here are the likelihood of occurrence and the potential financial risk impact in case of occurrence.

We have made an intersection of these two lists and we have neglected all those risks which are not clearly influenced by BIM in terms of likelihood of occurrence with regard to the mentioned BIM benefits in section 1.1. We have given the five remaining risks an aggregate name and assigned them to risk categories in line with [17]. Additionally, we ranked them according to their relative likelihood of occurrence and financial impact as reported by [21]. The resulting list is shown in **Table 1** and forms the basis for the probabilistic risk model in this study.

<table><caption>**Table 1.** Considered risk factors</caption><thead><tr><th>Risk factor</th><th>Category according to [17]</th><th>Risk ID</th><th>Rel. likelihood rank</th><th>Rel. impact rank</th></tr></thead><tbody><tr><td>Slow decision-making process</td><td>Managerial</td><td>M1</td><td>3</td><td>2</td></tr><tr><td>Frequent change orders by client</td><td>Technical</td><td>T1</td><td>2</td><td>4</td></tr><tr><td>Errors and omissions in design drawings</td><td>Technical</td><td>T2</td><td>1</td><td>1</td></tr></tbody></table>
<table><tr><td>Unavailability or shortage in specified material</td><td>Resource-related</td><td>R1</td><td>4</td><td>2</td></tr><tr><td>Delay in response to requests for information (generalized)</td><td>Managerial</td><td>M2</td><td>5</td><td>5</td></tr></table>

## 4.2 Expert Questionnaire & Hypotheses Testing

We elaborated a questionnaire for the configuration of the probabilistic risk model. The questionnaire goals are two-fold: (i) to evaluate how the likelihood of occurrence of the risk factors changes under BIM usage from the experts' point of view for the purpose of hypotheses testing, and (ii) to find out which dispersion parameters are appropriate for the impact distribution for MCS in the case the Null-Hypothesis has to be rejected. A total of 22 responses have been analyzed when this study was conducted.

As an example, out of these risk factors, the first risk factor (M1) is concerned with whether a project can be delayed due to decisions not taken or taken too late because the information needed was not (sufficiently) available. The experts were asked to express their opinion on the likelihood of the occurrence of this risk factor on a 4-point Likert-scale with the options (i) very likely; (ii) likely; (iii) unlikely and (iv) very unlikely in the two scenarios of traditional and BIM-based project delivery. By BIM project delivery, we mean a full use of all available functionalities (e.g. collision check, common data environment, automatic plan derivations, model-based quantity take-off etc.) by all project participants. The results are presented in Fig. 2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Risk factor M1</figcaption></figure>

These results show a change in the perception of risk occurrence from *very likely* to *unlikely*. In fact, the number of votes for *very likely* has been halved, which in turn increased the votes of an *unlikely* risk event in the BIM-based case. The number of votes for *likely* have approximately remained the same.

Furthermore, the respondents were asked to estimate the potential financial impact of the considered risk factors. For the risk factor M1 it has been estimated in the range from *significant* (68%) to *critical* (23%) with a medium degree of statistical dispersion, meaning that the financial impact of this risk factor is subject to uncertainties (not shown in the graphic).
The findings for the other four considered risk factors are of similar appearance, so that a detailed description of those is omitted at this point.

Subsequently, and in accordance with [22], who studied probability related terms and their corresponding values as percentages, the different options of the Likert-scale of the questionnaire have been assigned the following percentages as a numerical likelihood of occurrence as an assumption: (i) *Very likely* = 65%; (ii) *Likely* = 50%; (iii) *Unlikely* = 7,5%; (iv) *Very Unlikely* = 2,5%.

As a next step, we calculated the mean of the responses by creating ordinal data through assignment of numbers from 1 to 4 to the answer options (1 = *Very unlikely*; 2 = *Unlikely*; 3 = *Likely*; 4 = *Very unlikely*). The mean here is used to find out general tendency of the responses with respect to the numerical likelihood as percentages and eventually for hypotheses testing, making use of a paired t-test. The question to be analyzed with this paired t-test is whether BIM does impact the required project budget positively or not. Hence, the difference of the mean in both cases is in question, hypothesizing that $\mu_d \le 0$ (= $H_0$).

The paired t-test with an α-level of 0,1 and under t-distribution with 4 degrees of freedom has shown that the null hypothesis $H_0$ is to be rejected and the alternative hypothesis is to be accepted, which means that BIM has a statistically significant (positive) effect on the risk occurrence according to the questionnaire findings given the considered risk factors. It remains to be analyzed whether this discrepancy in perception on the effect on construction risks between traditional and BIM-based projects is also measurable trough numerical simulation of risk events.

The next section presents the probabilistic risk analysis framework to quantify this effect in terms of required contingency of a given project budget. In this study, the project budget is assumed to be 10 million € (project scenario: residential construction). Simulation results do not claim to be reliable or valid in terms of absolute numbers but are intended to show the relative difference between the two project delivery approaches (traditional vs. BIM-based) and to present this idea of measuring the BIM benefit in an indirect way to the scientific community.

## 4.3 Probabilistic Risk Model

This list of risk factors given in **Table 1** is the fundamental element of the probabilistic risk model for performing the MCS for contingency estimation. **Fig. 3** shows the risk model in the BIM-based configuration exemplarily.
<table><caption>Fig. 3. Risk model BIM-based project delivery</caption><thead><tr><th colspan="11">Risk Register</th><th colspan="2">Total project budget [€]</th></tr><tr><th colspan="4">Risk factor</th><th colspan="7">Financial impact</th><th colspan="2">€10.000.000.00</th></tr><tr><th rowspan="2">Risk_ID</th><th rowspan="2">Risk Event</th><th rowspan="2">Description</th><th rowspan="2">Likelihood with BIM [%]</th><th rowspan="2">Simulated occurrence [95%]</th><th colspan="7">Distribution form parameters</th><th rowspan="2">Strong Correlation to Others(x)</th><th rowspan="2">Resulting contingency allowance [€]</th></tr><tr><th>Impact</th><th>Rough Order Impact (k)</th><th>Impact probability distribution function</th><th>Mean Impact [€]</th><th>Standard Deviation (genuin/ungens u's)</th><th>Simulated Impact [€]</th><th>Strong Correlation to Others(x)</th></tr></thead><tbody><tr><td>M0</td><td>Slow decision making process</td><td>The project is delayed due to decisions not taken or taken too late because the information needed was not (or late) available</td><td>47.24%</td><td>0</td><td>Significant</td><td>€500.000.00</td><td>Log-normal</td><td>€500.000.00</td><td>€150.000.00</td><td>€701.485.24</td><td>M2</td><td>€0.00</td></tr><tr><td>T1</td><td>Future change orders by client</td><td>Preventive requests for changes from the client requires a traveling visit of all planning documents, and a request to submit a detailed list of all changes to the client during an appointment</td><td>47.97%</td><td>0</td><td>Significant</td><td>€500.000.00</td><td>Log-normal</td><td>€500.000.00</td><td>€150.000.00</td><td>€678.961.21</td><td>T2</td><td>€0.00</td></tr><tr><td>T2</td><td>Study and emissions in design drawings</td><td>Defective planning or collisions that are not detected or are detected but late lead to delays</td><td>43.60%</td><td>1</td><td>Significant</td><td>€500.000.00</td><td>Log-normal</td><td>€500.000.00</td><td>€150.000.00</td><td>€626.236.23</td><td>T1</td><td>€626.236.23</td></tr><tr><td>R1</td><td>Unwieldy or advantage in specified material</td><td>Unreliable quantity take-offs from planning results in material shortages, and thus delays on the construction site</td><td>38.30%</td><td>1</td><td>Critical</td><td>€1.000.000.00</td><td>Log-normal</td><td>€1.000.000.00</td><td>€300.000.00</td><td>€790.923.46</td><td>-</td><td>€790.923.46</td></tr><tr><td>M5</td><td>Delay in response to requests for information</td><td>The response to requested planning deliverables or detailed specifications is not done as scheduled and thus causes "Stallarations" in the planning or execution process and thus delays</td><td>42.84%</td><td>1</td><td>Critical</td><td>€1.000.000.00</td><td>Log-normal</td><td>€1.000.000.00</td><td>€300.000.00</td><td>€763.583.85</td><td>M1</td><td>€763.583.85</td></tr></tbody><tfoot><tr><td colspan="11" style="text-align: right;">SUM Contingency</td><td>€2.193.713.52</td></tr></tfoot></table>

It is conceptually derived from the risk register presented by [23] and composed of (i) the risk factors themselves and a short description; (ii) a quantitative likelihood of occurrence as derived from the questionnaire results; (iii) a qualitative impact in the range from negligible to crisis; (iv) a rough financial order of impact in terms of euros [€] as agreed with local construction experts; (v) the type of probability distribution for that impact and form parameters for stochastic analyses (mean and standard deviation) as well as (vi) the presumed correlation of the single risk factors.

In both cases, traditional and BIM-based project delivery, we assume a lognormal distribution for the probability density function (PDF), since only mathematically positive values for the financial impact are allowed (even though risk occurrence would create a negative impact on the project budget). Moreover, this selection is in line with other research on cost estimation and risk management in construction which argues that lognormal distributions fit construction cost components generally the best [17].

Regarding the estimation of form parameters of the PDF, we abstracted the screened literature and findings of semi-structured interviews with representatives of a total of five enterprises from the South Tyrolean construction sector. These enterprises comprise SMEs from the fields of timber construction, civil engineering, shell construction and HVAC. Their representatives were asked to roughly estimate the given risk factor's impact in case of occurrence. Together with literature indications [21] as well as the questionnaire results, we configured the risk model for both cases, traditional project delivery approach and BIM-based project delivery approach. The likelihood of occurrence was modelled as binomial distribution with form parameters according to the percentage values according to [22].

In construction cost estimation, simulation-based analysis is highly affected by potential correlations of single cost factors [24]. The cited author demonstrates a slight to medium positive correlation of cost components which was confirmed in expert discussions for the considered risk factors and incorporated accordingly in the probabilistic risk model.
# 5 Preliminary Simulation Results

The Monte Carlo simulation was run 10000 times in both cases. Fig. 4 shows the simulation results in a histogram. It can be seen that the maximum (2,56% discrepancy) and the standard deviation (1,12% discrepancy) differ only slightly from each other, while the mean (μ) and median (X̄) show a relatively large discrepancy: In the BIM-based case, the mean is more than 20% below the traditional project delivery. The BIM-based median is even 28,21% lower than in the traditional case.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Histogram of simulation results</figcaption></figure>

In both cases the distributions can be considered as approximately symmetric. Only a slight skewness to the right in the BIM-based distribution can be noted (0,39). Given the kurtoses of 2,12 (traditional) and 2.24 (BIM-based), both resulting distributions can be considered as platykurtic and thus having relatively low and broad central peaks compared with the normal distribution. The mode corresponds in both cases to 0€. However, in the BIM-based case the value of 0€ for the required contingency occurs almost twice as often as in traditional project delivery.

# 6 Inferences from Results & Discussion

The simulation results for the selected configuration of the risk model and the resulting descriptive statistics allow the following statements: There are relatively large differences in the mean (μ) and median (X̄) between traditional and BIM-based project delivery (21,52% and 28,21%, respectively), while the minima and maxima, as well as the standard deviation σₓ, are relatively close to each other. The standard deviation itself appears as approx. 60% respectively 76% of the mean. Within two standard deviations there are more than 95% of all simulated values, and within three standard deviations there are more than 99,7% of all simulated values which is an indication for normal distribution. A high standard deviation indicates many outliers, which means that the median is the more appropriate single value to describe the whole sample with only one
value. From this background, the median difference of almost 30% indicates a signifi-
cantly positive effect of the BIM approach on contingency estimation.

However, these results should only be interpreted as a general tendency, since on the
one hand the modeling and simulation results are associated with some limitations (see
section 7) and on the other hand the results from the mere histogram analysis with the
usual confidence percentile of, for example, P90, only shows relatively small differ-
ences. Since these high confidence intervals are usually the ones on the basis of which
strategic decisions are made (such as investment decisions), it remains to be checked
whether the proven relative differences between traditional and BIM-based project de-
livery are sufficient to justify such - especially for SMEs drastic - changes as a complete
BIM changeover.

7 Limitations, Conclusion & Outlook

This study has some worth mentioning limitations: only few risk factors have been
considered in the probabilistic risk model, thus, the underlying risk register should be
expanded in further studies. Furthermore, the assumed financial impact is only a rough
estimation by the construction experts surveyed with reference to a hypothetical project
scenario. These impacts would have to be determined for a larger number of different
project scenarios and in a more robust manner - for example, by evaluating historical
project data - which is out of scope of this study, since the focus is on presenting the
benefit evaluation concept itself.

Even if this were to be done in the future, it could still be argued with the criticism
that every project in the construction industry is different and it is therefore difficult to
make general statements about the BIM impact. This criticism would still be justified -
as explained in the section on related work in other examples - but in our view, the
Monte Carlo simulation approach helps to refute this counter-argument, because this
type of numerical simulation aims to play through as many random scenarios as possi-
ble in order to derive general inferences.

With respect to the chosen probability distributions there is also uncertainty, espe-
cially since the choice of the distribution has a considerable influence on the simulation
results as explained by [25]. However, the presented probabilistic risk model aims at
providing reasonable assumptions for the final choice of the log-normal distributions,
which are explained in section 4.1. In addition, this benefit assessment framework aims
at providing an example of the relative difference in contingency requirements compar-
ing a traditional and BIM-based project delivery approach, which we believe is possible
even if the absolute numbers are not sufficiently reliable.

In the light of future questionnaire extensions, this study represents an invitation to
the research community to evaluate the influence of BIM on risk factors in a supra-
regional context in order to obtain a more complete picture of the appropriateness of
the suggested indirect evaluation approach.

To conclude the results of this study, the initially stated research objectives of (i)
formulation of a quantitative BIM-benefit evaluation framework and (ii) analysis of the
BIM influence on known risk factors in construction have been both addressed and validated respectively through the simulation and questionnaire results.

Given the reference to TVD in this study, it may also be useful to examine projects where both BIM and TVD have been used for the combined impact on contingency determination. This could lead to a contribution to the body of knowledge in the area of practical BIM benefits, as well as new BIM-Lean synergies that have not yet been explored.

Future considerations using this BIM benefit evaluation framework should be also put in relation to the incurred costs, which require a changeover to BIM-based project delivery. In the area of BIM implementation, these costs mainly concern the investment in software, training, process changeover and personnel. Only in this overall context, quantitative BIM evaluation can provide reliable information about strategic management investment decisions of SMEs.

**Acknowledgements.** This work is part of BIM Simulation Lab – FESR 1086, a research project financed by the European Development Fund (ERDF) Südtirol/Alto Adige.
References

1. NBS (2019) What is building information modelling. https://www.thenbs.com/knowledge/what-is-building-information-modelling-bim. Accessed 9 Mar 2020

2. Eastman CM, Teicholz P, Sacks R, Liston K (2008) BIM Handbook: A guide to Building Information Modeling for owners, managers, designers, engineers and contractors, 1st ed. Wiley, Hoboken, N.J

3. NBS (2019) National BIM Report 2019. https://www.thenbs.com/knowledge/national-bim-report-2019. Accessed 9 Mar 2020

4. Government H (2013) Construction 2025. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/a ttachment_data/file/210099/bis-13-955-construction-2025-industrial- strategy.pdf. Accessed 9 Mar 2020

5. Azhar S (2011) Building information modeling (BIM): Trends, benefits, risks, and challenges for the AEC industry. Leadersh Manag Eng 11:241–252. https://doi.org/10.1061/(ASCE)LM.1943-5630.0000127

6. Ghaffarianhoseini A, Tookey J, Ghaffarianhoseini A, et al (2017) Building Information Modelling (BIM) uptake: Clear benefits, understanding its implementation, risks and challenges. Renew Sustain Energy Rev 75:1046–1053. https://doi.org/10.1016/j.rser.2016.11.083

7. Guerriero A, Kubicki S, Reiter S (2016) Building information modeling in use: How to evaluate the return on investment? In: eWork and eBusiness in Architecture, Engineering and Construction - Proceedings of the 11th European Conference on Product and Process Modelling, ECPPM 2016. pp 537–544

8. NRC (2012) NRC Publications Archive (NPARC) Archives des publications du CNRC (NPARC)

9. Snyder CS (2014) A guide to the project management body of knowledge: PMBOK (®) guide

10. Girmscheid G, Busch TA (2008) Unternehmensrisikomanagement in der Bauwirtschaft (in German). Bauwerk Verlag

11. Haid N, Schneidewind A (2010) Risikomanagement in Bauunternehmen: Risiken beherrschen, nicht ertragen (in German). Straßen- und Tiefbau 7:5–9

12. Johnson PE (2011) Monte Carlo Analysis in Academic Research. In: Little TD (ed) The Oxford Handbook of Quantitative Methods in Psychology, Vol. 1. Oxford University Press, pp 1–67

13. Lemieux C (2009) Monte Carlo and Quasi-Monte Carlo Sampling, 1st ed. Springer, New York

14. Rider G, Milkovich S, Stool D, et al (2000) Quantitative Risk Analysis. Inj Control Saf Promot 7:115–133. https://doi.org/10.1002/9780470924860.ch5

15. Allahi F, Cassettari L, Mosca M (2017) Stochastic risk analysis and cost contingency allocation approach for construction projects applying Monte Carlo simulation. Lect Notes Eng Comput Sci 2229:385–391

16. Medgenberg J, Nemuth T (2011) Potential der Monte-Carlo-Simulation für
Risikoanalyse im Projektmanagement (in German). In: 1. Internationaler BBB-Kongress. pp 153–171

17. Peleskei CA, Dorca V, Munteanu RA, Munteanu R (2015) Risk Consideration and Cost Estimation in Construction Projects Using Monte Carlo Simulation. Manag 10:163–176

18. Do D, Chen C, Ballard G, Tommelein ID (2014) Target Value Design as a Method for Controlling Project Cost Overruns. 22nd Annu Conf Int Gr Lean Constr 1:171–181

19. Siraj NB, Fayek AR (2019) Risk Identification and Common Risks in Construction: Literature Review and Content Analysis. J Constr Eng Manag 145: https://doi.org/10.1061/(ASCE)CO.1943-7862.0001685

20. Jarkas AM, Haupt TC (2015) Major construction risk factors considered by general contractors in Qatar. J Eng Des Technol 13:165–194. https://doi.org/10.1108/MBE-09-2016-0047

21. Chileshe N, Boadua Yirenkyi-Fianko A (2012) An evaluation of risk factors impacting construction projects in Ghana. J Eng Des Technol 10:306–329. https://doi.org/10.1108/17260531211274693

22. Hillson DA (2005) Describing probability : The limitations of natural language Dimensions of risk. Risk Manag 1–7

23. Dziadosz A, Rejment M (2015) Risk Analysis in Construction Project - Chosen Methods. Procedia Eng 122:258–265. https://doi.org/10.1016/j.proeng.2015.10.034

24. Yang IT (2005) Simulation-based estimation for correlated cost elements. Int J Proj Manag 23:275–282. https://doi.org/10.1016/j.ijproman.2004.12.002

25. Arnold U, Yildiz Ö (2015) Economic risk analysis of decentralized renewable energy infrastructures - A Monte Carlo Simulation approach. Renew Energy 77:227–239. https://doi.org/10.1016/j.renene.2014.11.059