# Evaluating the impact of public policies supporting digital transformation in the construction sector: towards a systematic evaluation framework

Victoria Lerognon, Elodie Hochscheid, Gilles Halin, Erik Andrew Poirier

► To cite this version:

Victoria Lerognon, Elodie Hochscheid, Gilles Halin, Erik Andrew Poirier. Evaluating the impact of public policies supporting digital transformation in the construction sector: towards a systematic evaluation framework. Proceedings of Smart and Sustainable Built Environment 2025, Lille, France (SASBE2025), Nov 2025, Lille, France. hal-05446361

HAL Id: hal-05446361

https://hal.science/hal-05446361v1

Submitted on 7 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
SASBE 2025 aims to encourage the
international exchange of innovative
ideas between researchers from
academia and industry. In addition to
knowledge dissemination, the
conference offers a valuable platform
for professional networking,
particularly benefiting university
professors, graduate students, and
postdoctoral researchers.

Research Article

# Evaluating the impact of public policies supporting digital transformation in the construction sector: towards a systematic evaluation framework

Victoria Lerognon<fnref n="1" />,<fnref n="2" />,<fnref n="3" />, Elodie Hochscheid<fnref n="2" />,<fnref n="5" />, Gilles Halin<fnref n="2" />,<fnref n="6" />, Erik Poirier<fnref n="1" />,<fnref n="4" />,

Copyright: Copyright: © 2025 by the
authors.

SASBE is an open-access
proceedings distributed under the
terms of the Creative Commons
Attribution 4.0 International License
(CC BY 4.0).

View this license's legal deed at
https://creativecommons.org/licenses/by/4.0/

<figure><img src="image_2.png" /></figure>

## Abstract (250 words)

Digital transformation in the public construction sector has given rise to major public policy
worldwide initiatives, such as France's Digital Transition Plan for the Construction Industry
(PTNB), the BIM Plan 2022, and Quebec's Roadmap for built asset information modeling.
These initiatives reflect a sustained political commitment to fostering digital transformation,
though engagement levels vary significantly across regions. Yet their actual impacts remain
largely under-evaluated, partly due to the novelty of these programs and the fact that only
now—at a moment when Building Information Modeling (BIM) practices are maturing—do
we have sufficient hindsight and data for meaningful assessment.

This article is part of a broader research project aimed at developing robust approaches for
evaluating public policies supporting digital transformation in construction. The present
study focuses specifically on the methods available for assessing policy impacts and seeks
to contribute to a better understanding of how to evaluate these transitions effectively.

The paper begins by reviewing the main typologies of public policies associated with digital
transformation, in order to clarify the diversity of policy instruments and intervention logics
in this domain. It then proposes a critical analysis of existing policy evaluation methods—
ranging from cost-benefit analysis to theory-based and mixed-method approaches—and
assesses their relevance to the construction sector. Drawing on these insights, the paper
proposes a methodological framework designed to guide future evaluations, while
accounting for the specific challenges of the sector, such as fragmentation, long project
cycles, and the hybrid nature of public-private partnerships.

**Keywords:** public policy, BIM, digital transformation, construction sector, impact evaluation

## Highlights

* Digital transformation needs multi-method, mixed-evidence evaluation.

* A choice matrix links questions, contexts, and method bundles for construction's digital shift.

* Sequencing methods across the policy cycle captures short- and long-term digital effects.

<footnote n="1">¹Groupe de recherche en intégration et développement durable en environnement bâti (GRIDD), Construction Engineering Department, École de technologie supérieure, Montréal, QC, Canada</footnote>

<footnote n="2">²Unité de recherche multisite Modèles pour l'Architecture et le Patrimoine (URM MAP research lab), Centre de recherche en architecture et ingénierie (CRAI), École Nationale Supérieure d'Architecture de Nancy, France</footnote>

<footnote n="3">³victoria.lerognon.1@ens.etsmtl.ca</footnote>

<footnote n="4">⁴erik.poirier@etsmtl.ca</footnote>

<footnote n="5">⁵e.hochscheid@nancy.archi.fr</footnote>

<footnote n="6">⁶gilles.halin@univ-lorraine.fr</footnote>
# 1 Introduction

Digital transformation is a major lever for modernizing public action, particularly in construction—a sector historically under-digitised and characterised by significant economic and environmental stakes (Bernard, Lichère, & Micalef, 2024). In France and Québec alike, the digitalisation of public procurement in construction relies on instruments such as Building Information Modeling (BIM), tendering/e-procurement, and digital information-sharing platforms.

These orientations reflect the implementation of public policies in the sense of Lascoumes & Le Galès, (2018) : coherent sets of instruments and decisions directed at solving collective problems. Following Boussaguet, Jacquot, & Ravinet, (2010), these policy instruments—legal, incentive-based or informational—are tools mobilised by the state to steer behaviour and achieve collective goals. They embody a particular mode of governing—a “political technology” shaping behaviours and relations among actors—and express a specific problem framing: the performance and productivity deficit in construction.

While ambitions are high, the actual effects of these policies remain under-documented. National initiatives target tool adoption, productivity and transparency, yet evaluation remains limited. Because digital transformation is a wicked problem as defined by Fountain (2019), outcomes are multi-factorial and difficult to attribute to a single cause. Evaluation therefore cannot be reduced to deployment tracking; it must combine causal attribution, mechanism analysis, and contextualisation.

Two dimensions are especially salient for framing evaluation:

(i) the institutional embedding of the sector (legal frameworks for public procurement, multi-level governance, administrative capacity), which conditions implementation pathways;

(ii) policy layering, whereby new measures rarely arrive “in a vacuum”: they stack on top of prior instruments, generate interaction effects and inherit trajectories (path dependence). Many evaluation designs assume a simple counterfactual or a clear baseline—assumptions often unrealistic here.

In the digital transformation of the construction industry, evaluation is crucial to move beyond intent and measure actual effects on sector modernisation, technology adoption, reconfiguration of professional practices, improved project performance, stakeholder satisfaction, and public value creation. It also faces sector-specific challenges (C1 to C5) :

* C1 – Rapid technological change: the multiplicity and pace of innovations hinder the definition of stable standards and the measurement of effects over time (Hassan, Negash, & Hanum, 2024).

* C2 – Diversity of policy instruments: instruments vary widely (financial incentives, standards, training, governance strategies), complicating comparative impact assessment (Xia, Liu, & Wang, 2025).

* C3 – Sectoral and territorial heterogeneity: policies do not produce the same effects across countries, regions or segments of the construction industry (Rinchen, Banihashemi, & Alkilani, 2024).

* C4 – Organisational and cultural change: deep shifts that do not immediately translate into measurable gains in productivity or quality (Gabuthy, Jacquemet, & L'Haridon, 2021).
* C5 – Sector fragmentation and multiple actors: clients/owners, firms, engineering practices, users and public authorities pursue different logics, complicating data collection and impact attribution (Alsofiani, 2024).

It also encounters methodological issues (I1 to I5) (Naji, Gunduz, Alhenzab, Al-Hababi, & Al-Qahtani, 2024; Samuelson & Stehn, 2023):

* I1 – Attribution of effects: distinguishing the policy's own effect from market trends, private initiatives, and international dynamics.

* I2 – Result measurement: selecting indicators for digital transformation (technology adoption, productivity gains, asset quality, innovation, satisfaction).

* I3 – Data collection: accessing reliable, comparable and up-to-date data in a fragmented and only partly digitised sector.

* I4 – Delayed effects: measuring medium- and long-term impacts, as digital transformation may take years to materialise.

* I5 – Qualitative dimensions: accounting for organisational, cultural and behavioural changes that condition policy success.

In light of these constraints, appropriate evaluation methods are required—able to capture transformation complexity, identify causal mechanisms, measure direct and indirect effects, and account for contextual diversity and stakeholder interactions (Bozio, 2018).

This article offers an analytical review of evaluation methods applicable to these policies, mapping them against sectoral challenges C1–C5 and methodological issues I1–I5. The analysis focuses on France and Québec, whose administrative and legal trajectories differ; this choice stems from the project's institutional set-up and prepares subsequent research actions. The objectives are to produce a rigorous mapping of available approaches and to identify effective method combinations to build, in the next phase, an evaluation framework tailored to the digital transformation of the construction sector.

# 2 Methodology

This study is based on a literature review designed to build a framework of evaluation methods applicable to public policies for digital transformation, with a specific focus on the construction sector. The review follows an analytical and classificatory approach intended to support the subsequent development of an operational assessment framework.

## 2.1 Step 1 - Research objectives

The primary objective of this first review is to identify, structure, and analyse existing methodological approaches to public policy evaluation in the context of digital transformation. Accordingly, the review addresses the following questions:

* Which methods are most commonly used to evaluate public policies for digital transformation in the construction sector?

* To what extent do these methods address the sectoral challenges identified (C1–C5)?

* To what extent do they account for the methodological issues (I1–I5)?
* Under what conditions are some methods more appropriate than others (given evaluation objectives, level of intervention, available resources, or policy type)?

In this article, policy evaluation is understood broadly, encompassing both impact-oriented approaches—which seek to isolate the causal effect of a policy—and comprehensive evaluation approaches that address instrument design, implementation processes, and qualitative effects. This distinction matters: while impact methods offer strong internal validity, they do not, on their own, capture the complexity of digital transformation policies, which involve multiple actors and organisational dimensions.

## 2.2 Step 2 – Literature corpus selection

The corpus was assembled through an exploratory bibliographic search using academic databases (Google Scholar, Cairn.info, JSTOR, Scopus) and institutional reports (OECD, public bodies, sectoral think tanks). French- and English-language keywords combined policy evaluation, digital transformation, construction sector, public sector innovation, impact assessment, BIM evaluation, e-government. Documents were retained if they met three criteria: direct relevance to the research topic; explicit description of the evaluation methods employed; and methodological contemporaneity (publications after 2000).

## 2.3 Step 3 – Analytical framework

The selected corpus was analysed using a multidimensional grid, assessing the ability of each method to address sectoral challenges (C1–C5) and methodological issues (I1–I5) set out in the introduction. This mapping enables a comparative assessment of available approaches, taking into account their contributions, limitations, and conditions of application. The study proposes a structured analytical framework. It allows methods to be compared, highlights their complementarities, and points to avenues for designing an evaluation framework adapted to the construction sector and its specificities.

The results are presented in the next section as a comparative analytical framework built from the evaluation grid.

# 3 Results and discussions

We present two complementary tables to make the analytical process explicit.

Table 1 shows that the literature mobilises a broad spectrum of methods, ranging from experimental and quasi-experimental designs to qualitative and theory-based approaches, as well as ex ante economic tools, policy content analyses based on text corpora, and configurational comparisons. This structure offers a homogeneous basis for comparison: for each method, it clarifies what is measured, at which scale the tool is most appropriate, and under which data constraints it can be implemented.

Table 2 shifts from description to diagnosis. It assesses, for each method, its relative contribution to Challenges C1–C5 (rapid technological change; diversity of instruments; sectoral and territorial heterogeneity; organisational and cultural transformations; fragmentation/multiple actors) and to methodological Issues I1–I5 (attribution of the policy's own effect; measurement and indicators; data collection; delayed/indirect effects; qualitative dimensions). The coding used (• well addressed; ~ partially; / little or not) makes clear that no tool covers all dimensions on its own and that overall robustness requires methodological combinations.
<table><caption>Table 1: Summary of the main public policy evaluation methodologies identified, with their characteristics and grouped into 3 main categories.</caption><thead><tr><td>Method</td><td>Type (approach)</td><td>Objectives (purpose of the evaluation)</td><td>Scope (level of intervention)</td><td>Data required</td><td>Key references</td></tr></thead><tbody><tr><td>Randomized Controlled Trial (RCT)</td><td>Quantitative (experimental)</td><td>Measure the causal impact of an intervention by comparing a treated group to a control group randomly assigned.</td><td>Micro (individuals, schools, etc., prior to any scaling-up)</td><td>Observed outcome data for each group (often dedicated data collection).</td><td>(Athey & Imbens, 2017); (Revillard, 2023)</td></tr><tr><td>Difference-in-Differences (DiD)</td><td>Quantitative (quasi-experimental)</td><td>Estimate an average policy effect by comparing before/after changes in an indicator between an exposed and an unexposed population.</td><td>Meso to macro (regions, sectors, or targeted vs. non-targeted groups)</td><td>Longitudinal or panel data (group time series, surveys or administrative statistics).</td><td>(Baïz & Guyot, 2022); (Varazzani, Emmerling, Brusoni, Fontanesi, & Tuomaila, 2023)</td></tr><tr><td>Regression Discontinuity Design (RDD)</td><td>Quantitative (quasi-experimental)</td><td>Estimate a local causal impact using a threshold eligibility rule: compare units just below/above the cut-off as quasi-equivalent.</td><td>Micro to meso (beneficiaries/non-beneficiaries near the threshold)</td><td>Detailed data on the running/score variable and outcomes of interest around the threshold.</td><td>(European Commission, 2012); (Bozio, 2015); (Baslé, Josselin, & Maux, 2018)</td></tr><tr><td>Matching / Propensity-Score Matching</td><td>Quantitative (quasi-experimental)</td><td>Evaluate ex post the effect of a program by building an artificial comparison group with characteristics close to those of beneficiaries.</td><td>Micro (individuals, firms) or meso (establishments) – requires large samples</td><td>Individual-level data rich in relevant covariates + participation status (surveys, detailed administrative registers).</td><td>(Bozio, 2015); (Athey & Imbens, 2017)</td></tr><tr><td>Cost-Benefit / Cost-Effectiveness Analysis (CBA/CEA)</td><td>Quantitative (economic, ex ante or ex post)</td><td>Assess efficiency: relate costs to discounted monetary benefits (CBA) or compare relative costs of alternative strategies for a common outcome (CEA).</td><td>Macro or meso (programs, regulations) – aggregate, social perspective</td><td>Financial data (budgetary costs, compliance costs) and monetary valuations of effects, or single-metric effectiveness measures.</td><td>(Crato & Paruolo, 2019); (Gregoir, 2014); (Coglianese, 2012)</td></tr><tr><td>Microsimulation (ex ante models)</td><td>Quantitative (modelling)</td><td>Simulate the expected impact of a reform on a whole population by reproducing, at the micro level, unit behaviours and policy rules (anticipates winners/losers, total costs).</td><td>Macro, bottom-up (national/regional populations) with micro granularity (individuals/households/firms)</td><td>Representative micro-data (household surveys, tax/social data) feeding a calculator that encodes current law and the proposed reform.</td><td>(Baïz & Guyot, 2022); (Bozio, 2018); (Revillard, 2023)</td></tr><tr><td>Case Study (qualitative evaluation)</td><td>Qualitative (descriptive, analytical)</td><td>In-depth understanding of how an intervention works in its real context, implementation processes, and stakeholder perspectives—usually via a holistic analysis of one/few cases.</td><td>Micro to meso (a project; a local program; possibly multi-site or longitudinal)</td><td>Field qualitative data: semi-structured interviews, observation notes, archives/internal reports; descriptive local statistics (to set the scene).</td><td>(Knoepfel, Larrue, Varone, & Savard, 2015); (Baïz & Guyot, 2022); (Baslé et al., 2018); (Revillard, 2023)</td></tr></tbody></table>
<table><tr><td>Policy Delphi (Delphi for public policy)</td><td>Qualitative (participatory, prospective)</td><td>Collect and confront views from a multidisciplinary expert panel on a complex public problem and its future evolution via iterative anonymous questionnaires. In Policy Delphi, the aim is to surface the full range of options/arguments rather than force consensus.</td><td>Macro – strategic/policy level</td><td>Expert knowledge from participants; structured questionnaires (often online) on trends, priorities, solutions; analysis of response statistics and qualitative comments at each round.</td><td>(Maleki, 2014)</td></tr><tr><td>Theory of Change (ToC)</td><td>Qualitative (theory-based, planning)</td><td>Make explicit the expected causal chains (inputs → activities → outputs → outcomes → impacts) and underlying assumptions to guide monitoring, evaluation, and learning.</td><td>All levels (micro to macro)</td><td>Co-design workshop(s) with stakeholders to build the causal map and indicators; then quantitative/qualitative evidence to test each hypothesised link during evaluation.</td><td>(Stein & Valters, 2012); (Revillard, 2023)</td></tr><tr><td>Realist Evaluation (implementation –functioning)</td><td>Qualitative & mixed (theory-based)</td><td>Explain results by identifying which mechanisms were activated for whom in which contexts. Aim is to formulate middle-range theories of how an intervention works rather than a single average effect.</td><td>Meso (program) to macro (national policy)</td><td>Mainly qualitative: in-depth interviews, focus groups, documentary data. May integrate quantitative data to test posited mechanisms.</td><td>(Revillard, 2023)</td></tr><tr><td>Contribution Analysis</td><td>Qualitative & mixed (theory-based)</td><td>Assess to what extent an intervention contributed to observed changes without claiming full attribution. Formulate and test plausible contribution claims using multiple evidence sources to reduce uncertainty in multi-causal settings.</td><td>Meso to macro – programs and policies with diffuse, multiple effects</td><td>Mixed evidence: existing outcome indicators, prior evaluations, perception surveys, interviews with key actors, local case studies—triangled to confirm/refute each contribution hypothesis.</td><td>(Baïz & Guyot, 2022); (Baslé et al., 2018)</td></tr><tr><td>Policy Content Analysis</td><td>Quantitative & qualitative (text mining/content)</td><td>Systematically examine policy documents (laws, decrees, plans) to extract themes, priorities, instruments and assess their evolution or internal coherence.</td><td>Macro (a national strategy or a corpus of sectoral policies)</td><td>Corpus of policy texts (official documents, strategic plans), optionally with meta-data (dates, levels); assisted analysis tools (NLP software, NVivo, etc.).</td><td>(He, Wu, Li, Li, & Wang, 2023); (Yang, Zhang, Hua, & Wang, 2025); (Zhang et al., 2023)</td></tr><tr><td>Qualitative Comparative Analysis (QCA)</td><td>Mixed (configurational comparison)</td><td>Identify which combinations of conditions are associated with a policy outcome by systematically comparing a limited number of cases; reveals multiple causal “recipes” (equifinality) and diverse trajectories.</td><td>Meso</td><td>Qualitative and quantitative data calibrated into binary or fuzzy-set conditions per case (e.g., presence/absence of X; high/low on an indicator). Calibration quality and per-case scoring are crucial.</td><td>(Revillard, 2023)</td></tr></table>

The structured comparison of methods (Table 1) highlights four complementary families of evaluation methods.

* **Quantitative causal-identification approaches (RCT, DiD, RDD, matching)** — orange in the tables. These approaches aim for high internal validity, meaning the ability to isolate the policy's own effect. Causal interpretation rests on identification conditions that must be tested and documented (parallel trends for DiD; continuity around a cut-off for RDD; ignorability/balancing
of covariates for matching; random assignment for RCTs). They perform well on Methodological issue I1 (attribution) and, partly, on I3 (selection bias). Conversely, they capture sectoral complexity and actor heterogeneity only imperfectly (Challenges C1–C2), they rarely account for delayed effects (C3), and their external validity can be limited when contexts vary substantially.

* **Ex-ante economic methods (CBA/CEA, microsimulation)** — blue in the tables. These methods inform efficiency trade-offs and incorporate medium/long-term horizons, including the distribution of effects across actor groups (Challenges C2–C3). They are suitable for simulating scenarios and aggregating costs and benefits at the collective level. Their main limitation lies in their dependence on valuation and behavioural assumptions; they therefore benefit from being fed ex post by causal estimates when available.

* **Qualitative and theory-based approaches (case studies, Theory of Change, realist/CMO evaluation, contribution analysis)** — yellow in the tables. These approaches explain how and why the policy produces effects, for whom and in which contexts, by uncovering mechanisms and causal chains. They directly address complexity and plurality of actors (Challenges C1–C2), consider indirect/external effects (Methodological issue I4), document organisational and cultural transformations (C4), and analyse multi-level coordination (C5). Their attribution is less “hard” than that provided by counterfactual designs, but this limitation can be mitigated through triangulation of sources and methods.

* **Cross-cutting tools** — green in the tables. Policy content analysis (PMC, text mining) evaluates the internal coherence and instrumental coverage of the policy mix (Challenge C5) and facilitates inter-territorial comparisons, although it does not by itself measure realised impact. QCA (configurational comparison) identifies, in small comparable samples, configurations of conditions associated with results (equifinality), which is useful when several causal combinations coexist in digital policies.

<table><caption>Table 2: Adequation of evaluation methods with Challenges and Issues (•: fully addressed; ~: partly; /: weakly or not)</caption><thead><tr><th>Method</th><th>C1</th><th>C2</th><th>C3</th><th>C4</th><th>C5</th><th>I1</th><th>I2</th><th>I3</th><th>I4</th><th>I5</th></tr></thead><tbody><tr><td>Randomized Controlled Trial (RCT)</td><td>/</td><td>~</td><td>/</td><td>/</td><td>/</td><td>•</td><td>~</td><td>•</td><td>/</td><td>~</td></tr><tr><td>Difference-in-Differences (DiD)</td><td>~</td><td>~</td><td>~</td><td>/</td><td>~</td><td>~</td><td>•</td><td>~</td><td>~</td><td>~</td></tr><tr><td>Regression Discontinuity (RDD)</td><td>/</td><td>~</td><td>/</td><td>~</td><td>/</td><td>~</td><td>•</td><td>•</td><td>~</td><td>/</td></tr><tr><td>Matching / Propensity Score</td><td>/</td><td>~</td><td>~</td><td>~</td><td>/</td><td>~</td><td>~</td><td>~</td><td>/</td><td>~</td></tr><tr><td>Cost-Benefit / Cost-Effectiveness (CBA/CEA)</td><td>~</td><td>/</td><td>•</td><td>~</td><td>~</td><td>/</td><td>~</td><td>/</td><td>~</td><td>~</td></tr><tr><td>Microsimulation (ex-ante)</td><td>~</td><td>•</td><td>•</td><td>~</td><td>•</td><td>/</td><td>~</td><td>/</td><td>~</td><td>~</td></tr><tr><td>Case Study</td><td>•</td><td>•</td><td>~</td><td>•</td><td>•</td><td>/</td><td>~</td><td>/</td><td>•</td><td>/</td></tr><tr><td>Policy Delphi</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>/</td><td>~</td><td>/</td><td>•</td><td>~</td></tr><tr><td>Theory of Change (ToC)</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>/</td><td>~</td><td>/</td><td>•</td><td>~</td></tr><tr><td>Realist Evaluation (CMO)</td><td>•</td><td>•</td><td>~</td><td>•</td><td>•</td><td>~</td><td>~</td><td>/</td><td>•</td><td>~</td></tr><tr><td>Contribution Analysis</td><td>•</td><td>•</td><td>~</td><td>•</td><td>•</td><td>~</td><td>~</td><td>/</td><td>•</td><td>~</td></tr><tr><td>Policy Content Analysis (text mining/PMC)</td><td>~</td><td>/</td><td>/</td><td>•</td><td>•</td><td>/</td><td>•</td><td>/</td><td>~</td><td>•</td></tr></tbody></table>
<table><tr><td>Qualitative Comparative Analysis (QCA)</td><td>•</td><td>•</td><td>/</td><td>~</td><td>~</td><td>~</td><td>~</td><td>/</td><td>•</td><td>~</td></tr></table>

From Table 2, it is clear that a single methodology is insufficient to evaluate public policies relating to digital transformation in construction. Quantitative causal designs primarily address Methodological issues I1 (attribution) and I3 (selection bias); theory-based approaches primarily address Challenges C1–C2 (complexity; plurality of actors) and Methodological issue I4 (externalities, mechanisms); CBA/microsimulation address Challenge C3 (long time horizons) and efficiency trade-offs; Delphi/ToC and content analysis address Challenges C4–C5 (vigilance to technological disruption; multi-level coherence).

In practice, a combination strategy is advisable: (i) measure a priority outcome (e.g., BIM adoption or productivity) using DiD/RDD/Matching with diagnostics of identifying assumptions; (ii) explain mechanisms and differences across sub-populations through case studies, realist evaluation, or contribution analysis anchored in a Theory of Change; (iii) arbitrate efficiency and capture delayed effects through CBA/CEA and microsimulation; (iv) steer policy-mix coherence via text mining/PMC and, where relevant, compare territorial configurations using QCA. This sequencing supports iterative and adaptive evaluation: design (ToC/Delphi) → ex ante (CBA/Microsimulation) → ex post (quantitative causal designs) → explanation (qualitative/theory-based) → strategy revision (content analysis/QCA).

Applied to a fragmented sector with long project cycles and numerous public-private partnerships three particularly pertinent methodological bundles are highlighted. Policies targeting efficiency and long-time horizons gain from combining CBA/CEA, microsimulation, and Delphi to incorporate technological uncertainty and delayed effects. Policies focusing on adoption and heterogeneous responses benefit from a DiD/matching pair to quantify outcomes, complemented by case/realist/contribution approaches to document barriers, enablers, and mechanisms. Policies centred on governance and coherence benefit from content analysis to map instruments and levels, a Theory of Change to link inputs and results, and QCA to identify winning configurations across territories and segments. These combinations jointly maximise internal validity (ability to isolate the policy's own effect), contextual relevance, and decision utility (learning and steering), which is the central requirement for evaluating digital transformation in construction.

This synthesis underscores that no method addresses the full set of problems on its own; combining approaches is often necessary for robust evaluation in the complex context of the construction sector's digital transformation. Two main lessons follow:

* The literature offers a wide range of methods, but their ability to address all Challenges and Methodological issues is uneven. Some approaches are robust for measuring economic or organisational impacts, yet struggle to account for qualitative dimensions or delayed effects.

* This heterogeneity confirms the need for an analytical framework that guides method choice according to context, objectives, and constraints specific to digital transformation policies in construction.

## 4 Conclusions

This article has offered a structured reading of the evaluation methods that can be mobilised for digital transformation policies in the construction sector. Based on a literature corpus and an analytical grid, two main results emerge: (i) a mapping of methods that made it possible to identify families (causal identification, ex ante economic tools, qualitative and theory-based approaches, text-data-driven
instruments, configurational comparisons); and (ii) a systematic review of how these methods address the key challenges (C1-C5) and methodological issues (I1-I5) of public policy evaluation identified in the literature. The evidence shows that no single approach covers all required dimensions on its own: counterfactual designs secure attribution, theory-based approaches explain mechanisms and heterogeneity, economic tools incorporate long time horizons and efficiency, while content analysis and QCA inform multi-level coherence and “recipes” for success.

These findings pave the way for a sector-adapted evaluation framework built on four principles:

* Decision matrix: link each evaluation question (and its context/data constraints) to appropriate combinations of methods.

* Sequencing along the policy cycle: design (ToC/Delphi) → ex ante (CBA, microsimulation) → ex post (quantitative causal designs) → learning/explanation (realist, contribution) → policy-mix revision (text mining/PMC, QCA).

* Explicit comparative criteria (time horizon, level of analysis, data requirements and quality, feasibility/cost, stakeholder engagement, ability to isolate the policy's own effect).

* Operational integration of C1-C5 and I1-I5 via a coverage grid to identify gaps and guide trade-offs.

This synthesis remains subject to several limitations: (i) the scope of the corpus (language and sector coverage biases, weight of some regions); (ii) heterogeneity and policy layering—new measures act on an existing stack of instruments, which many methods assuming a simple counterfactual capture poorly; (iii) data availability/quality (access to administrative/private data; unsettled productivity/quality indicators); (iv) temporal windows (delayed effects not observable over short horizons); (v) measuring organisational and cultural dimensions, which are difficult to quantify; and (vi) transferability of results across institutional contexts.

Future work will test the framework through case studies, standardise a core indicator set (adoption, productivity, quality, sustainability, satisfaction), organise data governance (administrative and digital-trace data), and equip inter-territorial comparison. The objective is to deliver a modular, mixed, and adaptive framework that reconciles evidence of effect, understanding of mechanisms, and policy-mix steering for digital transformation in construction.

## Funding

This work is funded by the National Sciences and Engineering Research Council of Canada (NSERC) : Alliance Grant ALLRP 601757 - 24 and by a French CIFRE PhD fellowship (Convention Industrielle de Formation par la Recherche) supported by the French Ministry of Higher Education and Research and administered by ANRT.

## Data Availability Statement

No specific data sets were used nor produced in the context of this research

## Conflicts of Interest

The authors declare no conflict of interest

## References

Alsofiani, M. A. (2024). Digitalization in Infrastructure Construction Projects: A PRISMA-Based Review of Benefits and Obstacles. https://doi.org/10.48550/arXiv.2405.16875

Athey, S., & Imbens, G. W. (2017). The State of Applied Econometrics: Causality and Policy Evaluation. *Journal of Economic Perspectives*, 31(2), 3-32. https://doi.org/10.1257/jep.31.2.3

Baïz, A., & Guyot, M. (2022). QUELLES ÉVALUATIONS DES POLITIQUES PUBLIQUES POUR QUELLES UTILISATIONS ?
Baslé, M., Josselin, J. M., & Maux, B. L. (2018). Dispositifs d'évaluation des politiques publiques et des programmes: connaissances de base, choix des méthodes, sociogramme des acteurs et études de cas.

Bernard, C., Lichère, F., & Micalef, R. (2024). *Digitalisation de la commande publique : état des lieux et perspectives d'évolution*.

Boussaguet, L., Jacquot, S., & Ravinet, P. (2010). *Dictionnaire des politiques publiques: 3e édition actualisée et augmentée*. (S.l.) : Presses de Sciences Po. https://doi.org/10.3917/scpo.bouss.2010.01

Bozio, A. (2015). L'évaluation des politiques publiques : enjeux, méthodes et institutions: *Revue française d'économie, Volume XXIX*(4), 59-85. https://doi.org/10.3917/rfe.144.0059

Bozio, A. (2018). Les méthodes d'évaluation des politiques publiques: *Idées économiques et sociales, N° 193*(3), 28-33.
https://doi.org/10.3917/idee.193.0028

Coglianese, C. (2012). *Measuring regulatory performance : Evaluating the impact of regulation and regulatory policy*. OECD Publishing.

Crato, N., & Paruolo, P. (Éds). (2019). *Data-Driven Policy Impact Evaluation: How Access to Microdata is Transforming Policy Design*. Cham : Springer International Publishing. https://doi.org/10.1007/978-3-319-78461-8

European Commission. (2012). *A note on the impact evaluation of public policies: the counterfactual analysis*. LU : Joint Research Centre. Institute for the Protection and the Security of the Citizen. Repéré à
https://data.europa.eu/doi/10.2788/50327

Fountain, J. (2019). The Wicked Nature of Digital Transformation: A policy perspective. *Dubai Policy Review, 1*(1), 40-45. https://doi.org/10.46993/DPR/EN005

Gabuthy, Y., Jacquemet, N., & L'Haridon, O. (2021). *Économie comportementale des politiques publiques*. Paris : La Découverte. https://doi.org/10.3917/dec.gabut.2021.01

Gregoir, S. (2014). L'évaluation des politiques publiques : qui et comment ?

Hassan, A. M., Negash, Y. T., & Hanum, F. (2024). An assessment of barriers to digital transformation in circular Construction: An application of stakeholder theory. *Ain Shams Engineering Journal, 15*(7), 102787.
https://doi.org/10.1016/j.asej.2024.102787

He, Q., Wu, Z., Li, S., Li, H., & Wang, Y. (2023). Two decades of the evolution of China's green building policy: insights from text mining. *Building Research & Information, 51*(2), 158-178.
https://doi.org/10.1080/09613218.2022.2142498

Knoepfel, P., Larrue, C., Varone, F., & Savard, J.-F. (2015). *Analyse et pilotage des politiques publiques: France, Suisse, Canada*. Québec : Presses de l'Université du Québec.

Lascoumes, P., & Le Galès, P. (2018). *Sociologie de l'action publique: (S.l.)* : Armand Colin.
https://doi.org/10.3917/arco.oumes.2018.01

Maleki, K. (2014). Delphi de politiques publiques comme une méthode de gouvernance participative.

Naji, K. K., Gunduz, M., Alhenzab, F., Al-Hababi, H., & Al-Qahtani, A. (2024). Assessing the Digital Transformation Readiness of the Construction Industry Utilizing the Delphi Method. *Buildings, 14*(3), 601.
https://doi.org/10.3390/buildings14030601

Revillard, A. (2023). Policy Evaluation: Methods and Approaches.

Rinchen, S., Banihashemi, S., & Alkilani, S. (2024). Driving digital transformation in construction: Strategic insights into building information modelling adoption in developing countries. *Project Leadership and Society, 5*, 100138.
https://doi.org/10.1016/j.plas.2024.100138

Samuelson, O., & Stehn, L. (2023). Digital transformation in construction – a review. *Journal of Information Technology in Construction, 28*, 385-404. https://doi.org/10.36680/j.itcon.2023.020

Stein, D., & Valters, C. (2012). UNDERSTANDING ‘THEORY OF CHANGE’ IN INTERNATIONAL DEVELOPMENT:

Varazzani, C., Emmerling, T., Brusoni, S., Fontanesi, L., & Tuomaila, H. (2023). *Seven routes to experimentation in policymaking: A guide to applied behavioural science methods* (Rapport No. 64). https://doi.org/10.1787/918b6a04-en

Xia, X., Liu, L., & Wang, Z. (2025). A Progressive Policy Evaluation Framework for Construction Digitalization in China: Evidence from Wuhan. *Buildings, 15*(11), 1925. https://doi.org/10.3390/buildings15111925

Yang, Y., Zhang, S., Hua, Y., & Wang, H. (2025). Mapping the digital transformation of AEC industry: Content analysis of digital public policy in China. *Developments in the Built Environment, 21*, 100621.
https://doi.org/10.1016/j.dibe.2025.100621

Zhang, J., Chen, M., Ballesteros-Pérez, P., Ke, Y., Gong, Z., & Ni, Q. (2023). A new framework to evaluate and optimize digital transformation policies in the construction industry: A China case study. *Journal of Building Engineering, 70*, 106388. https://doi.org/10.1016/j.jobe.2023.106388

### Disclaimer/Publisher's Note

The statements, opinions, and data contained in all publications are solely those of the individual author(s) and contributor(s) and do not reflect the views of the Architecture, Buildings, Construction and Cities (ABC2) Journal and/or its editor(s). ABC2 Journal and/or its editor(s) disclaim any responsibility for any injury to people or property resulting from any ideas, methods, instructions, or products referred to in the content.