## 2 Modelling Framework  
Our primary deliverable is a quantitative model designed to bridge the gap between recognising the need for machine learning (ML) in decision-making and implementing it. We define a \"quantitative\" model following [46], where causal relationships between variables are defined and quantified. The aim is to shift from qualitative conceptual models that suggest how joint human-ML decision-making might be structured to quantitative models that evaluate whether existing organisational policies and processes are conducive to successful ML implementation.  
This approach follows established guidelines for system dynamics modelling [45; 47], emphasising the need for a holistic understanding of problem-solving [48]. The framework operates within the evaluation phase of decision-making, which bridges conceptualisation and implementation. It leverages systems thinking principles, where the structure of the system determines its behaviour [28].  
As depicted in Figure 2, the evaluation phase comprises four main steps: Causal Mapping, Formulation of Simulation and Machine Learning Models, Partial Model Testing and Integration and Policy Analysis. Each step plays a crucial role in transitioning from conceptual understanding to an actionable and testable model.  
**Causal Mapping** is the foundation, representing current and future decision-making states through causal diagrams. This step abstracts operational details to define system boundaries, identify critical feedback structures, and set the stage for more detailed model formulation. The purpose is to make explicit the interdependencies and feedback loops within the system, ensuring that critical elements influencing decision outcomes are accounted for.  
Figure 1: Double-loop learning applied to refine collaborative human–AI decision-making.  
**Formulation of Simulation and Machine Learning Models** follows, employing tools such as Vensim, PySD and ML techniques (e.g., Recurrent Neural Network (RNN)). Here, causal maps guide the development of simulation models that integrate both ML and heuristic methods. The simulation models are structured to allow interaction with ML submodels, facilitating the exchange of information between predictive algorithms and rule-based processes. Data collation and transformation, shown in the diagram, prepare input data for ML training and testing, while simulation elements are parametrised to create and test various scenarios using different configurations of control variables.  
**Partial Model Testing** is crucial for verifying the local rationality of subsystems, ensuring alignment with process-level objectives before full integration. In this phase, code-based simulations with embedded ML models are run to assess the performance of individual subsystems, such as those depicted as Subsystems A, B, C and D. This iterative testing step helps identify potential biases or misalignments within isolated components, allowing for adjustments before the entire model is synthesised.  
**Integration and Policy Analysis** assesses the global rationality of the intervention by simulating system-wide outcomes and comparing them against organisational goals. This step explores potential misalignments and examines the robustness of different policy choices, informing adjustments needed for improved performance. The final output of this phase serves as a low-risk tool for evaluating whether to proceed with or adapt the human-AI strategy in a real-world implementation.  
In summary, this modelling framework leverages the strengths of both simulation and ML to create a cohesive system that bridges conceptualisation and implementation. Using tools like PySD for programmatic simulation and incorporating a combination of heuristic and algorithmic approaches, the framework facilitates double-loop learning; this ensures that real-world feedback informs and adapts both heuristic and ML components, fostering a more adaptive human-AI collaboration.  
Figure 2: Overview of the modelling framework.