## 3 Description of the ALCOAi Model  
The ALCOA+ principles are of paramount importance in today's pharmaceutical industry as these principles emphasize describing and validating the drug creation process in an end-to-end manner. Since it is now possible to quantify the ALCOA+ principles, pharmaceutical manufacturing industries can define thresholds under which individual principles can be considered violated. Violations of the ALCOA+ principles are a common occurrence and pharmaceuticals companies must dedicate sufficient resources to ensure ALCOA+ compliance. While much of the data needed to assess ALCOA+ compliance are compiled automatically by the automated systems of the production lines, many types of data, and especially human-derived data are often missing or partially compiled, with a consequent decrease in data quality and integrity. However, and given the workload of pharmaceutical industries, operator behavior trends often lead them to overlook compiling seemingly unimportant information, such as operator identification, or adding notes about crucial production events, with a consequent detrimental effect on ALCOA+ adherence. For all these reasons, the research and development work performed in the context of the SPuMoNI European project (www.spumoni.eu) aims at maximizing the possibility of maintaining a high ALCOA+ compliance standard and, as a project result, a set of predictive AI-based models has been developed. These models monitor pharmaceutical production data continuously and make predictions about ALCOA+ principle values trends based on current and past data compilation behaviors. The output of these models can be used by the pharmaceutical production managers as a decision support tool for making decisions regarding adopting necessary measures for best data quality and integrity practices. The ensemble of these models is called ALCOAi and it is composed of two main components:  
* An ALCOA+ regression model
* An ALCOA+ forecasting model  
For each lot of pharmaceuticals manufactured, or else, for each production batch, the data and meta-data generated during the manufacturing process are associated with a set of ALCOA+ values. The ALCOAi regression model is used to calculate ALCOA+ values based on the complete available data regarding each production batch. The ALCOAi forecasting network employs a transformer network capable of forecasting ALCOA+ principle values based on past batch production data and the corresponding ALCOA+ values. More specifically, the general architecture of the ALCOAi model is shown in Figure 1. The model is an ensemble of deep learning networks tailored to processing different kinds of data. Operator comments (in the form of free-text) are processed by a language modeling transformer [25], and raw sensor data streams are processed by neurons able to identify patterns in data sequences (GRUs [26]). Categorical and single numerical values are processed by linear neural networks. The output of each individual model is then concatenated and fed to a linear regressor network where ALCOAi values are computed. The modularity of the approach permits the adaptability to different datasets. In fact, in the case of different data types being introduced in the datasets (e.g., images), a CNN feature extractor could be employed in the processing stage of the ALCOAi pipeline and concatenated to the input layer of the regression layer. The network is trained to minimize the mean squared error loss function:  
\[L_{MSE}=\frac{1}{N}\sum_{i}^{n}(p_{i}-y_{i})^{2} \tag{1}\]  
where \(p_{i}\) is the estimated set of each ALCOAi principle value (\(i\)) and \(y_{i}\) is the actual one. Transformers are a type of neural network architecture that have been traditionally used for natural language processing tasks. However, they have recently gained popularity in time-series forecasting [10; 27], where they are able to effectively capture complex patterns in the time-series data as well as relationships with other types of variables.