# Machine Learning for predicting industrial performance: example of the dry matter content of Emmental-type cheese

Manon Perrignon, Mathieu Emily, Mélanie Munch, Romain Jeantet, Thomas Croguennec

<figure><img src="image_1.png" /></figure>

## To cite this version:

Manon Perrignon, Mathieu Emily, Mélanie Munch, Romain Jeantet, Thomas Croguennec. Machine Learning for predicting industrial performance: example of the dry matter content of Emmental-type cheese. International Dairy Journal, In press, 162, pp.106143. 10.1016/j.idairyj.2024.106143. hal-04802584v2

HAL Id: hal-04802584

https://hal.science/hal-04802584v2

Submitted on 16 May 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

# Machine learning for predicting industrial performance: Example of the dry matter content of emmental-type cheese

Manon Perrignon<sup>a, *</sup>, Mathieu Emily<sup>b</sup>, Mélanie Munch<sup>c</sup>, Romain Jeantet<sup>a</sup>, Thomas Croguennec<sup>a</sup>

## ARTICLE INFO

### Keywords:

- Artificial intelligence

- Dry matter content

- Machine learning

- Cheese production

- Emmental-type cheese

## ABSTRACT

Controlling the dry matter content of cheese is essential to defining the performance of cheese production. For Emmental-type cheese, dry matter content has to be above but as close as possible to a minimal value that is defined by legislation. The means for achieving the target dry matter content was mostly left to the discretion of the cheese experts, who target a dry matter objective based on his expert knowledge and the deviation of cheese production. To date, the prediction of performance indicators, such as cheese dry matter content, can help cheesemakers to improve their production performance. Several Machine Learning models and classical statistical methods were compared to predict the dry matter of Emmental cheese for a set of data coming from one selected cheese industry. The Random Forest method emerged as the most effective model (RMSE = 0.28 and R² = 0.67). The weight of variables in explaining the variability of cheese dry matter content was also calculated, helping cheese experts to interpret the model and apply corrective actions to improve cheese production performance. The ability to predict cheese dry matter content and understand its variability from cheese manufacturing data offer new perspectives for the cheese industry. This method can be transferred to other indicators and assist in decision-making to enhance industry performance.

## 1. Introduction

Although standardized, cheese production is subject to numerous sources of variability, stemming from the natural variability of raw materials, human perception, human intervention in the process, and the capabilities and maintenance of processing tools, among others. Overall, these factors affect the reproducibility of the cheese's characteristics, leading to deviations from the target. These deviations can directly impact industry performance indicators (yield, quality indicators, etc.). Improving industrial performance is a major challenge in an increasingly competitive environment. In this context, the ability to predict performance indicators proves to be a powerful tool for enhancing production.

Historically, linear methods such as multiple linear regression have been favored for their simplicity of interpretation and implementation (Kern et al., 2019). However, these approaches frequently assume linear relationship between parameters which is commonly assumed to be unrealistic (Ma et al., 2020). Thanks to the advent of Industry 4.0, a

large amount of data related to the product and to processes are recorded throughout the cheese manufacturing process (Shankarrao Patange & Bharatkumar Pandya, 2022). These data remain underexploited thus offering an interesting potential for a better control of cheese manufacture. To leverage this data and overcome the limitations of linear methods, the emergence of artificial intelligence methods such as Machine Learning allows for modelling complex and non-linear relationships between manufacturing parameters measured throughout the cheese-making process (Garre et al., 2020). Machine Learning methods, like Random Forest or Support Vector Machine, offer interesting prospects for predicting performance indicators (Shankarrao Patange & Bharatkumar Pandya, 2022). Nevertheless, the adoption of these data-driven methods raises new challenges. First, managing overfitting is crucial to ensure unbiased evaluation of models. Second, selecting appropriate evaluation metrics is key for accurately measuring model performance in a specific context. Finally, Machine Learning models, compared to a simple linear model, requires the tuning of hyperparameters that drive the learning process. These

<footnote><sup>a</sup> L'Institut Agro, INRAE, STLO (Science et Technologie Du Lait et de L'œuf), Rennes, France</footnote>

<footnote><sup>b</sup> L'Institut Agro, Université de Rennes, CNRS, IRMAR (Institut de Recherche Mathématique de Rennes)-UMR 6625, Rennes, France</footnote>

<footnote><sup>c</sup> INRAE, STLO (Science et Technologie Du Lait et de L'œuf), Rennes, France</footnote>

<footnote>* Corresponding author. author.</footnote>

<footnote>E-mail address: manon.perrignon@agrocampus-ouest.fr (M. Perrignon).</footnote>
hyperparameters, such as the number of trees in a Random Forest, influence the behavior of the model and require careful optimization to achieve optimal results. Some of these methods can also be considered as "black-box" methods, making their understanding and interpretation challenging and limited (Ma et al., 2020). However, Machine Learning offers new perspectives and has already been introduced in the food sector and more recently in the dairy sector (Garre et al., 2020; Golzarijalal et al., 2024; Khan et al., 2022; Wan et al., 2022), emphasizing its potential for improving cheese production (Przybyl, 2024).

One of the key indicators defining the performance of cheese production is cheese dry matter content (Kern et al., 2019). Cheese dry matter content has to be above a regulatory value (>60% at the time of sale for Emmental-type cheese, a pressed and cooked cheese), and the goal of the cheese makers is to minimize deviation from this target. Cheeses with dry matter content below 60% are non-conformed while the increase of cheese dry matter content above 60% is of higher cost for the cheese company. Currently, cheese dry matter content is controlled through milk standardization that allows the composition of cheese milk to be adjusted, such as casein micelle content, and by adjusting manufacturing parameters such as stirring time (Guinee, 2021; Xia et al., 2021). However, these adjustments are often empirical, based on expert knowledge; moreover, they are set for each cheese batch but with limited scope for corrections explaining the standard deviations regarding cheese characteristics, such as cheese dry matter content. Therefore, Machine Learning methods pave the way on new perspectives to predict and interpret the variability of cheese dry matter content from data collected throughout the manufacturing process and finally help improving cheese production.

The aim of this study is to evaluate the performance of Machine Learning methods in predicting the performance indicator for the cheese industry, using dry matter content as an example. The entirety of the manufacturing parameters, from milk collection to brining (including raw material composition and process parameters), was incorporated into the modelling process. Several Machine Learning methods will be tested and compared with more traditional linear modelling. This comprehensive approach seeks to provide a holistic view and subsequently to assess the impact of different parameters on the variability of cheese dry matter content.

# 2. Materials and methods

## 2.1. Data collection

The data come from one year of production of an industrial cheese (Emmental-type) plant. The parameters are recorded throughout the cheese manufacturing process, from milk collection to cheese aging. There are parameters defining raw milk composition, milk standardization parameters (such as maturation time and temperature, and ingredient quantities), vat operations (like setting time, stirring speed or heating ramp, etc), as well as on the evolution of product's composition throughout the process, such as pH and traceability information. Data come from both automated sensors and manual input. Automated sensors continuously monitor key process parameters such as temperature, humidity, and pH levels throughout the production process. Variability in the data collected is inherent to cheese production due to fluctuations in raw materials composition or processability. This natural variability is an essential aspect of the data, as it provides a realistic representation of the operational conditions and potential variations in production outcomes. In addition, data include other sources of variability. While these sensors are regularly checked through preventive maintenance to guarantee accuracy, occasional calibration drift or temporary malfunctions can introduce variability in the data. At various steps, notably when entering raw material characteristics and quality control checks, the data are entered manually in the database by operators, introducing the possibility of human error in the form of data entry mistakes. Data is compiled into a spreadsheet where each row represents a cheese vat and

the columns are the parameters. Dataset consists of 3500 cheese vats (individuals) for which 300 parameters (variables) were measured.

## 2.2. Preliminary processing

The raw dataset was composed of quantitative variables and qualitative variables and were pre-processed before further analysis. The dataset was filtered in order to retain only the essential information and avoid redundancy. Data pre-processing is divided into three steps. The first step focuses on detecting outliers. For this, data visualization techniques such as boxplots or distribution graphs were used. This filtering was performed with the help of cheese experts. Experts are consulted to determine whether outliers provide valuable information or if they are simply data entry errors. Their insights help in assessing the nature of these outliers. The second step involves the removal of redundant variables, as some variables can be represented with various units (i.e., minutes or seconds), or variables that are out of the scope of the study (i.e., traceability of the final product). The final step is to transform the initial dataset. For this analysis, we chose to normalize the data by centering and scaling them to give all variables equal weight. This transformation was applied only to quantitative variables. Further details on the preliminary processing steps will be provided in section 3.1.

## 2.3. Modelling approaches

### 2.3.1. Tree and kernel based models

For this study, two categories of Machine Learning methods were used: tree-based models (Random Forest and Gradient Boosting Machine) and kernel-based models (Support Vector Machine). These methods are the most commonly used machine learning methods. They are based on different principles, making their comparison relevant. While Random Forest and Gradient Boosting are tree-based approaches, Support Vector Machine aims to identify a hyperplane that optimally separates the data.

Tree-based models, i.e. Random Forest (RF) and Gradient Boosting Machine (GBM), are constructed using an ensemble of decision trees built from the dataset. The principle of the Random Forest (Breiman, 2001) is to build a large number of trees from a subset of the dataset. The Random Forest algorithm selects subsets and variables for tree construction randomly, to obtain a diversity of trees (Garre et al., 2020; Speiser et al., 2019). Each tree returns a prediction of the output variable, i.e. cheese dry matter content, and the final prediction is calculated by averaging the predictions of each tree (Koay et al., 2022; Ma & Cheng, 2016). For this method, the following hyperparameters have been optimized: the number of variables randomly selected at each node (mtry) and the minimum node size (min.node.size) (Ma & Cheng, 2016). This method has been implemented in Rstudio version 2023.09.0, using *ranger* package.

The Gradient Boosting algorithm (Friedman, 2002) constructs decision trees sequentially. Each new tree corrects the error of the previous one in order to obtain better predictions as the process progresses (Ma & Cheng, 2016; Wan et al., 2022). For this method, the following hyperparameters have been optimized: number of trees to be built (n.trees), the maximum depth of the tree (interactiondepth), the learning rate (shrinkage) and the minimum node size (min.node.size). This method has been implemented in Rstudio version 2023.09.0, using *gbm* package.

Support Vector Machine (SVM) methods (Boser et al., 1992) separate data using a boundary called a hyperplane, in such a way that the distance between different data groups and the separating boundary is maximized. This boundary assumes that the data are linearly separable, which is rarely the case. To overcome this limitation, SVMs use "kernels," that is mathematical functions that separate data by projecting them into a new space. In this study, radial and polynomial kernels were used. For these methods, the following hyperparameters have been optimized: the regularisation parameter (C), the kernel parameter (sigma) for radial
kernel and the degree of the polynomial function for the polynomial kernel (Li & Wu, 2022). These methods have been implemented in Rstudio version 2023.09.0, using *kernlab* package.

### 2.3.2. Multilayer perceptron

A multilayer perceptron (MLP) was used to predict dry matter content. MLP is a type of artificial neural network composed of several layers of neurons: an input layer, one or more hidden layers, and an output layer (Bhagya Raj & Dash, 2022). Each neuron in a layer is connected to each neuron in the next layer, with adjustable weights. MLPs use non-linear activation functions to allow complex relationships to be modelled. This method has been implemented in Rstudio version 2023.09.0, using *RSNNS* package, with an architecture consisting of three hidden layers.

### 2.3.3. Linear models

A linear regression model was used to predict a linear relationship between an outcome variable $y_i$, i.e. cheese dry matter content, and one or more explanatory variables $x_{ip}$. The linear regression model can be written as follows:

$$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_p x_{ip} + \varepsilon_i$$

with $i = 1, \dots, n$ and where $x_{ip}$, is the value of the p-th variable for the i-th individual. The parameters $\beta_p$ of the model are unknown and need to be estimated. The parameter $\beta_0$ represents the intercept of the model. $\varepsilon_i$ represents the error of the model.

To cope with the curse of dimensionality, regularized linear models have also been estimated. Regularisation models introduce a penalty on the complexity of the model by adding a regularisation term. Lasso (Tibshirani, 1996), ridge (Hoerl & Kennard, 2000) and elastic net models, that differ regarding the regularisation term, were used. Elastic net model is a combination of lasso and ridge models with a penalty function constructed by summing the penalty functions of lasso and ridge model (Garre et al., 2020). These methods have been implemented in Rstudio version 2023.09.0, using *glmnet* package.

### 2.3.4. Multivariate adaptive regression spline

Multivariate adaptive regression spline has also been estimated in order to test a non-linear modelling of the relationship between dry-matter and variables. The multivariate adaptive regression spline (Spline) introduces non-linearity assumptions. Spline regression is a non-parametric regression technique (Friedman, 2002). It is an extension of the classic linear model that captures non-linearity. This method divided data into segments and fitting functions within each segment. The points where the segments meet is called "knots." The main idea is to ensure that the fitted pieces meet at the knots to maintain continuity. By doing this, regression spline can capture complex relationships in the data more effectively than simple regression (Friedman, 1991; Garre et al., 2020). This method has been implemented in Rstudio version 2023.09.0, using *earth* package.

## 2.4. Evaluation and comparison of models

The dataset was randomly split into two subsets: a training set, comprising 80% of the individuals, and a test set, containing the remaining 20% individuals. The training set was used to train the models and optimize the hyperparameters. The test set was used to evaluate the model's performance (Wan et al., 2022). Hyperparameter tuning was performed to optimize model performance. Key hyper-parameters, such as learning rate, the number of trees or the number of neurons in a layer for a deep learning model, were adjusted using a grid search approach. Five-fold cross-validation to evaluate different combinations of hyperparameters was employed, ensuring that the models were not overfitting to specific subsets of the data. Cross-validation is a technique used to evaluate a model's performance by splitting the data

into multiple subsets, training the model on some subsets, and validating it on the remaining subsets. This allows for better estimation of how the model will generalize to new, unseen data. "Five-fold" means that the dataset is divided randomly into 5 equal parts or "folds". The model is trained 5 times, and each time, it uses 4 of these 5 parts as the training set and the remaining 1 part as the validation set. This process is repeated so that each of the 5 parts represents the validation set only once (Münch et al., 2023). The final set of hyperparameters was selected based on the model's performance, defined by the minimization of the root mean square error metrics (RMSE). This ensures the model's robustness on unseen data.

After hyperparameter optimization, the trained model predicts the cheese dry matter using the test set. To obtain a 95% confidence interval for the predictions, this step was repeated 100 times. The training and test subsets were randomly separated 100 times using the bootstrap method, which involves creating multiple random samples with replacement from the original dataset. For each iteration, the model is trained on a bootstrap sample (train set) and tested on the "out-of-bag" data (test set). This approach allows the model's performance to be evaluated across different samples, providing a robust estimate of its predictive accuracy and variability. The 95% confidence interval is derived from these repeated predictions, giving a reliable range for the cheese dry matter content. The accuracy of the predictions is assessed using root mean square error (RMSE) and R² between the observed cheese dry matter content and the predicted values (Golzarijalal et al., 2024; Wang et al., 2022). The RMSE metric is the deviation of dry matter content (in %) between the model's predictions and the analytical measurements. The R² metric quantifies the proportion of variance in the response variable (dry matter content) that is explained by the model. A low RMSE value indicates that the model predictions are close to the cheese dry matter content measurements and a high R² value indicates a good model accuracy. The combination of RMSE and R² provides a comprehensive view of the model's performance. All the functions required for the model building have been implemented in Rstudio version 2023.09.0, using *caret* packages.

# 3. Results

## 3.1. Preprocessing step

The dataset (with 3500 individuals corresponding to cheese vats and 300 columns corresponding to variables or measured parameters) was pre-processed in order to make the database analysable and to obtain a subset of data that gathers all relevant information for the study. For this, a representation of the distribution of each variable was used to identify outliers as they can significantly impact models. This step was carried out in collaboration with experts from the cheese plant. For instance, data coming from operator input errors (unrealistic pH and temperature values) or data collection problems (automatic data registration when the processing plant is shut down) were considered as outliers. Data coming from cheese trials interspersed between cheese making batches were not considered as outliers even if they are outside the distribution, as they give indications on realistic trajectories of the cheese making process. Individuals in the dataset with outliers were removed.

The variables were also cleaned to remove duplicates or irrelevant data. For instance, some processing times were recorded in duplicate, in both seconds and minutes. In order to avoid overexpressing the "time" variables in models, variables in minutes were removed and only variables in seconds were kept. Variables for product traceability were removed as they are irrelevant to this study, as well as the variables related to cheese ripening since only the dry matter content of unma-tured cheeses was considered for analysis.

After data pre-processing, the final dataset consisted of 2833 individuals or cheese vats, and 76 variables: 1 response variable (cheese dry matter content) and 75 explanatory variables that represent all
parameters measured during production; the latter can be either quantitative or qualitative (with multiple modalities). The distribution of the dry matter content of the cheeses made from the 2833 cheese vats shows that 2% were below 60% and most of the cheese have a dry matter content around 61%.

This pre-processing step is time-consuming but the cleaned database can be used without further handling for other performance indicators (cheese weight, cheese fat recovery, etc.). However, the pre-processing depends on the nature of the data and cannot be replicated to other datasets. Each dataset is unique and requires a tailored pre-processing step specific to its characteristics.

## 3.2. Comparison of modelling methods for cheese dry matter content

For model construction, 2268 individuals (80%) were used for the training set and 565 individuals (20%) for the test set. After hyper-parameters optimization (Table S1) and models training, model's performances were assessed by calculating RMSE and R² using the test set. Although R² is a well-known metric that provides a measure of how well the model fits the data, RMSE offers a more intuitive assessment of prediction error, as it is expressed in the same units as the cheese dry matter content. Using this metric, industry experts achieve a clearer and more practical understanding of model evaluation, enhancing its relevance and accessibility for interpreting prediction.

A comparison of the results of the different predictive models is presented in Table 1 and Fig. 1 where both performance metrics (RMSE and R²) are displayed. Fig. 1 visualizes the performance of the model using box plots, with RMSE on the left part of the figure and R² on the right part.

The results show that the four best-performing methods are Random Forest (RF), Gradient Boosting Machine (GBM) and two variants of Support Vector Machines (SVM poly and SVM radial). Linear models (LinMod, Lasso, Ridge, and Elastic Net) and Spline regression (Spline), on the other hand, show similar performance among themselves but are inferior to the previously cited models. Lastly, Multilayer Perceptron (MLP) stand out with much lower performance compared to the other methods.

According to Fig. 1 the median of the RF boxplot is slightly higher for R² compared to GBM and SVMs, and slightly lower for RMSE, with an RMSE of 0.28 for RF compared to 0.29 for radial SVM's and GBM and 0.30 for polynomial SVM's (Table 1). Additionally, the RF model has smaller confidence intervals (0.26–0.29 for RMSE), suggesting a narrower prediction distribution of the cheese dry matter content than GBM and SVMs. This is an important element to consider at industrial scale with aim to reduce variability of the dry matter content in the final

<caption>**Table 1**</caption>

Performance of different models in predicting dry matter content. Evaluation metrics (Root Mean Square Error (RMSE) and R²) were calculated on the test set. 95% interval confidence was calculated with a bootstrap step (100 iteration). RF = Random Forest, GBM = Gradient Boosting Machine, SVM = Support Vector Machine, MLP = Multilayer Perceptron, LinMod = Linear Regression.

<table><thead><tr><th>Methods</th><th>RMSE</th><th>R<sup>2</sup></th></tr></thead><tbody><tr><td><strong>RF</strong></td><td><strong>0.28</strong> [0.26-0.29]</td><td><strong>0.67</strong> [0.64-0.70]</td></tr><tr><td>GBM</td><td>0.29 [0.28-0.31]</td><td>0.63 [0.59-0.66]</td></tr><tr><td>SVM poly</td><td>0.30 [0.28-0.33]</td><td>0.61 [0.55-0.67]</td></tr><tr><td>SVM radial</td><td>0.29 [0.28-0.32]</td><td>0.62 [0.57-0.67]</td></tr><tr><td>MLP</td><td>0.46 [0.34-0.91]</td><td>0.49 [0.13-0.55]</td></tr><tr><td>LinMod</td><td>0.38 [0.36-0.39]</td><td>0.39 [0.34-0.42]</td></tr><tr><td>Lasso</td><td>0.37 [0.36-0.39]</td><td>0.39 [0.35-0.43]</td></tr><tr><td>Ridge</td><td>0.38 [0.36-0.39]</td><td>0.38 [0.35-0.42]</td></tr><tr><td>Elastic_net</td><td>0.38 [0.36-0.39]</td><td>0.39 [0.35-0.42]</td></tr><tr><td>Spline</td><td>0.37 [0.34-0.42]</td><td>0.43 [0.32-0.50]</td></tr></tbody></table>

product.

The spline regression model, which incorporates non-linearity, shows slightly better performance compared to linear models, with a R² of 0.43. According to Fig. 1, the Multilayer Perceptron (MLP) model appears to be the least effective, displaying a significantly higher RMSE in contrast to the other models. Additionally, A wide confidence interval is observed for this model, indicating a lack of precision in its predictions (Table 1).

RF model is selected because it achieved the highest evaluation metrics and maintained a narrow confidence interval, reflecting strong model reliability. Fig. 2 illustrates the observed versus predicted values of cheese dry matter with RF. This representation helps evaluate the model's precision and its ability to accurately reflect the variability of the cheese dry matter. RF was therefore deeply investigated to better understand the role of each variable in the prediction of the cheese dry matter content.

## 3.3. Interpretation of dry matter variability using Random Forest model

The weight of variables in predicting cheese dry matter content using RF model was calculated by examining the evolution of the prediction error when the value of one variable change while the other variables remain fixed. The analysis was performed on a training set comprising 2268 individuals, and a new calculation is performed each time a tree is built (Ma & Cheng, 2016). The outcome of this analysis is a classification of the order of importance of the variables in explaining the variability of the cheese dry matter content.

The summary of the 5 relevant variables is depicted in Fig. 3. The variable “Strain” refers to the lactic acid bacteria strains added to cheese milk for its maturation before renneting. Several strains are used in rotation in cheese factories, particularly to mitigate phage infection risks. In this study, changing the strain appears to have the most significant impact on cheese dry matter content variability.

The variable “Milk Fat” has the second strongest impact on cheese dry matter content variability. This variable denotes the fat content of raw milk. Note, that the cheese milk is standardized for fat and protein contents prior to cheese making as the raw milk composition, especially fat, change with seasonality and cow’s diet (Jaeggi et al., 2005; Lopez et al., 2007).

The third most important variable, “Position in brine” represents the position of the cheeses in the brine pool. During the brining step, cheeses are placed individually in racks stacked and immersed in a brine pool. The position of the racks in the brine is determined, and this position impacts the variability of the dry matter content of the cheeses.

The milk standardization tank appears twice among the impactful variables, with variables “tank” and “tank order.” The variable “tank” represents the standardization tanks used to store milk before its transfer to the cheese vats. Each standardization tank is a new mixture including raw milk and other ingredients (skimmed milk concentrate, cream, etc.) in proportion adapted to meet the standardization targets (protein and fat contents). The variable “tank order” indicates the order in which cheese vats are drawn from one standardization tank. The first vat drawn from the tank is numbered 1, and so forth until the tank is empty (7–9 cheese vats filled with one standardization tank).

Machine learning models can be complex, making transparency and interpretability essential, especially for industrial applications. Complex models often act as “black boxes” (Madoumier et al., 2019), with decisions that are hard to understand or trust. Explainable AI (XAI) aims to address this drawback by providing clarity on model decisions, helping users understand and trust predictions (Przybyl, 2024). There are several methods to help interpret the models (Ali et al., 2023) but this article focuses on SHapley Additive exPlanations (SHAP) (Lundberg and Lee, 2017), a model-agnostic method that explains both models and predictions. This method enables the assessment of the individual impact of variables and can ascertain whether a variable has a predominantly positive or negative effect on cheese dry matter content,
<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Comparison of prediction performance of cheese dry matter content based on different models. Root Mean Square Error (RMSE) and R² were calculated for the test set for each predictive model. The boxplots were constructed using the results from the test set after the bootstrap step (100 iterations). RF = Random Forest, GBM = Gradient Boosting Machine, SVM = Support Vector Machine, LinMod = Linear Regression, MLP = Multilayer Perceptron.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Representation of the observed and predicted values of cheese dry matter content determined by the RF model on the test set.</figcaption></figure>

providing additional insights that are valuable for operational decisions during cheese making. This method was applied to the variables “Milk fat” (Fig. 4) and “Strain” (Fig. 5), which are the two most impactful variables. The purpose of this method is to evaluate how the cheese dry matter content varies with the milk fat in the raw milk used to prepare the standardized cheese milk and the strain used. Fig. 4 illustrates this effect on “Milk fat”. The points in the figure represent predictions made by the RF model. Red points indicate high values of the explanatory variable (high milk fat content in raw milk), while blue points represent low values of the explanatory variable (low milk fat content in raw milk). The red points are positioned on the left side of the graph, showing that a high milk fat content in the raw milk used to prepare the standardized cheese milk leads to a decrease in dry matter content of cheeses. For the strain used (qualitative variable) the SHAP results can be examined using a dependence plot (Fig. 5). This graph illustrates the

effect of the strains used on the prediction of the cheese dry matter content. Fig. 5 shows that strain “O” results in dry matter content higher than the average prediction, while strain “1” leads to dry matter content lower than the average prediction. These results and visualizations help experts better understand the sources of variability in the cheese dry matter content and adjust their processes accordingly.

## 4. Discussion

RF demonstrate superior performance in predicting dry matter content than traditionally used linear models. This improvement is evidenced by calculated dry matter content consistent with actual values and a narrower confidence interval than the standard deviation observed in the factory. Thus, our study demonstrates the potential of RF to predict dry matter in the cheese industry. However, once the model
<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Representation of 5 influencing variables responsible for predicting cheese dry matter content. Their importance is measured by RF model and ranked in descending order.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**Fig. 4.** SHAP analysis of the impact of milk fat on the prediction of cheese dry matter content (x-axis) using the RF model. Each point represents an individual prediction, with red indicating high values of milk fat content and blue indicating low values. On the x-axis, 0 corresponds to the mean of the dry matter content predictions.</figcaption></figure>

has been selected and the variables influencing for the cheese dry matter content identified, it is important to cross-check the results with cheese experts in order to establish direct links with the process. This discussion is essential to interpret some variables that may contain hidden information (confounding factor). For example, it is difficult to establish a direct link between the “position in brine” variable and the cheese dry matter content. Even if the brine pool is agitated, variations in salinity or temperature between positions in the brine may exist. These parameters constitute hidden information that are not measured in the cheese process but that affect water transfer from the cheese to the brine and consequently cheese dry matter content (Guinee & Fox, 2017). To check this hypothesis, sensors were placed at various positions within the brine and confirmed that temperature differences within the brine pool exist. This difference is a highly plausible explanation of the variation of the dry matter content between cheeses according to their position in the brine pool. Conversely, some impactful variables on the cheese dry matter content are already well-identified in the industrial context. For example, the strains used for cheese milk maturation are known to influence the cheese dry matter content. However, regular strains rotation is essential to avoid phage infection risks. Through the variable importance analysis, the impact of strains on the cheese’s dry matter content was confirmed and the SHAP clearly highlights the strains that increase

or decrease the cheese dry matter content. Presently the explanation behind this correlation is unclear. Though the strains included in a rotation are selected to give equal technological performance some slight differences between strain activities, such as the relative abundance of bacterial species, production of key metabolites, proteolytic activities, etc., may exist but this hypothesis cannot be confirmed in the present analysis (Decadt et al., 2023). On the other hand, fat content in raw milk influences the cheese’s dry matter content. Previous studies have shown a connection between these two variables but when making cheese with milk that varies in fat content (Fenelon & Guinee, 1999). In the present study, cheese milk was standardized with various proportions of raw milk and skimmed milk in order to take into account the various fat content in raw milk. As the amount of skim milk do not affect cheese dry matter content, it is plausible that the composition of the fat fraction is at play but this information is not yet in the database. The relevance of the model and its interpretation lies in the ability to correlate the effects of different strains and milk composition to cheese dry matter content. Based on cheese expertise, corrections can be implemented on the process according to the strain or composition raw milk used to prepare the standardized cheese milk.

Other relationships that arise from this study need to be verified and validated by cheese experts, such as the link between the cheese dry
<figure><img src="image_8.png" /><figcaption>**Fig. 5.** SHAP dependence plot of the effect of strain type on cheese dry matter content predictions. Each point represents a prediction using RF model. The strain variable is a qualitative variable with 7 levels representing the 7 strains used in the industry (from 0 to 6). On the y-axis, 0 corresponds to the mean of the dry matter content predictions.</figcaption></figure>

matter content and the order in which cheese vats are filled with standardized milk from the standardized milk tank. The importance of the "tank" and "tank order" variables may be explained by the heterogeneity of cheese milk composition in the tank. Since milk is a suspension, a gradient of fat content from the bottom to the top of the tank may appear over time for example if the agitation in the tank is insufficient. Moreover, some fat can hold back on the wall on the tank. These effects could induce slight variations in cheese milk composition from tank to tank, which in turn may influence the cheese dry matter content.

These observations highlight the importance of data collection to improve model prediction. The R² value indicate that 67% of the variability in dry matter is explained by the Random Forest model. This result can be significantly improved by enriching the initial database. Some parameters (such as temperatures or salinity at various points in the brine pool) are not currently measured but could significantly enhance the prediction model. Conversely, some recorded parameters do not influence the prediction model, raising concerns about the resources spent on collecting them. This highlights the importance of having a comprehensive database with relevant measurements before conducting such analyses. Unlike the R<sup>2</sup> values, the differences between the RMSE values for all the methods are small which can be explained by the low dispersion of the response variable. Indeed, since the models learn from the data, the differences between the models predictions are limited by the low variability of the measured dry matter content. The RMSE represents the average deviation of the predictions from the observed values, which explains why the performance differences between the methods remain minimal.

Machine Learning methods are inherently data-driven, meaning their performance heavily relies on the characteristics of the data they are trained on. Therefore, comparing multiple methods is essential to identify the most suitable approach based on the specificity of the prediction problem and quantity and quality of the data collected. Comparison of predictive models to calculate the dry matter of cheese from process parameters shows disparities between the performance of machine learning methods. RF was found to be the most adapted to our study. RF is known for their very good predictive performance and their genericity, imposing very few restrictions on the type of the data and the

relationships between variables. This good performance can also be associated with its easy implementation, with few hyperparameters to optimize. It has already proven its efficiency in the dairy sector (Freire et al., 2024; Fröhlich-Wyder et al., 2023). Unlike RF, MLP did not provide satisfactory results performance (Table 1). MLP method operates using a neuron architecture and can achieve very promising prediction performances if the network architecture is suitable for the data. This discrepancy in the performance could be due to the implementation of the method itself. This model has a complex structure, which makes the hyperparameter optimization step more challenging. Proper tuning of hyperparameters—such as the number of hidden layers for MLP—is critical to achieving accurate results and preventing overfitting. Optimizing hyperparameters helps mitigate this risk, but it requires a sufficiently large amount of data to properly calibrate the models. It is highly possible that in this study, the number of data is insufficient for this type of model to learn efficiently. Each method is unique in its approach and performance, which varies depending on the specific problem and dataset. This underscores the importance of comparing different models to identify the most effective solution for a given context.

Machine learning models do not rely on any modelling assumptions but instead allow the data to speak for themselves. These methods are particularly useful when the relationships between parameters are complex, such as predicting the cheese dry matter content from cheese process parameters. However, it is often difficult to establish hypotheses between the response variable (cheese dry matter content) and explanatory variables (process parameters) when employing Machine Learning as black box model (Wan et al., 2022). In an industrial setting, prediction alone is insufficient for process improvement. It is crucial to interpret the models to understand variability and support decision-making. The interpretability of Machine Learning models is an emergent topic and therefore requires new methods to better understand these models (Tomsett et al., 2018). The importance of variables estimated by the model only reflects the order of importance of variables in explaining the variability of dry matter content. With this information alone, it's not possible to determine how each variable impacts dry matter content (whether it has a positive or negative correlation). To gain deeper insights and improve interpretation, methods such as SHAP
values, provide more detailed explanations of machine learning models. Ali et al. (2023) provide a review of various methods for interpreting machine learning models. Explainable AI (XAI) methods are promising for understanding these models, making them more applicable and useful in industrial applications.

## 5. Conclusion

This study demonstrates the effectiveness of the Random Forest method in accurately predicting the dry matter content of Emmental cheese. While traditional linear models are limited by their strong initial assumptions, machine learning techniques such as RF have the advantage of learning directly from the data, thus avoiding any assumptions. However, the performance of machine learning methods is linked to the nature and quality of the training dataset, as well as the specific context of the study. Therefore, a rigorous comparison of different approaches remains essential to identify the most relevant and best-suited model for each particular situation to analyse. This careful selection leads to more accurate and reliable applications in the cheese industry. The accuracy of the models is linked to the quality and quantity of the collected data. In this regard, conducting industrial trials would enrich the existing database, paving the way for model retraining and, consequently, a potential improvement in their predictive performance. Incorporating additional variables or exploring different machine learning techniques and methods for interpreting models could offer new insights and may improve further model performance. The variables explaining the variability of cheese dry matter content provides insights into key parameters that need to be managed to better control production and reduce variability in cheese dry matter content. Depending on the strain and fat content of the raw milk, it will be now possible to define better process trajectory for optimizing cheese dry matter content. Temperature sensors in the brine can be used to define the brining time for reaching the targeted cheese dry matter content, depending on their position in the brine. An in-depth interpretation of the influencing variables should involve collaborations with experts and additional analyses on focused part of the process (temperature and salinity variations in the brine, fine cheese milk composition in the standardization tank, heterogeneity of milk components, especially fat, in the standardization tank, etc.), in order to reinforce and refine the findings. Although this study focused on dry matter, many other indicators can be used to evaluate performance of industry. The methodology developed here has the advantage of being adaptable to the prediction of many other indicators (cheese yield, fat recovery in cheeses, etc.), offering promising prospects for challenging cheese plant economic performance, improving the understanding of cheese quality (texture, meltability, etc.) as well as identifying more sustainable processing routes (energy or water consumption, etc.).

## Funding sources

This project was supported by Fondation Institut Agro.

### CRediT authorship contribution statement

**Manon Perrignon**: Writing – original draft, Visualization, Methodology, Investigation, Conceptualization. **Mathieu Emily**: Writing – review & editing, Supervision, Methodology, Conceptualization. **Mélanie Munch**: Writing – review & editing. **Romain Jeanet**: Writing – review & editing, Supervision, Project administration, Funding acquisition, Conceptualization. **Thomas Croguennec**: Writing – review & editing, Supervision, Conceptualization.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.idairyj.2024.106143.

### Abbreviations

RF Random Forest

GBM Gradient Boosting Machine

SVM Support Vector Machine

MLP Multilayer Perceptron

LinMod Linear Regression

RMSE Root Mean Square Error

### References

Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable artificial intelligence (XAI) : What we know and what is left to attain trustworthy artificial intelligence. *Information Fusion, 99*, Article 101805. https://doi.org/10.1016/j.inffus.2023.101805

Bhagya Raj, G. V. S., & Dash, K. K. (2022). Comprehensive study on applications of artificial neural network in food process modeling. *Critical Reviews in Food Science and Nutrition, 62*(10), 2756-2783. https://doi.org/10.1080/10408398.2020.1858398

Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. *Proceedings of the fifth annual workshop on Computational learning theory*. https://doi.org/10.1145/130385.130401, 144-152.

Breiman, L. (2001). Random forests. *Machine Learning, 45*(1), 5-32. https://doi.org/10.1023/A:1010933404324

Decadt, H., Weckx, S., & De Vuyst, L. (2023). The rotation of primary starter culture mixtures results in batch-to-batch variations during Gouda cheese production. *Frontiers in Microbiology, 14*. https://doi.org/10.3389/fmicb.2023.1128394

Fenelon, M. A., & Guinee, T. P. (1999). The effect of milk fat on cheddar cheese yield and its prediction, using modifications of the van slyke cheese yield formula. *Journal of Dairy Science, 82*(11), 2287-2299. https://doi.org/10.3168/jds.S0022-0302(99)75477-9

Fröhlich-Wyder, M.-T., Bachmann, H.-P., & Schmidt, R. S. (2023). Classification of cheese varieties from Switzerland using machine learning methods : Free volatile carboxylic acids. *LWT, 184*, Article 115095. https://doi.org/10.1016/j.lwt.2023.115095

Freire, P., Freire, D., & Licon, C. C. (2024). A comprehensive review of machine learning and its application to dairy products. *Critical Reviews in Food Science and Nutrition, 0*(0), 1-16. https://doi.org/10.1080/10408398.2024.2312537

Friedman, J. H. (1991). Multivariate adaptive regression splines. *Annals of Statistics, 19*(1), 1-67. https://doi.org/10.1214/aos/1176347963

Friedman, J. H. (2002). Stochastic gradient boosting. *Computational Statistics & Data Analysis, 38*(4), 367-378. https://doi.org/10.1016/S0167-9473(01)00065-2

Garre, A., Ruiz, M. C., & Hontoria, E. (2020). Application of Machine Learning to support production planning of a food industry in the context of waste generation under uncertainty. *Operations Research Perspectives, 7*, Article 100147. https://doi.org/10.1016/j.orp.2020.100147

Golzarijalal, M., Ong, L., Neoh, C. R., Harvie, D. J. E., & Gras, S. L. (2024). Machine learning for the prediction of proteolysis in Mozzarella and Cheddar cheese. *Food and Bioproducts Processing, 144*, 132-144. https://doi.org/10.1016/j.fbp.2024.01.009

Guinee, T. P. (2021). Effect of high-temperature treatment of milk and whey protein denaturation on the properties of rennet-curd cheese : A review. *International Dairy Journal, 121*, Article 105095. https://doi.org/10.1016/j.idairyj.2021.105095

Guinee, T. P., & Fox, P. F. (2017). Chapter 13 - salt in cheese : Physical, chemical and biological aspects. In P. L. H. McSweeney, P. F. Fox, P. D. Cotter, & D. W. Everett (Eds.), *Cheese* (4th ed., pp. 317-375). Academic Press. https://doi.org/10.1016/B978-0-12-417012-4.00013-2

Hoerl, A. E., & Kennard, R. W. (2000). Ridge Regression : Biased Estimation for Nonorthogonal Problems. *Technometrics, 42*(1), 80-86. https://doi.org/10.2307/1271436.

Jaeggi, J. J., Wendorff, W. L., Romero, J., Berger, Y. M., & Johnson, M. E. (2005). Impact of seasonal changes in ovine milk on composition and yield of a hard-pressed cheese. *Journal of Dairy Science, 88*(4), 1358-1363. https://doi.org/10.3168/jds.S0022-0302(05)072802-2

Kern, C., Stefan, T., & Hinrichs, J. (2019). Multiple linear regression modeling : Prediction of cheese curd dry matter during curd treatment. *Food Research International, 121*, 471-478. https://doi.org/10.1016/j.foodres.2018.11.061

Khan, M. I. H., Sablani, S. S., Nayak, R., & Gu, Y. (2022). Machine learning-based modeling in food processing applications : State of the art. *Comprehensive Reviews in Food Science and Food Safety, 21*(2), 1409-1438. https://doi.org/10.1111/1541-4337.12912

Koay, H. V., Chuah, J. H., Chow, C.-O., & Chang, Y.-L. (2022). Detecting and recognizing driver distraction through various data modality using machine learning : A review, recent advances, simplified framework and open challenges (2014-2021). *Engineering Applications of Artificial Intelligence, 115*, Article 105309. https://doi.org/10.1016/j.engappai.2022.105309
Li, W., & Wu, Z. (2022). A methodology for dam parameter identification combining machine learning, multi-objective optimization and multiple decision criteria. *Applied Soft Computing*, **128**, Article 109476. https://doi.org/10.1016/j.asoc.2022.109476

Lopez, C., Camier, B., & Gassi, J.-Y. (2007). Development of the milk fat microstructure during the manufacture and ripening of Emmental cheese observed by confocal laser scanning microscopy. *International Dairy Journal*, **17**(3), 235-247. https://doi.org/10.1016/j.idairyj.2005.12.015

Ma, J., & Cheng, J. C. P. (2016). Identifying the influential features on the regional energy use intensity of residential buildings based on Random Forests. *Applied Energy*, **183**, 193-201. https://doi.org/10.1016/j.apenergy.2016.08.096

Ma, J., Ding, Y., Cheng, J. C. P., Jiang, F., Tan, Y., Gan, V. J. L., & Wan, Z. (2020). Identification of high impact factors of air quality on a national scale using big data and machine learning techniques. *Journal of Cleaner Production*, **244**, Article 118955. https://doi.org/10.1016/j.jclepro.2019.118955

Münch, M., Guillard, V., Gaucel, S., Destercke, S., Thévenot, J., & Buche, P. (2023). Composition-based statistical model for predicting CO2 solubility in modified atmosphere packaging application. *Journal of Food Engineering*, **340**, 111283. https://doi.org/10.1016/j.jfoodeng.2022.111283

Przybyl, K. (2024). Explainable AI: Machine learning interpretation in blackcurrant powders. *Sensors*, **24**(10), 10. https://doi.org/10.3390/s24103198

Shankarrao Patange, G., & Bharatkumar Pandya, A. (2022). How artificial intelligence and machine learning assist in industry 4.0 for mechanical engineers. *Materials*

*Today: Proceedings*, S221478532205369X. https://doi.org/10.1016/j.matpr.2022.08.201

Speiser, J. L., Miller, M. E., Tooze, J., & Ip, E. (2019). A comparison of random forest variable selection methods for classification prediction modeling. *Expert Systems with Applications*, **134**, 93-101. https://doi.org/10.1016/j.eswa.2019.05.028

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. *Journal of the Royal Statistical Society: Series B*, **58**(1), 267-288.

Tomsett, R., Braines, D., Harborne, D., Preece, A., & Chakraborty, S. (2018). *Interpretable to whom ? A role-based model for analyzing interpretable machine learning systems*.

Wan, X., Li, J., Xie, L., Wei, Z., Wu, J., Wah Tong, Y., Wang, X., He, Y., & Zhang, J. (2022). Machine learning framework for intelligent prediction of compost maturity towards automation of food waste composting system. *Bioresource Technology*, **365**, Article 128107. https://doi.org/10.1016/j.biortech.2022.128107

Wang, Z., Li, J., Rangaiah, G. P., & Wu, Z. (2022). Machine learning aided multi-objective optimization and multi-criteria decision making: Framework and two applications in chemical engineering. *Computers & Chemical Engineering*, **165**, Article 107945. https://doi.org/10.1016/j.compchemeng.2022.107945

Xia, X., Tobin, J., Subhir, S., Fenelon, M., Corrigan, B., McSweeney, P., & Sheehan, J. (2021). Effect of β-casein reduction and high heat treatment of micellar casein concentrate on the rennet coagulation properties, composition and yield of Emmental cheese made therefrom. *International Dairy Journal*, **126**, Article 105240. https://doi.org/10.1016/j.idairyj.2021.105240