# Allocating Orders to Printing Machines for Defect Minimization: A Comparative Machine Learning Approach

Angelos Angelopoulos, Anastasios Giannopoulos, Sotirios Spantideas, Nikolaos Kapsalis, Chris Trochoutsos, Stamatis Voliotis, Panagiotis Trakadas

► To cite this version:

Angelos Angelopoulos, Anastasios Giannopoulos, Sotirios Spantideas, Nikolaos Kapsalis, Chris Trochoutsos, et al. Allocating Orders to Printing Machines for Defect Minimization: A Comparative Machine Learning Approach. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.79-88, 10.1007/978-3-031-08337-2_7 hal-04668646

HAL Id: hal-04668646

https://inria.hal.science/hal-04668646v1

Submitted on 7 Aug 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Allocating Orders to Printing Machines for Defect Minimization: A Comparative Machine Learning Approach

Angelos Angelopoulos¹, Anastasios Giannopoulos¹, Sotirios Spantideas¹, Nikolaos Kapsalis¹, Chris Trochoutsos², Stamatis Voliotis¹ and Panagiotis Trakadas¹

¹ National and Kapodistrian University of Athens, Psachna, Evia, 34400, Greece
a.angelopoulos@uoa.gr, angianno@uoa.gr, sospanti@uoa.gr, ncapsalis@pms.uoa.gr, svoliotis@uoa.gr, ptrakadas@uoa.gr

² Pressious Arvanitidis, Chalandri, Athens, 15232, Greece
chtrox@pressious.com

**Abstract.** Zero-Defect Manufacturing (ZDM) is continuously emerging as the most critical target of the Industry 4.0 era. Minimization of the defected products in the industrial production chain can contribute towards significant improvements of the operational costs, the production efficiency and speed, as well as the environmental footprint. This work proposes a Machine Learning (ML) based scheme for intelligently and proactively allocating orders to printing machines, so as to ensure minimization of the defected products. Based on a historical dataset extracted by a printing company, ten supervised learning regression models were trained to estimate the machine-specific defect ratio of new orders, given their multi-feature requirements. To optimize the machine selection policy, several widely-used ML schemes were compared in terms of their performance on unseen data samples. Extensive simulations were carried out to stabilize the hyperparameters of the ML models, including single-model, ensemble and Deep Learning based regressors. Results showed that Multi-Layer Perceptron (MLP) outperform the rest of the benchmarking regressors in accurately predicting the defect rate of each machine, with the ensemble methods presenting also enhanced accuracy. Finally, the misclassification ratio of the proposed algorithm was assessed by quantifying the number of optimally allocated orders to printing machines, exhibiting that the majority of orders are correctly classified.

**Keywords:** Machine Learning, Industry 4.0, Zero-Defect-Manufacturing, Supervised Learning.

## 1 Introduction

From the early eras of the industrial revolution, the problem of defected manufacturing (due to human errors, material deficiencies or manufacturing processes misconfigurations) still remains unsolved. In the framework of Industry 4.0, conventional methods for reducing defected products are gradually converted towards proactive and predic-
tive measures in the industrial production chain, involving the digitization of the man-
ufacturing process [1]. The emerging concept of Zero-Defect Manufacturing (ZDM)
primarily aimed to detect the defects in the production chain and identify required cor-
rective actions to minimize the loss of the raw materials and the associated cost, while
in parallel optimizing the resource allocation and management in the production chain
[2]. Importantly, the wasted resources involved in the production of defected products
have additionally a significant contribution to the environmental footprint of the man-
ufacturing industry.

Although the digital revolution was a crucial step towards expediting the digitaliza-
tion of the manufacturing process, the emergence of Artificial Intelligence (AI) solu-
tions is considered the substantial breakthrough towards ZDM. AI-assisted solutions
and Machine Learning (ML) algorithms have been developed to monitor and manage
the manufacturing production, targeting to optimize the available resources and the us-
age of raw materials [3]. The extensive utilization of AI/ML methods has therefore
boosted the manufacturing industry towards decreasing the occurrence of defected
products, eliminating the wasted resources and costs of the defects and minimizing its
environmental footprint [2].

The majority of the ML algorithms that are typically employed in the manufacturing
domain require vast amounts of historically gathered data. Through data processing and
analysis, ML models are trained based on a specific dataset and are then able to provide
estimations and predictions or find hidden patterns, complex correlations and irregular-
ities [4], [5]. The ML algorithms can be categorized in: (i) Supervised Learning (SL)
algorithms that require labeled data in the form of input and labeled output data pairs.
SL algorithms can be used for classification or regression problems, depending on
whether the output takes discrete values (classes) or continuous values; (ii) Unsuper-
vised Learning (UL) methods that do not require labeled output dataset; instead, they
use the input data to find clusters that exhibit similar features; (iii) Reinforcement
Learning (RL) algorithms that are based on a trial and error process of an agent inter-
acting with the environment [6].

It is worth noting that although the ML deployment methods are well-established in
the manufacturing domain, ML models abide by the following constraints: (i) as afore-
mentioned, these algorithms require huge amounts of data in order to provide efficient
solutions and predictions; (ii) the quality of the collected data has a direct impact on the
training procedure of ML models (for instance well-known overfitting or underfitting
issues [4]). In this way, each trained ML model is data-dependent and may possibly
lead to inaccurate estimates in case of insufficient or noisy dataset; (iii) ML models that
are trained to optimize the parameters of an individual machine or to enhance the effi-
ciency of a specific manufacturing process cannot be easily generalized to other pieces
of equipment or different processes.

Offset printing is a widely-used printing process for multiple products including, amongst others, newspapers, magazines, labels and books. The offset printing process consists of three phases, namely the pre-press, press and post-press. In each one of these three phases, several (raw, organic, chemical and recycled) materials are used, including paper, water, ink, aluminum, alcohol solutions, having a direct impact on the environmental footprint of the printing process. It is estimated that over the course of a year,
the production-related CO₂ emissions of a regular offset printing press amount to around 7,100 metric tons for 36 million printed sheets per year, while the paper waste accounts for 230 metric tons [7].

Despite the efforts spent on the formalization and standardization of the different procedures across the various offset printing processes, as well as on providing incentives to the personnel to provide products of high quality, defective products appear along the manufacturing production chain. The main issues that are currently faced within the offset printing production chain include: (i) lack of digitalization of physical processes that would enable real time decision making; (ii) plethora and diversity of production characteristics that do not allow direct process standardization; (iii) defected products are identified at the latest stages of the offset printing production chain, resulting in overdue a-posteriori manual corrective actions; (iv) the environmental footprint of the offset printing industry is considerable, since various waste materials are produced throughout the production chain [8].

The present paper outlines the implementation and comparison of SL algorithms in a labeled dataset from the offset printing industry, targeting to demonstrate the potency of ML-assisted methods towards ZDM. The purpose of the SL modeling is to link the specific characteristics of the received customer orders with the amount of defects in the printing process. The accurate correlation of order features with the defected products will result in beneficial machine selection policy, which in turn will contribute to the enhancement of the production efficiency, the minimization of defected products and the reduction of the company's environmental footprint. In summary, the contributions of this work can be identified as: (i) the exploitation of historical knowledge extracted by a real printing industry environment to obtain accurate and machine-specific supervised learning models, (ii) the proposition of a machine selection policy targeting at minimizing the printing defects by proactive allocation of orders to machines, (iii) the training, testing and comparison of multiple regression models (namely 10) to properly map the regressors to machine-specific datasets, (iv) the implementation of both single-model and multi-model (or ensemble) regressors to investigate potential accuracy benefits in using combinations of multi-predictors and (v) the utilization of both classical ML and Deep Learning based schemes to estimate the printing accuracy of multi-feature orders.

# 2 Methodology and Algorithmic Approach

## 2.1 Available Dataset

The methodology presented in the current work is based on a labelled dataset, incorporating both historical (last 3 years) and recently collected data that will allow accurate training and verification of the developed ML tools, as well as comparison of the results with the actual historical events. Each of the collected data samples follows the press process of a particular printing order with specific characteristics. In order to maintain a balanced dataset amongst the five printing machines, 10K orders for each individual machine were selected as a representative subset of raw data, leading to a total of 50K historical samples. The order-specific characteristics include:
i. Associated machine ID: The ID of the machine (1 – 5) that the particular order was forwarded for printing.

ii. Quantity: The amount of paper pieces requested in this specific order. *Quantity* takes integer values extending to up to 1000 pieces, depending on the type of the printing assignment (e.g. newspaper, poster, etc.).

iii. *Quality*: The quality of the paper associated with the specific printing order. The *Quality* parameter takes discrete values (1, 2 or 3) depending on the properties of the requested paper. In this context, class 1 stands for ‘Velvet’ paper quality (the most-frequently used), whereas categories 2 and 3 represent the ‘Uncoated’ and ‘Illustration/Gloss’ paper quality respectively.

iv. Color: This categorical variable denotes the color requirements of the particular printing assignment. The most requested color requirement is the 4-color printing (class 1), followed by 4+1 color printing (class 2 involving the use of gold and/or silver colors) and grayscale printing (class 3 entailing only black and white colors).

v. *Ink*: The estimated consumed ink level per paper piece (typically 0.1 to 1 gr).

vi. *Type*: Type 1 designates that the order shall follow the ‘Book’ printing process, type 2 describes the ‘Poster’ requirements, while type 3 determines the ‘Journal’ specifications.

vii. *Accuracy ratio*: The amount of satisfactory printed paper pieces are reported against the required *Quantity*. The ratio between them indirectly reveals the percentage of defected products, ranging from 0 – 1.

## 2.2 Machine Selection Framework

The proposed SL modeling framework involves the training of machine-specific ML models based on the described dataset, i.e. the orders that have been historically executed in the five printing machines. For these purposes, the order characteristics (*Quantity*, *Quality*, *Color*, *Ink* and *Type*) are considered as the input features of the ML modeling methods, while the *Accuracy* ratio is the labeled output variable.

Once these machine-specific ML models have been adequately trained and validated via testing data (not encountered during the training), they can be used for inference purposes on previously unseen orders. Fig. 1 illustrates the machine selection framework. A new arriving order is accompanied by specific order characteristics. Then, these order features are used as input parameters to all five ML models for each individual machine, enabling the prediction of the machine-specific *Accuracy* ratio for the new order. The ML model of the machine that exhibits the maximum *Accuracy* ratio specifies the selection of the machine that will yield the minimum number of defects during the printing process of the particular order.
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** ML-assisted framework for selecting the best printing machine. During the inference phase, the features of the new order pass through all machine-specific regression models and the printing machine exhibiting the best estimated Accuracy is selected.</figcaption></figure>

## 2.3 ML Algorithms for Accuracy Prediction

In principle, ML algorithms can be employed to create a regression model for each machine, using historical order-specific features as inputs and the respective accuracy score (good amount divided by the total amount) as labels. To find the best regressor for each machine, several ML regression algorithms were employed and tested on each machine-specific datasets [4], [9] and [10]:

i. *Linear Regression (LR)* assumes that the target variable can be predicted through a weighted linear combination of the *N* independent variables. It results into a linear model with coefficients $w_i$ ($i$ = 1, 2, ..., *N*), graphically represented by a straight line that minimizes the residual sum of squares between the actual and predicted *Accuracy* values.

ii. *Decision Tree (DT)* regressor is a non-parametric ML model that decides where and how to split the feature space, so as to ensure high information entropy in each decision area. As a result, a rule-based tree is constructed based on the previous separations of the feature space until no further splitting is required.

iii. *Support Vector Regression (SVR)* uses an $\varepsilon$-insensitive tube to allow for more flexibility in errors. Unlike *Linear Regression*, *SVR* penalizes only the data points that have at least $\varepsilon$ distance from the fitting curve. The target of the *SVR* is to fit the best line within a threshold value ($\varepsilon$), reflecting the distance between the hyperplane and boundary line.

iv. *Stochastic Gradient Descent (SGD)* regression is actually an optimization method that produces a linear model by minimizing a regularized empirical loss. The method relies on the calculation of the loss gradient of each sample, whereas the model parameters are gradually updated depending on the selected learning rate.

v. *Random Forest (RF)* regression is an ensemble learning model that is constructed by fitting *N* decision trees in *N* subsets of the whole dataset. Given a
new data point, the final prediction is obtained by averaging the individual
prediction of each tree.

vi. AdaBoost regression is a meta-estimator that creates multiple copies of a base estimator (for instance, *Decision Tree* regressor already fitted on the original dataset). Each copy adjusts its weights according to the prediction error of the base estimator and new predictions can be performed by inferring the ensemble of these models.

vii. Bagging ensemble method is a meta-estimator constructed by randomly picking *N* different subsets of the original dataset, each one being trained on an initial base estimator. The individual predictions of the base regressors are the averaged to form the final output.

viii. *Voting regressor* is used to combine previously trained ML regressors (even ensemble estimators) in order to build a powerful collaborative model. New predictions are derived by consulting each individual regressor and keeping the average output value.

ix. *Stacking* regressor employs the outputs of the individual selected estimators and uses them as input in an additional regression model to estimate the final output.

x. *Multi-Layer Perceptron (MLP)* or *Artificial Neural Networks (ANNs)* is a supervised learning scheme using multiple stacked layers (input, hidden and output). Each layer consists of multiple units (neurons) that compute the weighted sum of all the previous-layer neurons and then apply an activation function (e.g. softmax, ReLu, etc.). MLP properly adjusts the weights of the network so as to ensure minimization of the prediction error (or loss function).

# 3 Numerical Results

For each of the following models, a 90/10% training/test split was performed for each machine-specific dataset. Notably, to account for large variations of the features values during the training, the scalar order features (*Ink* and *Quantity*) were scaled before training so as to range between [0,1] using Min-Max scaler. All algorithmic implementations were conducted in Python 3.0 using the scikit-learn ML library [11].

## 3.1 Training of the Multi-Layer Perceptron

The performance of the MLPs is strongly influenced by the network depth (number of hidden layers) and the considered learning rate. In this section, the optimal hyper-parameters of the MLP are fine-tuned for each machine. To stabilize the values of the learning parameters, we conducted extensive simulations with varying learning rates ($\alpha$ = 0.1, 0.01, 0.001, 0.0001, 0.00001) and number of hidden layers ($H$ = 1, 2, 3, 4, 5). The number of neurons in the $i^{th}$ hidden layer is $2^i \times 10$. For each ($\alpha$, $H$) combination, the validation metric was the mean squared error (MSE) between the actual and predicted *Accuracy* on the testing samples. The optimal ($\alpha$, $H$) combination corresponded to the minimum convergence loss. Fig. 2 illustrates the training loss
curves for the optimal $(\alpha, H)$ configuration for each individual machine (Fig. 2A-E), as well as the validation MSE per machine (Fig. 2F).

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Training insights of ANNs. **A**–**E**. Training loss curves for machines 1–5 employing the optimal learning rate and number of hidden layers. **F**. Performance of the derived models in terms of validation MSE on the testing data.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Impact of the number of estimators on the validation MSE loss of the Ensemble Learning Schemes (RF, AdaBoost, Bagging regressors).</figcaption></figure>

## 3.2 Ensemble Learning Techniques

In this section, we illustrate the convergence validation loss of the ensemble learning algorithms as a function of the number of the estimators used in the ensemble models.
Specifically, we investigated the impact of the number of estimators on the performance of *RF*, *AdaBoost* and *Bagging* regressors. Fig. 3 depicts the validation MSE loss for varying (10 – 200) number of individual estimators in each printing machine. The optimal number of estimators for (*RF*, *AdaBoost*, *Bagging*) was (180, 10, 180), (180, 10, 180), (80, 10, 80), (180, 10, 180) and (180, 10, 170) for machines 1 – 5, respectively. Generally, we noticed that *RF* and *Bagging* regressors become more accurate with increasing number of estimators, whereas *AdaBoost* was insensitive to the considered dataset splits.

### 3.3 Comparison between ML methods

In this section, the regression performance is compared across 10 machine-specific ML models, namely *Linear Regression*, *Decision Tree*, *Support Vector Regression*, *Stochastic Gradient Descent*, *Random Forest*, *AdaBoost*, *Bagging*, *Voting*, *Stacking* and *Multi-Layer Perceptron* (ANNs). Note that, to guarantee optimal performance in each individual model, ensemble learning schemes have been configured in their optimal number of estimators (see section 3.2), whereas the Neural Networks have been inferred using the optimal $(\alpha, H)$ combination (see section 3.1). In addition, meta-estimators *Voting* and *Stacking* have been configured with the top three regressors in terms of their prediction accuracy, namely the *AdaBoost*, *Bagging* and *Random Forest* regressors.

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Comparison of all developed ML algorithms in terms of validation MSE between actual and predicted *Accuracy* values.</figcaption></figure>

Fig. 4 shows the MSE calculated between the ground-truth and model derived *Accuracy* scores across the 1K testing samples. Evidently, *ANNs* and *AdaBoost* constantly outperform the rest of the models in each machine, achieving a validation MSE of ~0.01. Closely to this performance were the *Stacking* and *Voting* ensemble schemes, whereas the worst performance was observed for *Decision Tree* regressors. Conclusively, *ANNs* or ensemble schemes can be used to guide the machine selection policy presented in Fig. 1, given their excessive *Accuracy* fitting in the testing data.
## 3.4 Evaluation of the proposed Machine Selection Policy

Here we evaluate the machine selection policy presented in Fig. 1. Firstly, we use the ANN models configured in their optimal hyperparameters, given their superiority in providing more precise *Accuracy* predictions against the other ML methods (see section 3.3). The evaluation dataset was composed by additional 100 orders, containing 20 order for each machine (i.e. 20× 5) that have perfectly executed at the respective machine (*Accuracy* values > 98.5%). In this sense, it is known a-priori that the first 20 samples have to be classified in machine 1, the second 20 samples to machine 2, and so on. The goal of the presented evaluation scheme is (i) to validate whether the proposed machine selection policy (see Fig.1) properly classifies the orders to machines and (ii) to quantify the true positive rate (i.e. number of correctly allocated orders to the respective machine) of each machine. Fig. 5 illustrates the confusion matrix of the classification scheme, with the element (i, j) reflecting the number of orders the were predicted to be executed in machine j when the optimal selection was the machine i. As evidently shown by the non-diagonal elements of each row, a misclassification rate of 3/20, 6/20, 0/20, 5/20 and 4/20 was derived for machines 1-5, respectively.

<figure><img src="image_7.png" /><figcaption>**Fig. 5.** Confusion Matrix of the Machine Selection policy on the evaluation dataset. Diagonal elements represent the number of orders that were correctly classified to the respective machine.</figcaption></figure>

# 4 Conclusion

In this paper, an ML-based policy for assigning orders to printing machines is proposed. Using a real dataset from an offset printing company, we trained a machine-specific multi-feature regression model to accurately predict the defect ratio for new orders, targeting at proactively selecting the proper machine to execute a given order. In addition, ten different regression schemes were contrasted for comparative purposes, including classical ML, ensemble and Deep Learning methods. Towards minimizing the defects, numerical outcomes confirmed that the proposed machine selection scheme
can efficiently provide order accuracy predictions, with the ANN and ensemble meth-
ods outperforming the rest of the regression algorithms. Results can be used for further
validating experiments as benchmark.

Acknowledgment

This work has been partially supported by the project Offspring, under the open call of EFPF project, funded by the European Commission under Grant Agreement number 825075 through the Horizon 2020 program.

References

1. Dalenogare, L. S., Benitez, G. B., Ayala, N. F., & Frank, A. G.: The expected contribution of Industry 4.0 technologies for industrial performance. *International Journal of Production Economics, 204*, 383-394 (2018).

2. Psarommatis, F., Sousa, J., Mendonça, J. P., & Kiritsis, D: Zero-defect manufacturing the approach for higher manufacturing sustainability in the era of industry 4.0: a position paper. *International Journal of Production Research*, 1-19 (2021).

3. Angelopoulos, A., Michailidis, E. T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S. and Zahariadis, T: Tackling faults in the industry 4.0 era—a survey of machine-learning solutions and key aspects. *Sensors*, 20(1), 109 (2020).

4. Caiazzo, B., Di Nardo, M., Murino, T., Petrillo, A., Piccirillo, G., & Santini, S.: Towards Zero Defect Manufacturing paradigm: A review of the state-of-the-art methods and open challenges. *Computers in Industry, 134*, 103548 (2022).

5. Trakadas, P., Simoens, P., Gkonis, P., Sarakis, L., Angelopoulos, A., Ramallo-González, A.P., Skarmeta, A., Trochoutsos, C., Calvo, D., Pariente, T. and Chintamani, K.: An artificial intelligence-based collaboration approach in industrial iot manufacturing: Key concepts, architectural extensions and potential applications. *Sensors*, 20(19), p.5480 (2020).

6. Kaloxylos, A., Gavras, A., Camps Mur, D., Ghoraishi, M., & Hrasnica, H. (2020).: AI and ML—Enablers for beyond 5G networks. *Zenodo, Honolulu, HI, USA, Tech. Rep.*

7. Amon-Tran, I., Anayath, R., Pai, A. S., Kamath, N., Shenoy, S., & Harikrishnan, A.: An approach to minimize carbon footprint for an environmental friendly printing by optimizing an offset machine in a printing facility. *Procedia-Social and Behavioral Sciences, 37*, 514-527 (2012).

8. Villalba-Diez, J., Schmidt, D., Gevers, R., Ordieres-Meré, J., Buchwitz, M., & Wellbrock, W.: Deep learning for industrial computer vision quality control in the printing industry 4.0. *Sensors, 19*(18), 3987 (2019).

9. Angelopoulos, A., Giannopoulos, A. E., Kapsalis, N. C., Spantideas, S. T., Sarakis, L., Votiotis, S., & Trakadas, P.: Impact of Classifiers to Drift Detection Method: A Comparison. In *International Conference on Engineering Applications of Neural Networks* (pp. 399-410). Springer, Cham (2021, June).

10. Dengler, S., Lahriri, S., Trunzer, E., & Vogel-Heuser, B. (2021). Applied machine learning for a zero defect tolerance system in the automated assembly of pharmaceutical devices. *Decision Support Systems, 146*, 113540.

11. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, E.: Scikit-learn: Machine learning in Python. *the Journal of machine Learning research, 12*, 2825-2830 (2011).