## 4 Machine Learning Applications for Production Machines Real Time Control  
Real-time control of manufacturing machines is a further essential element of production machines to achieve repeatable and stable processes with high-quality parts. Applications range from accurate positioning of machine axes to adaptive control of process parameters as the laser power during laser welding. Today's industrial control systems are mainly based on highly consolidated solutions, as for example PID controllers for axis control. The simple structure, ease of implementation and long experience in parameter tuning lead to an established technology for control applications. However, there were many further developments in the controller design and automated parameter tuning [63]. AI-based methods have been increasingly adopted in recent years to optimize control parameters tuning or to develop new types of control methods. In general, machine learning methods can be divided into supervised, unsupervised and reinforcement learning (RL) [64]. Supervised learning trains for example an NN by reducing the mean squared error between predictions and the corresponding labelled solution. In unsupervised learning, no information about the ground truth is available. Unsupervised learning often consists in clustering or reducing data dimensionality by grouping correlated data points. In contrast to supervised and unsupervised learning, RL forms a semi-supervised learning approach, where an agent interacts with the environment and a reward is given for performed actions. The algorithms explore the environment by executing actions and receiving rewards for the actions and decisions taken. The optimal execution order of the actions, the policy, is not known and will be determined during the learning process. A good introduction into Deep RL is given by [PERSON] et al. [65]. RL can be further subdivided into dynamic programming, Monte Carlo methods and temporal difference methods [66]. Regarding PID parameter tuning, RL has been successfully applied to find suitable controller parameters. Besides traditional approaches such as trial and error or Ziegler-Nichols [67], [PERSON] et al. [68] utilized an actor critic RL approach to determine the PID parameters. The results demonstrated the algorithm's ability to track non-linear systems while ensuring adaptability and robustness. Besides parameter tuning, RL also experiences an increased adoption within process control applications. Table 3 lists the currently most adopted RL algorithms for control applications. \begin{table}
\begin{tabular}{c c c} \hline
**Modelling Type** & **Modelling Methods** & **Applications** \\ \hline Analytical & Analytical & Well-known and simpler models \\ \hline Numerical & FEM, FVM & Complex processes, limited domain size and computational expensive \\ \hline \multirow{6}{*}{Empirical/Data-driven} & Statistical analysis, e.g., linear regression & Well-known, low computational cost, for less complex problems \\ \cline{2-3}  & Neural network & Mapping of nearly arbitrary relations, with sufficient experimental data \\ \cline{2-3}  & Gaussian processes & Process parameter optimization, difficulties in handling high-dimensional problems, small amounts of available data \\ \cline{2-3}  & Physics-Informed Neural Networks & Process modelling, small amounts of data \\ \cline{2-3}  & Fourier Neural Operator & Mesh invariant modelling methods. Novel approach needs to be tested for real use cases \\ \hline \end{tabular}
\end{table} 
Table 2: Overview of sensor fusion algorithms. [PERSON] et al. [66], [PERSON] et al. [63] and [PERSON] et al. [69] recently published reviews of machine learning and process control methodologies. In particular, [PERSON] et al. [63] compared reinforcement learning with model predictive control algorithms. For complex and dynamical systems, model predictive controllers (MPC) represent an effective alternative to PID controllers.