## 3 Process Simulation
### Machine Learning-Based Process Model  
The first step in the proposed framework involves training a surrogate model of the manufacturing process. The high-fidelity simulation developed using Simufact (cf. Section 3.1) exhibits substantial inference latency, rendering it impractical for direct use in RL agent training. To address this, machine learning-based surrogate models were employed to approximate the simulation outputs and significantly reduce inference time. These models were trained on synthetic data generated from the simulation environment. The dataset comprises four input parameters and fourteen output variables, as summarized in Table 2. Among the outputs, twelve correspond to force measurements recorded during the process, while the remaining two, flange thickness and flange radius, serve as key quality indicators of the manufactured pinion. An illustration of the pinion geometry is provided in Figure 2.  
A total of 600 samples were generated using the simulation tool. The surrogate model was designed to predict the final flange thickness and flange radius based on the four input parameters listed in Table 2. The twelve force-related outputs were excluded from training, as they reflect process dynamics rather than quality characteristics. The process model \begin{table}
\begin{tabular}{l c c} \hline
**Factor** & **Symbol** & **Levels** \\ \hline Machine stiffness & \(k\) & Low, Medium, High \\ Stroke & \(h\) & \(h_{1}\), \((h_{1}+\Delta h/2)\), \(h_{2}\) \\ Initial tool temperature & \(T_{\text{tool\_0}}\) & 20 \({}^{\circ}\)C, 100 \({}^{\circ}\)C, 200 \({}^{\circ}\)C \\ Initial flange thickness & \(b_{\text{FL\_0}}\) & 3.0 mm, 3.5 mm, 4.0 mm \\ \hline \end{tabular}
\end{table} 
Table 1: Experimental design parameters and their corresponding levels. \begin{table}
\begin{tabular}{l l} \hline
**Process Parameter [Unit]** & **Description** \\ \hline Stroke & Stroke applied during the forming process \\ Tool temperature & Temperature of the tool during forming \\ Machine stiffness & Machine stiffness at room temperature \\ Flange thickness & Initial thickness of the flange \\ Output Forces & Twelve force values recorded at equal stroke intervals \\ Output Flange thickness & Final flange thickness of the product \\ Output Flange radius & Final flange radius of the product \\ \hline \end{tabular}
\end{table} 
Table 2: Description of parameters simulated using Simufact. Figure 2: Image of a manufactured pinion gear with integrated ball bearings and internal splines, designed for high-performance mechanical systems. Image © ESW GmbH, used with permission. [[https://www.esw-group.eu/fertigungsverfahren/praegetechnik/](https://www.esw-group.eu/fertigungsverfahren/praegetechnik/)]([https://www.esw-group.eu/fertigungsverfahren/praegetechnik/](https://www.esw-group.eu/fertigungsverfahren/praegetechnik/)) (accessed on 30 March 2025). comprises two independent feed-forward neural networks, each tasked with predicting one of the two target quality metrics: flange thickness and flange radius. Neural networks were selected over tree-based regression models due to their lower inference latency and capability to compute gradients with respect to input parameters, which is an essential feature required for validating Pareto-optimality. Hyperparameter tuning was conducted using the random search method implemented via KerasTuner. Each model was trained for a maximum of 500 epochs with a learning rate of 0.001, subject to early stopping based on validation loss. The final models demonstrated high predictive performance, achieving \(R^{2}\) scores of 0.97 for flange radius and 0.99 for flange thickness on a 30% test split. These trained models were subsequently used to emulate the pinion manufacturing environment for training the RL agents. While surrogate models provide efficient approximations of complex physical processes, their accuracy is inherently limited by the quality and scope of the training data. They may also fail to capture nonlinearities, interactions, or dynamic behaviors present in real-world systems.