## 7 Conclusions
### Limitations and Future Work  
Despite its strengths, the proposed approach has certain limitations. First, the performance of the RL agent may deteriorate when faced with highly sparse or noisy data, particularly if the surrogate model lacks sufficient fidelity. Second, the use of gradient-based Pareto validation relies on the assumption of continuous differentiability, which may not be valid for categorical or hybrid control parameters without the use of approximation techniques. Third, the current framework does not support online learning; all retraining must be conducted offline, limiting the system's adaptability in production environments that evolve over time.  
These limitations point to important directions for future research. While a preliminary out-of-distribution (OOD) detection mechanism was implemented (cf. Figure 8), a more comprehensive analysis is planned. Future work will explore integrating RL methods with online model updating strategies, allowing the agent to adapt continuously without requiring full retraining cycles. This capability is expected to enhance the system's responsiveness and robustness in dynamic production environments. Additionally, extending the optimization framework to handle mixed-variable inputs, combining both continuous and discrete control parameters such as machine modes or configuration settings, will enhance its applicability across a wider range of industrial use cases. These advancements are expected to improve both the robustness and flexibility of the framework in real-world, data-constrained manufacturing settings.  
Conceptualization, A.P. and N.Q.; methodology, A.P. and N.Q.; software, A.P., N.Q. and L.U.; validation, A.P., N.Q. and B.B.; formal analysis, A.P.; investigation, A.P. and N.Q.; resources, L.U.; data curation, L.U.; writing--original draft preparation, A.P., N.Q. and L.U.; writing--review and editing, A.P. and N.Q.; visualization, A.P. and L.U.; review and editing, D.W. and A.P.; supervision, R.H.S., B.B. and T.B.; project administration, R.H.S.; funding acquisition, not applicable. All authors have read and agreed to the published version of the manuscript.  
This research was sponsored by the Federal Ministry of Education and Research in Germany (BMBF) within the collaborative research project IRLeQuM (Industrial Reinforcement Learning for Quality Control of Massive Forming Processes), with funding reference code 02P20A073. The research project did not fund the APC.  
Not applicable Data Availability Statement: The data availability statement is provided. All data used in this study are publicly available and can be accessed at the following GitHub repository: [[https://github.com/akshayparanjape/multi_rl_MDPI](https://github.com/akshayparanjape/multi_rl_MDPI)]([https://github.com/akshayparanjape/multi_rl_MDPI](https://github.com/akshayparanjape/multi_rl_MDPI)) (accessed on 1 January 2020).  
The author would like to thank the industrial partner ESW ([[https://www.eswgroup.eu/en/](https://www.eswgroup.eu/en/)]([https://www.eswgroup.eu/en/](https://www.eswgroup.eu/en/)) (accessed on 30 May 2025) for their support in data generation and the industrial testing of the implemented method on-premise. Special thanks are extended to [PERSON] and [PERSON] from ESW for their contributions. The author would also like to thank the ARES team at IconPro GmbH for their ongoing feedback and valuable discussion sessions throughout this research.  
Authors [PERSON] and [PERSON] were employed by the company IconPro GmbH. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.