## 5 Experiments and Results
### Validation
#### 5.3.1 Pareto-Optimal Validation  
[PERSON] et al. [28] claim that the MO-MPO algorithm returns Pareto-optimal solutions in multi-objective settings. This section provides a formal validation of that claim by analyzing whether the obtained results satisfy the conditions for Pareto optimality, using a gradient-based geometric approach [43]. Initially, only continuous input parameters are considered for the gradient-based validation; the analysis is later extended to include discrete input parameters as well. Let \(F_{i}:X\to\mathbb{R}\) denote the predictive loss of objective \(i\) (\(i=1,\ldots,n\)) on the continuous input space \(X\subset\mathbb{R}^{d}\), and let \(x\in X\) be a candidate solution. 1. Gradient calculation: The negative direction of the gradient of an objective loss function at the current solution indicates the direction of improvement. The first step is to calculate the negative gradient of the loss function for all objectives at the current solution \(x\), which is \(-\
abla_{x}F_{i}(x);\quad i=1,\ldots,n\). 2. Objective-wise improvement half-spaces: If a direction \(z\in X\) exists such that the dot product between \(z\) and the negative gradient \(-\
abla_{x}F_{i}(x)\) is greater than zero, it means that for objective \(i\), there is a direction of improvement. The set of all such directions constitutes the half-space \[H_{i}(x)\ =\ \big{\{}\,x+z\ \big{|}\ \ z\in X,\ \langle z,-\
abla_{x}F_{i}(x) \rangle>0\big{\}},\qquad i=1,\ldots,n.\] (10)
3. Intersection of half-spaces: Considering all objectives, if the intersection of half-spaces is not empty, any element of this intersection represents a direction that improves all objectives. Thus, \(x\) cannot be a Pareto-optimal solution. Let us define the intersection of half-spaces as \(A(x)=\bigcap_{i=1}^{n}H_{i}(x)\). * If \(A(x)\
eq\varnothing\), a single direction exists that decreases every \(F_{i}\) simultaneously; hence, \(x\) is not Pareto-optimal, as all objectives can be improved at the current solution. * If \(A(x)=\varnothing\), no direction exists that can improve all objectives at once; consequently, \(x\) satisfies the necessary condition for Pareto optimality in the continuous subspace. This reasoning is formally captured as follows:  
\[\text{If }A=\bigcap_{i=1}^{n}H(\
abla_{x}F_{i})(x)\
eq\varnothing\Rightarrow x \text{ is not Pareto optimal for the continuous part,} \tag{11}\]  
If \(A=\varnothing\) and \(\
abla_{x}F_{i}(x)\
eq 0\ \forall\ i=1,\ldots,n\Rightarrow x\) is Pareto optimal for the continuous part. Figure 6 illustrates the scenario for two objective loss functions, \(F_{1}\) and \(F_{2}\), where the intersection is not empty. Therefore, there exists a direction in which both objectives can improve. The lines shown in the figure represent the level curves of \(F_{1}\) and \(F_{2}\). Once the solution is validated in the continuous space, it must also be verified for categorical or discrete inputs (if present). For discrete variables, the validation is based on the continuity of their dependency. If a small perturbation in the continuous variablesdoes not affect the output of the discrete objectives, the solution can be considered Pareto optimal in the discrete part as well. This is formalized as follows:  
\[D_{i}(x+\epsilon z,y)=D_{i}(x,y)\quad\text{for}\;z\in A,\;\epsilon\ll 1,\;i=1, \ldots,n \tag{12}\]  
This provides numerical evidence of Pareto optimality for discrete objectives, although it does not constitute a full validation due to the approximate nature of continuous optimization. The validation procedure is implemented using linear programming techniques, and implementation-specific details are provided in the next section. The described validation methodology was applied to the pinion manufacturing dataset and the open-source multi-stage continuous-flow dataset. Given that certain process parameters are non-controllable, not all optimization results are expected to satisfy Pareto optimality.