# Compensating Data Shortages in Manufacturing with Monotonicity Knowledge
## 2 Semi-Infinite Optimization Approach to Monotonic Regression
### Algorithm and Implementation Details  
In the following, our adaptive discretization algorithm is described in detail. As has already been pointed out above, it is a variant of the general algorithm developedby ([39], Section 2), and it is explained after Algorithm 1 how our variant differs from its prototype [39]. Choose a coarse (but non-empty) rectangular grid \(X^{0}\) in \(X\). Set \(k=0\) and iterate over \(k\). 1. Solve the \(k\)th discretized problem (11) to obtain optimal model parameters \(\mathbf{w}^{k}\in W\). 2. Solve the \((k,j)\)th lower-level problem (12) approximately for every \(j\in J\) to find approximate global minimizers \(\mathbf{x}^{k,j}\in X\). Add those of the points \(\mathbf{x}^{k,j}\) for which substantial monotonicity violations occur, i.e., for which \[g_{j}(\mathbf{w}^{k},\mathbf{x}^{k,j})<-\varepsilon_{j},\] to the current discretization \(X^{k}\) and go to Step 1 with \(k=k+1\). If for none of the points \(\mathbf{x}^{k,j}\) substantial monotonicity violations occur, go to Step 3. 3. Check for monotonicity violations on a fixed, fine rectangular reference grid \(X_{\text{ref}}\subset X\). If there are no such violations, that is, if \[g_{j}(\mathbf{w}^{k},\mathbf{x})\geq-\varepsilon_{j}\] for all \(j\in J\) and \(\mathbf{x}\in X_{\text{ref}}\), then terminate. Else, for every direction \(j\) with violations, add the reference grid point \(\mathbf{x}^{k,j}_{\text{ref}}\) with the largest violation to \(X^{k}\) and go to Step 1 with \(k=k+1\). **Algorithm 1** Adaptive discretization algorithm for monotonic regression. In contrast to [39], the algorithm above does not require exact solutions of the (non-convex) lower-level problems. Indeed, Step 2 of Algorithm 1 only requires finding an approximate solution. Also, slight constraint violations are tolerated (Step 2 and 3), and a feasibility check on a reference grid (Step 3) is performed before termination. Without the feasibility check on the reference grid, it could happen that the algorithm--because of the merely approximate solutions of the lower-level problems--terminates at models \(\tilde{y}_{\mathbf{w}^{k}}\) which do not satisfy the imposed monotonicity constraints sufficiently well. Another difference to the algorithm from [39] is that there there are several lower-level problems in each iteration in this work and not just one, because monotonicity is enforced in multiple coordinate directions in general. In our specific applications, the parameters inherent in Algorithm 1 were chosen as follows. The degrees \(m\) of the polynomial models in this work were chosen as the largest possible values that did not result in an overfit, because increasing \(m\) enhances the model's accuracy in general. In this respect, the number of model parameters was allowed to exceed the number of data points (\(N_{m}\geq N\)), since the constraints represent additional information supplementing the data. As for the parameter \(r\) in (5), one only has to make sure that it is so large that the resulting box constraints in the discretized problems (11) actually do not restrain the solutions \(\mathbf{w}^{k}\) (Step 1 of Algorithm 1). In other words, \(r\) should be so large that relaxing or even dropping the pertaining box constraints does not improve the minimizer computed for (11) anymore. In the specific applications considered here, \(r=10^{5}\) turned out to meet this requirement. As for the tolerances \(\varepsilon_{j}\) (Steps 2 and 3 of Algorithm 1), a monotonicity violation of \(1\,\%\) of the ranges covered by the in- and output training data was allowed for:  
\[\varepsilon_{j}=0.01\frac{\max_{l=1,\ldots,N}t_{l}-\min_{l=1,\ldots,N}t_{l}}{ \max_{l=1,\ldots,N}x_{l,j}-\min_{l=1,\ldots,N}x_{l,j}}. \tag{14}\]  
And finally, concerning the reference grid \(X_{\text{ref}}\) in the finalization step (Step 3 of Algorithm 1), twenty values per input dimension, equidistantly distributed from the lower to the upper bound along each direction, were used.