# Compensating Data Shortages in Manufacturing with Monotonicity Knowledge
## 1 Introduction  
Systematic decision making in manufacturing--such as finding optimal parameter settings for a manufacturing process--requires appropriate models for that process. In particular, such models have to be sufficiently accurate, and at the same time, sufficiently quick at evaluating. In principle, for many industrial processes, precise simulation models based on detailed physical modeling can be built. Yet, these so-called white-box models are typically too slow to be of any practical use in exploring the process parameter space and in eventually finding optimal process parameters (online and offline). In this respect, machine learning models can be very useful surrogates with short runtimes. Conventional machine learning models are purely data-based (so-called black-box models). Accordingly, the predictive power of such models is generally bad if the underlying training data \(\mathcal{D}=\{(\mathbf{x}_{l},t_{l}):l\in\{1,\dots,N\}\}\) are insufficient. Unfortunately, such data insufficiencies occur quite often, and they can come in one of the following forms: On the one hand, the available datasets can be too small and have too little variance in the input data points \(\mathbf{x}_{1},\dots,\mathbf{x}_{N}\). This problem frequently occurs in manufacturing [1] because varying the process parameters beyond well-tested operating windows is usually costly. On the other hand, the output data \(t_{1},\dots,t_{N}\) can be too noisy. Aside from potentially insufficient data, however, one often also has additional knowledge about the relation between the input variables and the responses to be learned. Such extra knowledge about the considered process is referred to as expert knowledge in the following. In [2], the interactions of users with software was tracked to capture their expert knowledge in a general form as training data for a classification problem. In [3], expert knowledge was used in the form of a specific algebraic relation between input and output to solve a parameter estimation problem with artificial neural networks. Such informed machine learning [4] techniques beneficially combine expert knowledge and data to build hybrid or gray-box models [5; 6; 7; 8; 9; 10; 11; 12], which predict the responses more accurately than purely data-based models. In other words, by using informed machine learning techniques, one can compensate data insufficiencies with expert knowledge. An important and common type of expert knowledge is prior information about the monotonicity behavior of the unknown functional relationship \(x\mapsto y(x)\) to be learned. A large variety of concrete application examples with monotonicity knowledge can be found in ([13], Section 4.1) and ([14], Section 1), for instance. The present article exclusively deals with regression under such monotonicity requirements. For classification under monotonicity constraints, see, e.g., [14; 15]. Along with convexity constraints, monotonicity constraints are probably the most intensively studied shape constraints [16] in the literature, and correspondingly, there exist plenty of different approaches to incorporate monotonicity knowledge in a machine learning model. See [17] for an extensive overview. Very roughly, these approaches can be categorized according to when the monotonicity knowledge is taken into account: in or only after the training phase. In the terminology of [4], this corresponds to the distinction between knowledge integration in the learning algorithm or in the final hypothesis. A lot of methods--especially from the mathematical statistics literature, such as [18; 19; 20; 21; 22; 23; 24]--incorporate monotonicity knowledge only after training. These articles start with a purely data-based initial model, which in general does not satisfy the monotonicity requirements, and then monotonize this initial model according to a suitable monotonization procedure, such as projection [18; 19; 20; 24], rearrangement [22; 23; 25] or tilting [21].