## 2 Materials and Methods
### Machine Learning Models  
As the focus of this work is to study the interaction of the optimizer and models and not to evaluate the power of any given ML model, we restrict ourselves to online linear regression, by performing a Sherman-Morrison update [40] to the inverted Hessian matrix every time a new data point is collected (i.e., a machine has finished a task). Should the model not yet have full rank, we opt to calculate the Moore-Penrose pseudo-inverse of the Hessian from scratch [41]. This results in a type of model that is fast to calculate and does not have any hyper-parameters that would require tuning and therefore impact results.  
As the focus of this work is to study the interaction of the optimizer and models and not to evaluate the power of any given ML model, we restrict ourselves to online linear regression. This choice is mainly because linear regression requires no parameters whose tuning could impact the results. With the Sherman-Morrison update [40], a well established and fast technique exists with witch to update the model when new training data are available. New training data become available whenever a machine finishes a task. The generated input vector (independent variables) for the model consists of the aggregated material properties, the time at which the action started, as well as the material properties, and start and finishing times of the _previous_ action performed on this machine (this allows the model to capture simple batching and pipe-lining behaviors). The dependent variable is then simply the time it took to perform the current action. Every machine is assigned an independent model, so transfer learning possibilities from machines of similar types are not in the scope of this work.  
In this work, we deliberately assume \"cold start\" conditions. Only the production layout is known, as well as the buffer capacities \(b_{1}\) and \(b_{2}\). The machine learning models predicting machine/setup times are initialized with the same constant value \(c=45\) s that is within an order of magnitude for all actual processing times as not to intentionally mislead them and allow for the generation of initial solutions.  
Additionally, when only a few data points are available, the number of training data points might be smaller than the degrees of freedom in the model or several input features might be co-linear to each other. In this case, the linear system is under-determined and we opt to calculate the Moore-Penrose pseudo-inverse of the Hessian matrix from scratch [41]. This essentially chooses from all possible linear models with equal quality, the model where the coefficients display the smallest Euclidean norm (i.e., the coefficients closest to zero). Since online updating the pseudo-matrix is more complex than the Sherman-Morrison formula, and this edge case is mainly relevant in stages where the number of data points is small and training is fast anyway, we recalculate this matrix whenever the calculation of the actual Hessian matrix is not possible.  
A note is made that given enough data, the linear models should be able to learn the _true_ behavior of all machines in this scenario (\(x_{1}\cdot x_{2}\) provides a singular input variable) and more complex structures (artificial neural networks, random forests, etc.) should not be required.