# Generation of Data-Driven Expected Energy Models for Photovoltaic Systems
## 2 Methodology
### Model Design and Training  
Similar to other machine learning models, regression techniques leverage input data to learn relationships and use those relationships to predict unseen quantities. These relationships are generally contained in model parameters (\(\hat{\beta}\)), which map predictors, as summarized in a design matrix \(\mathbf{X}\), to an output \(\hat{Y}=\mathbf{X}\hat{\beta}+\epsilon\) with residual model error \(\epsilon\). Many different regression techniques exist; these techniques typically vary in the structure of the cost function, which quantifies the error between predicted and expected values. This cost function (\(C\)) is usually captured as a summation of loss functions (calculated on each data point) across the training set. The set \(\hat{\beta}\), which renders the smallest cost, is defined as the learned parameters, mathematically notated as:  
\[\hat{\beta}=arg\min_{\theta}C. \tag{1}\]  
A popular regression model is the ordinary least squares (OLS), which defines its best model (\(\hat{\beta}_{OLS}=arg\min_{\theta}SSE\)) with an objective function equal to the sum of squared errors (\(SSE\)):  
\[SSE=\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}=\sum_{i=1}^{n}(y-\sum_{j=0}^{p}\hat{ \beta}_{j}x_{ij})^{2} \tag{2}\]  
where \(n\) is the number of samples, \(p\) is the number of predictors, and \(x_{ij}\) is the \(i^{th}\) value for the \(j^{th}\) explanatory variable. As shown in the equation, the \(SSE\) sums the squared difference between each sample (\(y\)) and its associated model estimate (\(\hat{y}\)). High emphasis is naturally placed on reducing high-error samples. Therefore, outliers can have a large effect on the learned parameters, so data preprocessing steps are required for robust model development. Additionally, OLS renders non-zero coefficients on all \(\hat{\beta}\), which can create small, insubstantial parameters which are likely components of the training dataset and therefore contribute to model overfitting and thus should be removed from the model. Alternate approaches to OLS include the Theil-Sen regressor [29], which is robust against outliers since it chooses the median of the slopes of all lines between pairs of points, as well as techniques such as Lasso regression [30] that explicitly address model overfitting by reducing model complexity (i.e., the number of parameters used). For this analysis, the latter was selected since Lasso regression models are able to incorporate both parameter regularization and residual sum of squares into the loss function. The cost function for \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**>100 kW Systems**} & \multicolumn{2}{c}{**<1000 kW Systems**} \\ \cline{2-5}
**Parameter** & **Mean** & **StDev** & **Mean** & **StDev** \\ \hline Irradiance & 571.459 & 324.199 & 413.533 & 286.110 \\ Capacity & 14,916.234 & 20,030.000 & 375.919 & 234.151 \\ Energy & 7449.152 & 12,054.525 & 119.008 & 119.829 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Mean and standard deviation (StDev) parameters used to standardize variables prior to training regression models. Lasso regression \(\hat{\beta}_{lasso}=\underset{\theta}{arg\min}\left(SSE+\alpha\sum_{j=1}^{p}|\beta _{j}|\right)\) incorporates an L1 regularization term \(\alpha\sum_{j=1}^{p}|\beta_{j}|\), which penalizes the magnitude of the \(\beta\) terms. This penalization tends to shrink coefficients to zero, rendering a more parsimonious model; we use an \(\alpha=0.003\) for defining the impact of the regularization on the regression kernel. Specifically, the penalization acts as a bias, which in turn can reduce overall error due to the bias-variance tradeoff [31]. Standardized variables are passed into Lasso regression to learn a linear model, which relates the input variables to energy. Multiple combinations of input variables were used to train the regression models (more details below). For all models, a randomized (80-20%) split is utilized to partition the preprocessed, standardized data into train and test partitions, respectively.