# Resource Profiling and Performance Modeling for Distributed Scientific Computing Environments
## 3 Resource Profiling and Performance Modeling
### Resource Profiling and Prediction Models
#### 3.2.1 Online Linear Regression (OLR)  
Typically, the measured or real outputs are highly variable because resource instances in a distributed scientific computing system are stochastic in nature. The conventional prediction methods such as linear regression (LR), autoregressive (AR) predict future resource availability using historical \(n\) data points. Recent dynamic changes in resource availability can be easily captured if we rely on the most recent trends of data rather than _all_ historical \(n\) points information. A simple linear model of one explanatory variable has two parameters such as \(\beta_{0}\) and \(\beta_{1}\) expressed as in the following Equation (1). Parameters \(\beta_{0}\) and \(\beta_{1}\) are estimated from the \(n\) observed data set. In the Equation (1), the subscript \(i\) denotes the observational unit, where \(i=\{1,2,\ldots,n\}\) and \(n\) is the total number of observed data. \[Y_{i}=\beta_{0}+\beta_{1}X_{i} \tag{1}\]  
The least squares estimation method [39] has been widely applied to estimate the parameter values because this technique provides the smallest possible sum of squared error of the observed \(Y_{i}\) from the estimates \(\hat{Y}\). Let \(\hat{\beta}_{0}\) and \(\hat{\beta}_{1}\) be numerical estimated values of the parameters \(\beta_{0}\) and \(\beta_{1}\) respectively. The Equation (1) can be written as follows:  
\[\hat{Y}_{i}=\hat{\beta}_{0}+\hat{\beta}_{1}X_{i} \tag{2}\]  
The least squares principle chooses \(\hat{\beta}_{0}\) and \(\hat{\beta}_{1}\) in a way that minimizes the sum of squared (\(SS\)) of magnitude of the residual \(\epsilon\). The \(i\)th residual \(\epsilon_{i}\) is the difference between the real output \(Y\) and the estimated output \(\hat{Y}\) at the data point \(i\). \[ \begin{split} SS(Residual)&=\sum_{i=1}^{n}(Y_{i}- \hat{Y}_{i})^{2}=\sum_{i=1}^{n}\epsilon_{i}^{2}\\ &=\sum_{i=1}^{n}(Y_{i}-\hat{\beta}_{0}-\hat{\beta}_{1}X_{i})^{2 }\end{split}  \tag{3}\]  
Applying the derivatives with respect \(\beta_{0}\) and \(\beta_{1}\), we can solve the Equation (3) to estimate the parameters. \[\frac{\partial SS}{\partial\beta_{0}}=-2\sum_{i=1}^{n}(Y_{i}-\hat{\beta}_{0}- \hat{\beta}_{1}X_{i})=0 \tag{4}\]  
\[\frac{\partial SS}{\partial\beta_{1}}=-2\sum_{i=1}^{n}(Y_{i}-\hat{\beta}_{0}- \hat{\beta}_{1}X_{i})=0 \tag{5}\]Simplifying the Equations (4) and (5) yields the \(\hat{\beta}_{0}\) and \(\hat{\beta}_{1}\) as follows:  
\[\hat{\beta}_{0}=\overline{Y}-\hat{\beta}_{1}\overline{X} \tag{6}\]  
\[\hat{\beta}_{1}=\frac{\sum_{i=1}^{n}(X_{i}-\overline{X})(Y_{i}-\overline{Y})}{ \sum_{i=1}^{n}(X_{i}-\overline{X})^{2}} \tag{7}\]  
where \(\overline{X}\) and \(\overline{Y}\) are the means of the \(X_{i}\) and \(Y_{i}\) respectively. It can be noted from the Equation (7) that \(\big{(}X_{i}-\overline{X}\big{)}\) and \(\big{(}Y_{i}-\overline{Y}\big{)}\) denote observations expressed as deviations from their sample mean \(\overline{X}\) and \(\overline{Y}\) respectively. Therefore, estimation of parameters give the linear regression model equation as follows:  
\[\hat{Y}_{i}=\hat{\beta}_{0}+\hat{\beta}_{1}X_{i} \tag{8}\]  
Online regression modeling makes forecasting by using a window with size \(w\) which contains the latest profiling information, i.e., \(w\) data points out of \(n\) (where \(w<n\)). Thus, predicting based on dynamic \(w\) interval is known as Online Linear Regression (OLR) model and the coefficient parameters \(\beta_{0}\) and \(\beta_{1}\) are estimated based on the Equations (9) and (10) where \(\overline{X}_{w}\) and \(\overline{Y}_{w}\) are the mean of sample data over \(w\) interval. This model is also known as moving linear regression because \(w\) is moving towards the recent trends.