# hLSTM-Aging: A Hybrid LSTM Model for Software Aging Forecast
## 3 Background
### Time Series  
A time series is a representation of collected data across time. Time series are commonly used for data description, interpretation, process management, and prediction [26]. Time-series prediction is a procedure that aims to examine information from data from previous periods of time to forecast future values for the same data [27]. The sequence of events in a time series can have a significant impact on the complexity of prediction models. Recurrent neural networks (RNN) are commonly employed in time-series prediction because they maintain the results of prior calculations. The main disadvantage of utilizing a conventional RNN is that it takes a long time to train a model. An alternative proposal is the long short-term memory networks (LSTM) [28], a type of recurrent neural network capable of learning order dependence in sequence prediction problems. As a consequence, it is also largely used to fit time-series data. An LSTM has a chain structure that includes four neural networks and several memory units known as cells. First, significant information is added to the neuron via the input gate. The forget gate then removes information that is no longer helpful in the present neuron state. The bias is applied to the current and prior inputs after they have been multiplied by the weight matrices. The result is fed into a binary activation function (similar to sigmoid). If the output state is zero, the information is deleted. If the output state is one, the information is saved for later use. Finally, the output gate is in charge of retrieving useful data from the neuron and sending it to the next neuron. There are also probabilistic methods in statistics, such as auto-regressive (AR), moving average (MA) [7], auto-regressive moving average (ARMA) [10], auto-regressive integrated moving average (ARIMA) [8], and seasonal auto-regressive integrated moving average (SARIMA) [9], that can be used for time-series forecasting. These techniques are all based on probabilistic notions. The auto-regressive (AR) model is a multiple regression model in statistics that anticipates the variable of interest in a time series using a linear combination of previous variable values. It is auto-regressive because it regresses the variable against itself. An auto-regressive model, often known as order \(p\), may be expressed as  
\[X_{t}=c+\sum_{i=1}^{p}\varphi_{i}X_{t-i}+\varepsilon_{t} \tag{1}\]  
where \(\varphi_{1},\dots,\varphi_{p}\varphi_{1},\dots,\varphi_{p}\) are parameters, \(c\) is a constant, and the random variable \(\varepsilon_{t}\varepsilon_{t}\) is white noise [29]. The moving average model [30] states a linear co-dependence of the time series current value with the past error terms. It is assumed that such error terms are independent and regularly distributed. MA(\(q\)) denotes the moving average model of \(q\) order. The actual value is expressed by the model as a linear combination of the time-series mean, the current error term, and the prior error term. Mathematically, the expression is,  
\[X_{t}=\mu+\varepsilon_{t}+\sum_{i=1}^{q}\theta_{i}\varepsilon_{t-i} \tag{2}\]  
where \(\mu\) stands for the time-series mean, \(\varepsilon\) represents the present error, and \(\theta\) is the past error terms. The auto-regressive-moving-average (ARMA) model is based on the assumption that the time series is stationary and varies evenly around a specific period. Because it depicts a time series by running it through a recursive filter and then a non-recursive linear filter, it operates as a hybrid of the MA and AR models. The ARMA order consists of two integers (\(p\),\(q\)) that serve as the orders of the AR and MA components, respectively. The ARMA model is noted as:  
\[X_{t}=c+\varepsilon_{t}+\sum_{i=1}^{p}\varphi_{i}x_{t-i}+\sum_{i=1}^{q}\theta _{i}\varepsilon_{t-i}. \tag{3}\]  
The auto-regressive integrated moving average (ARIMA) model is a generalization of the ARMA model. The \"integrated\" component of the term refers to the reversal of differences.