## 1 Introduction
### Background  
#### 1.1.1 Theory of LSTM Networks  
Machine learning has gained popularity over the past several years, and engineers have developed many different types of machine learning models. Two of the more popular models for prediction and forecasting, are support-vector machines (SVMs) and artificial neural networks (ANNs). In recent years, a significant body of research has been published on the application of ANNs for solar forecasting. As [PERSON] defines in his article _An Introduction to Artificial Neural Networks_, \"An Artificial Neural Network is an information processing model that is inspired by the way biological nervous systems, such as the brain, process information. They are loosely modeled after the neuronal structure of the mammalian cerebral cortex but on much smaller scales\" [2]. Neurons, the most basic components of an ANN take two or more inputs and give one output. They then compute an output based on a set of weights (applied to the inputs) and a bias (which modifies the output). When several neurons are organized into layers and connected such that the output of one layer feeds into the input of another layer, this creates a feedforward ANN that can learn to solve complex problems using the experience it gained during the training process [2]. ANNs learn by example. During the training process, an ANN is given a set of inputs to process. The output from the ANN is then compared to the desired output, and an error is calculated. After a set of training examples has been processed, a backpropagation algorithm tweaks the weights and biases using gradient descent to reduce the error. This process is repeated many times until the error function converges, or the researcher is satisfied with the ANN's performance. Once effectively trained, an ANN can be fed new data and predict the correct output with relatively high precision. Recurrent neural networks (RNNs) are popular for predicting time series data. For each timestep, the RNN receives not only the new input data but also its own output from the previous timestep, known as the hidden state. In this way, RNNs have a rudimentary form of short-term memory and are better at finding short-term trends in the data than plain feedforward networks [3]. LSTM networks are a type of RNN. The LSTM modules are typically referred to as cells rather than neurons and contain a series of gates. A diagram of an LSTM cell can be seen in Figure 1. Each LSTM cell has a form of longer-term memory in the form of a cell state that is updated through time. A forget gate looks at the new input and the hidden state and decides which information in the cell state can be safely forgotten. The input gate then decides what information from the new input should be added to the cell state to be remembered. Finally, the output gate takes information from the cell state, input, and hidden state and generates the output for the current time step. In this way, LSTM networks can remember information through many timesteps, making them ideal for finding longer-term trends in data. At the same time, the LSTM cell still uses the hidden state, and therefore has short-term memory as well. Overall, LSTM networks can be a powerful tool in time series forecasting [4,5]. ANNs perform best when the training data adheres to certain conditions. First, when all the data have been cleaned and processed so that there are no wrong, not-a-number values or significant time gaps in the data. Secondly, because ANNs have no concept of units but rather look only at the numerical value of the input, it is advisable to normalize all training inputs to be within the same range. Feature scaling prevents the ANN from learning false patterns that result from unit differences in the data. A common practice is to use feature scaling to map all the data to between 0 and 1. Feature scaling the inputs has the additional effect of speeding up the training process of the ANN because small changes in the weights and biases will have a greater and more consistent effect on the output [6].