# Prediction of Ocean Weather Based on Denoising AutoEncoder and Convolutional LSTM
## 2 Theoretical Background
### Conventional Long Short-Term Memory (LSTM)  
The DFN, which is the basic model of deep learning, consists of an input layer, hidden layers, and an output layer. This configuration is useful for finding an output value based on input values. However, it is not suitable for predicting data over the time series. Therefore, a recurrent neural network (RNN) has been proposed to train data with time series [8]. An RNN is a type of deep learning model in which a hidden node is connected to a directed edge to form a cyclic structure called a directed cycle. It is a model suitable for predicting sequentially arranged data, such as voice and text. In the RNN, the state of the previous time step is used in the next step, and the resulting value is affected by the earlier state. Among various RNN models, we used the LSTM to avoid the gradient vanishing. There are many examples of learning time-series data using the DFN, and the structure is also very simple. However, ocean weather has a complex causality between time, space, and parameters of oceanic weather. In addition, since it has periodicity over a long period of time, it can be said that the LSTM is more suitable than the DFN to understand these characteristics. Many deep learning methods have been developed to train time-series data. Among them, LSTM is the most used to date [9]. Figure 2 shows the configuration of the LSTM and details of the cell state \"A\". \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Study** & **Input Data** & **Estimation** & **Estimation** & **Estimation** & **Estimation** \\  & **Objectives** & **Models** & **Area** & **Time Range** \\ \hline [PERSON] et al. [6] & SST & SST & LSTM & Coastal seas & 1 day to 1 \\ \hline [PERSON] et al. [3] & \begin{tabular}{c} Wave height, \\ period, direction, \\ wind speed, \\ current speed \\ \end{tabular} & Wave height, & MLP, SVM & Monterey Bay & N/A \\ \hline This study & \begin{tabular}{c} SST, wave height, \\ wave period, wave \\ direction, wind speed \\ (U component), \\ current speed \\ (V component), \\ current speed \\ (V component), \\ current speed \\ (V component) \\ \end{tabular} & \begin{tabular}{c} SST, wave height, wave \\ period, wave \\ direction, wind speed \\ (U component), \\ wind speed \\ (V component), \\ current speed \\ (V component), \\ (V component), \\ current speed \\ (V component) \\ \end{tabular} & \
\begin{tabular}{c} Convolutional \\ LSTM + \\ Denoising \\ AutoEncoder \\ \end{tabular} &
\begin{tabular}{c} Worldwide \\ week \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Cont. The cell state serves as a type of conveyor belt. Therefore, even if the state has elapsed for a long time, the gradient of the input value can be propagated relatively well. The formula of LSTM cell state \"A\" is shown in Equations (1)-(6). \[f_{t}=\sigma(W_{xf}x_{t}+W_{hf}h_{t-1}+b_{f}) \tag{1}\]  
\[i_{t}=\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+b_{i}) \tag{2}\]  
\[o_{t}=\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+b_{o}) \tag{3}\]  
\[g_{t}=\tanh(W_{xg}x_{t}+W_{hg}h_{t-1}+b_{g}) \tag{4}\]  
\[c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot g_{t} \tag{5}\]  
\[h_{t}=o_{t}\odot\tanh(c_{t}) \tag{6}\]  
In Equations (1)-(6), \(\sigma\) is the sigmoid function, \(W\) is the weight for each layer (subscription is expressed for input and output layers), \(x_{t}\) is the input in time step \(t\), \(b\) is the bias (subscription is expressed for gates), and \(\tanh\) is a hyperbolic tangent function. In Equations (5) and (6), \(\bigcirc\) is the Hadamard product operator [10]. The forget gate \(f_{t}\) is a gate to forget past information. The value obtained by taking the sigmoid after receiving \(h_{t-1}\) and \(x_{t}\) is the value that the forget gate sends out. The output range of the sigmoid function is from 0 to 1. If the value is 0, information from the previous state is forgotten, and if it is 1, information from the previous state is completely remembered. The input gate \(i_{t}\bigcirc\bigcirc g_{t}\) is a gate for storing current information.