# A Review of the Artificial Neural Network Models for Water Quality Prediction
## 3 Three Basic Model Structures in Water Quality Prediction
### Feedforward Architectures
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Structure(s)** & **Advantage(s)** & **Reference(s)** \\ \hline MLPs & They are based on an understanding of the biological nervous system & Solving the nonlinear problems & [19; 23; 30; 32; 33; 34; 35] \\ \hline TDNNs & They are based on the structure of MLPs & Using time delay cells to deal with the dynamic nature of sample data & [36] \\ \hline RBFNNs & The structure of RBFNNs is similar to the MLPs & To overcome the local minimum problems & [5; 18; 37; 38] \\  & The radial basis activation function is in the hidden layer & & \\ \hline GRNNs & A modified form of the RBFNNs model & \\  & There is a pattern and a summation layer & Solving the small sample problems & [24; 39; 40; 41; 42; 43] \\  & between the input and output layers & & \\ \hline WNNs & Wavelet function replace the linear sigmoid activation functions of MLPs & Solving the non-stationary problems & [16; 44] \\ \hline ELMs & The structure of ELMs is similar to the MLPs & Reducing the computation problems & \\  & Only need to learn the output weight & because the weights of the input and & [31; 45; 46; 47; 48] \\ \hline  & & A constructive neural network that & \\  & Start with input and output layer without a & aims to solve the problems of the & \\  & hidden layer & determination of potential neurons & \\  & & which are not relevant to the & \\  & & output layer & \\ \hline  & A special feedforward network Choosing the & & \\  & neural network which have the maximum & Solving the problem of low & \\  & similarity between the inputs and centroids & prediction accuracy & [30; 50] \\  & of the cluster & & \\ \hline RNNNs & The RNNs are developed with the & Solving the problems of long-term & \\  & development of deep learning & dependence which are not captured & \\  & & by the feedforward network & \\ \hline LSTMs & Its structure is similar to RNNNs & Addressing the well-known vanishing & \\  & Memory cell state is added to hidden layer & gradient problem of RNNs & [15; 26; 45; 53; 54] \\ \hline  & Its structure is similar to MLPs & Reducing the influence of the noise & \\  & It has the local recurrent connections in the & and owning the advantage of & \\  & hidden layer & adaptive memory depth & \\ \hline  & Sub-classes of RNNs & & \\  & Their recurrent connections are from & Solving the problems of & \\  & the output & long-term dependence & \\ \hline Elman & A context layer that can store the internal & & \\  & states is added besides the traditional & & \\  & three layers & & \\ \hline  & Different from the above recurrent & & \\  & neural networks & To overcome the problems of the local & \\  & The three layers are input, reservoir, and & minima and gradient vanishing & \\  & readout layer & & \\ \hline  & They are based on the structure of ESN which & To overcome the ill-posed problem & \\  & has a large and sparsely connected reservoir & existing in the ESN & \\ \hline Hybrid & The combination of conventional or & & \\  & preprocess methods with ANNs & Exploring the advantages of & \\ methods & The internal integration of ANN methods or & each methods & \\ \hline CNN & Input, convolution, fully connection, and & An emerging method to solve the & \\  & output layers & dissolved oxygen prediction problem & \\ \hline  & They are based on the structure of DBN & Investigating the problem of & \\  & whose visible and hidden layers are & dynamically determining the & \\  & stacked sequentially & structure of DBN & \\ \hline  & & & \\ \hline  & & & \\ \end{tabular}
\end{table} 
Table 2: The developments and advantages of different ANNs architectures. TDNNs is a subclass of MLPs that learns temporal behavior from continuous past and present signals [36]. The major difference between RBFNNs and MLPs is that the hidden layer of RBFNNs is self-organizing while the latter is not, although the structure of RBFNNs is similar to MLPs. As the center of RBF, the training weights can be defined by a clustering algorithm. For example, the k-means algorithm is a commonly used one [24].