# A Review of the Artificial Neural Network Models for Water Quality Prediction
## 6 Discussion
### Model Is the Core  
Model structure determination problem: Most researchers use a trial-and-error method to determine the ANN structure, which does not fundamentally promote the further development of the model. This review summarizes some empirical formulas to determine the number of neurons in the hidden layer that future researchers can apply to their studies. This review does not reveal the science behind these formulas or the conditions under which they apply. To some extent, the use of these empirical formulas contributes to the determination of the model structure, because researchers build on previous studies rather than stay at the level of trial-and-error with no rules to find. Global methods can obtain topology and network weights, which have been developed to some extent in recent years. Compared with the trial-and-error method, the global method has a sound theoretical basis. Researchers can further study and improve global methods. Activation function determination problem: Most of the time, researchers choose S-shaped functions because they create a random nonlinear mapping between the input and output. The essential reason is that the S-type transfer function is differentiable, continuous, and monotonically increasing in its domain. Purelin is used more frequently in the output layer than other functions because its output can be arbitrary rather than limited to a small range compared to the sigmoid function. Model training problem: The reason for developing so many subclasses of training algorithms is that researchers want to use the appropriate matrix (e.g., Hessian matrix, Jacobian matrix) to accomplish the computing tasks easily. The Quasi-Newton method is suited for the situation that the matrix (whether Hessian or Jacobian) is difficult or even impossible to compute. In water quality prediction, the deterministic methods are more mature than the stochastic methods. One possible reason is that the former only looks for a single parameter vector, while the latter looks for the model parameter distribution, so the latter parameter is more uncertain. The online learning algorithm has the characteristics of real-time and rapid adjustment model which is suitable for prediction tasks. However, its application in water quality prediction is still very limited. Therefore, the algorithm is worthy of further study. Model structure selection problem: Many researchers utilized MLP architectures in ANN to complete prediction tasks between 2008-2019. This result is as same as the conclusion of the review between 1999 to 2007, which may be due to the fact that the MLPs architecture has the advantage of being easy to use, and they can approximate any relationship between input(s) and output(s) through the typical three layers [81]. Global methods (see Table 9), obtaining topology structure and network weights, are drawing the attention of researchers--in contrast to the previous review [27]. It must be noted that the GA, PSO, and ABC methods are typical examples of evolution-related methods. In general, evolutionary methods are combined with ANNs to meet different constraints. Much effort has been made regarding the data-intensive methods, while the model-intensive and technique-intensive approaches were implemented relatively infrequently. Wavelet analyses were widely used in data-intensive methods, while the decomposing approaches were used less. This may be because wavelet analysis has the ability to extract the trends, discontinuities, and breakdown points of the original data. Furthermore, it is also able to process signals by compressing or denoising. In recent years, CNN, as a new feedforward neural network method, has been used in water quality prediction. However, its application is rather limited. Researchers can further expand CNN's reach. RNN has good memory ability, so it can make full use of historical information and lay a solid foundation for realizing long-term prediction of water quality.