We can discuss only a few of these. The ability of neural nets to learn from a training set and interpolate reasonably between training points is called generalization, and is one of the main attractions of neural nets. Another neural net way of describing this NNAC feature is that NNAC cannot be overtained. Mathematically, we would say that it is truly an iterative function approximator. If you look at the NN FAQs page you will see that generalization is still a matter of guesswork for neural nets.
People still have to use tricks to avoid overfitting .
In fact, the phrase overfitting downplays the magnitude of the problem. Overtrained nets don't just interpolate too tightly to the data points they were trained on, they can make wildly inappropriate generalizations, way beyond the input data range. Really, most neural nets are iterative training point approximators, which transiently interpolate to approximate the function underlying the training set, if you meet certain stringent conditions. Nnac guarantees generalization, and cannot go way beyond the training data. Now there is a trade-off.