In the above example, the network has not only learned the function but it has also learned the noise present in the data. When the net has learned the noise, we say it has overtrained. The reason for this name is that as a net trains it first learns the rough structure of the data. A network is said to generalize well if it can accurately predict the correct output on data it has never seen. One example of this is weight decay.
Weight decay pushes the weights toward zero.