At lower, non-lethal concentrations, ammonia can cause chronic stress that reduces a fish's ability to fight disease, cuts food consumption and growth, inhibits reproduction and ultimately contributes to premature death.
It is very easy to understand why aquarists and pondkeepers are so preoccupied with this problem.