The mind set is that pushing yourself harder makes you stronger and faster. In truth it does, but only if there is sufficient rest to allow for recovery. Rest is what the athlete does, and comes in two forms, passive and active . Active rest is defined as very light exercise, often with stretching, which doesn't result in damage to the muscles. An example of this is a 30 minute easy spin on the bike.
Recovery is what the body does to repair the damage from a tough workout.
The soreness that one feels for a day or two after hard exercise is due to damaged muscle cells, not lactic acid as is commonly believed. Immediately after exercise, the body sets to work to repair the damage. The repair process involves an inflammatory response, and mediators of inflammation are what stimulate nerve endings resulting in soreness or pain. Given enough rest time the body will completely fix the broken-down muscles, and will then build them up a little bit more. This last phase is called super-compensation and is why, over time, athletes get stronger and faster.