In general, men begin to lose testosterone at a rate of 1%-2% each year after they turn 40. One study found that 20% of men over the age of 60 and 50% of those over the age of 80 experience lower levels of testosterone. However, not all people who experience age-related reductions in testosterone will have symptoms due to the change in testosterone levels.