You should replace the upper control arm bushes if you notice excessive movement, cracking, or deterioration, as this can affect the arm's performance and your vehicle's safety.