Yes, stress or fatigue can exacerbate symptoms of conjugate gaze palsy, especially in conditions like multiple sclerosis where eye movement may worsen with fatigue.