If you've had a long day at the office, consider skipping the evening ride until you're recovered from daily life. Adding a physical workout on top of emotional stress can push you into a fatigue hole. If you're sick, training should involve getting back to normal health, not riding your bike. Illness forces your body to expend energy fighting the virus or bacteria, and workouts will only slow recovery. The faster you get back to health, the faster you can train, but not vice versa.
New environments stress the body.
Cold or hot weather and altitude are big factors that should be considered and accounted for in the training and recovery process. Serious training is hard to do, but unnecessary fatigue shouldn't be a part of your problems. Experiment with different recovery methods, and don't let pressure to train hard block out adequate recovery.