Try adjusting your schedule slightly. Spend time with your hamster in the early evening when they're starting to wake up, or early morning before you start your day.