Caffeine can be a sleep killer. It should be no surprise that something that wakes you up can make it more difficult to go to sleep. For most people, caffeine wears off in about five or six hours.