Caffeine should generally be avoided in the hours leading up to the bed, as it can keep you awake. However, everyone's tolerance is different, so some people may be able to extend their caffeine intake into the later afternoon without difficulty.