If you put out a birdbath for hummers, be sure the water is very shallow. A leaky hose with water running over a flat rock will get their attention. They seem to prefer their bath in the sun with a perch nearby for preening. The answer is when you haven't seen a hummer for a week or 10 days. Like other birds, hummers' migration is instinctive and not dependent on food supply.
Hummingbirds often leave when there are flowers and feeders still available.
It is likely that shortening daylight hours is the signal to head south. Males often leave up to two weeks before the females, and they are the first to return in the spring. The last ones through in the fall are late-nesting females and their young. They need all the help they can get to complete the journey. If feeders are taken down too soon, many of them will not make it.