I would highly recommend a Discovery of Flight experience at your local airfield. They're usually free, and I think it will help you realize that there are a lot of things you aren't accounting for. Aviation isn't nearly as tidy an environment as you think it is, and safety margins need to be big. The number of protocols that have been instituted because of statistically insignificant events is one of the reasons aviation is as safe as it is.

From the air, five miles is quite close to an airfield. There's a significant amount of activity in all directions of an airfield that near to it–you have straight-in pattern entries, overhead pattern entries, diagonal pattern entries, base pattern entries, downwind pattern entries, teardrop pattern entries, aircraft transiting the airspace, helicopters running one traffic pattern, single engine aircraft running another, multi-engine aircraft another, jets another, and sometimes heavies yet another, with everyone else avoiding all the wake turbulence they produce.

All of this happens mostly within 5-10 miles of the field. Not everyone has the same training and experience, there are students and instructors in the mix, equipment failures happen, radar coverage is far from perfect, meteorological and geographic factors often crowd an otherwise-open airspace so everyone's now operating in a greatly reduced footprint, and so on.

All this is further compounded by go-arounds (when an aircraft can't make an approach for whatever reason, either called by a pilot or a controller), which regularly throw the traffic around a field into a state of fairly instant disarray, as controllers vector aircraft rapidly away from each other to try to prevent any collisions from occurring.

If an emergency is declared the airspace may have an aircraft operating anywhere above ground level they deem necessary

This complexity is enough to keep you busy when everyone's hearing everyone, and being coordinated by an adept controller. Drone operators aren't listening to controllers when they operate, don't have great vantage point of the airspace, and would know almost nothing of what's happening above while they're flying.

Beyond that, I can think of several situations where a pilot might find the safest course of action is to bolt from an airfield at <1,000 feet AGL for a few miles. It's probably not an everyday occurrence at any particular airfield, but it's happened to me multiple times (and I'm far from a high-time pilot) and it probably happens daily, multiple times, somewhere in the FAA's jurisdiction.

Also, consider:

91.119 Minimum safe altitudes; general

Except when necessary for takeoff or landing, no person may operate an aircraft below the following altitudes;

(a) ·Anywhere. ·An altitude allowing, if a power unit fails, an emergency landing without undue hazard to persons or property on the surface.

(b) ·Over congested areas. ·Over any congested area of a city, town, or settlement, or over any open air assembly of persons, an altitude of 1,000 feet above the highest obstacle within a horizontal radius of 2.000 feet of the aircraft.

(c) ·Over other than congested areas.
An altitude of 500 feet above the surface except over open water or sparsely populated areas. In that case, the aircraft may not be operated closer than 500 feet to any person, vessel, vehicle, or structure.

(d) ·Helicopters. ·Helicopters may be operated at less than the minimums prescribed In paragraph (b) or (c) of this section if the operation is conducted without hazard to persons or property on the surface. In addition, each person operating a helicopter shall comply with routes or altitudes specifically prescribed for helicopters by the Administrator.

·Helicopter operations may be conducted below the minimum altitudes set for fixed-wing aircraft. The reason? The helicopter's unique operating characteristics, the most important of which is its ability to execute pinpoint emergency landings during power failure. Further, the helicopter's increased use by law enforcement and emergency medical service agencies requires added flexibility in the application of many FAA provisions.

---

Plenty of drone activity happens over water, unpopulated areas, sparsely populated areas, etc.

Lastly, not all passenger aircraft are large jets.

Again, I can't recommend getting an hour or two of flying experience enough. HN commenters tend to have a high level of expertise within their respective domains, and perhaps as a result are almost famously prone to oversimplifying subjects outside of their domain expertise. It seems like every time an aviation-related thread comes up it usually highlights that tendency wonderfully.