We live at a 2,300-ft. elevation and get a lot of strong winds. The problem in your house is actually related to a generally ignored but critically important component of the house shell, the air barrier. When there are cold drafts coming into the house, there also must be an equal amount of warm indoor air belching out. Oddly, most of this leaking happens in the attic. Far better opportunities to reduce air leaks in a house are found in the attic.
Typical culprits are chimney and plumbing chases.
They can actually have gaps that add up to several square feet. Any kind of ceiling discontinuity is also suspect. Multilevel, sloped and plank ceilings can produce 4 sq. Recessed lighting can also spew huge volumes of warm air into the attic. Finally, smaller air-sealing opportunities, such as drywall gaps and wiring penetrations, lurk along the tops of every interior wall.