Add missing upstream compiler flags as an upstream patch rather than through debian/rules