Change the InvertX/InvertY defaults. Rely on the xorg.conf setup instead.