Remove unnecessary workaround from setup.py

Also update MANIFEST.in to exclude files that shouldn't be in the sdist.