dist tarball fixes

Cherry-picked the ones that were needed in the 3.6.x branch. Not sure if master needs more fixes.
Do we distribute the tests?