Only copy the file and the atime and mtime when installing from Wheel

Fixes #1547