More fixing of the remove source directory mess made by ExternalProject