Only run package specs when project path exists

Closes #4579