Do a shallow clone of the remote repository as @fcheung suggested.