Actually remove the submodule.