Remove dir which was a submodule