Just unload and delete the rvm directory instead