Make sure we properly remove the env script on removal

cc @wfarr