Improve remove logic

Thanks @kou1okada

http://github.com/transcode-open/apt-cyg/commit/1b3c49d#commitcomment-6329645