Improve the remove method by further splitting it

Now I find it more understandable when split in pieces with different
responsibilities.