Remove mode works to satisfaction

Remove is made as original when the mode to be remove is currently
active somewhere
TODO: Is there a more elegant way to deal with removal of currently
active modes?