Fix removeFrom bug in toggling-ui. Fixes #565