Removing notes from ever being toggled