Making remove button state changeable

This is actually a remainder of when the remove button was independent from edit mode which made
the remove button's state (enabled/disabled) to not be affected by changing the overall toolbar
state. Since the remove button does show up in edit mode only, now, it should be treated just like
the other buttons.

Change-Id: I7a96f573e561d4bf53b852275c60b6115dc139e3
Merge "Making remove button state changeable"