Prevent creating "remove" toolbar twice for a reference

When a "remove" toolbar is initialized already, this change set will prevent initializing
it again which would happen when adding a first reference.

Change-Id: I27add2df6bc4107aee9fa1b9c0ad77c42c6b514b
Merge "Prevent creating "remove" toolbar twice for a reference"