Only show the remove icon if we are in edit mode