remove layer object held in a dict when the layer is removed