remove layer object held in dict when the layer is removed