Don't call Layer#onAdd on a layer that has been removed
Don't fire layerremove if layeradd was never fired