Remove svg, not entire pane when removing d3geojson layer.