clean up handlers and panes on map remove, close #1621