Improve selection to allow deselecting from line too

Needs work when we have non-unique stop-ids