Move touch listener back to item views

* Multitouch breaks it
* Swiping away an item and then immediately swiping away an item below it will break the stuff
Swiping two items will remove the correct items

But it won't animate it nicely...
Handle multiple swipes properly