Cardslib: Do not consume touchevents on swipe down event

this can give on listview scroll kick start problems.

Change-Id: Ide6266512bb7f7f316214b3332adeb8c20636b54