Revert "on finish of pan gesture when scrolling, do partial refresh"

This reverts commit c4a990316272dc7ab5aa4a718af33170ed9fb55a.

Actually, a full refresh is needed to clean the clutter after fast
refresh.

Might fix #1305