Try to increment color only on ACTION_UP.