When adjusting scroll offset, prioritize using minimum offset to avoid miscalculation