Handle collapsing on refresh more smartly