User a sliding buffer so we only update on the latest scroll, and we don't blow the panel-rendering channel (favoring most recently-updated items, since those are likely to be more important)
Make the panels fixed-height for the time being (effect is the same for variable-height VSP, since it has to store heights/offsets still)