Attempting to only update cached heights and offets from a given start
index was buggy so they are always calculated from the beginning.