SitePen/dgrid — pull request #217 by scbird (2012-06-28 17:11:26+00:00)
Title: Scrolling large OnDemandLists

Continuing on from #117, as well as fixing some other related issues I've noticed in the meantime.

This is a large change in terms of lines of code because there are a number of underlying issues that it fixes. The current version of dgrid does not split preloads. This is a problem when the user jumps the scroll position around.

For example (assuming for argument's sake that 1 row = 30px, farOffRemoval = 10000px (300 rows), and that there are 2000 rows in total):

User scrolls to 30000px (row 1000), then to 33000px (row 1100). The first scroll caused rows 1000 to 1030 to be rendered, and the second scroll caused 1100 to 1130 to be rendered. However since the first set of rows are still within farOffRemoval (=10000px), they are not removed, but we now need 3 preload nodes: rows 0-999, 1031-1099, 1131-1999. Without being able to split the preload nodes, the grid cannot handle this situation. It will try, but things end up getting rendered in the wrong place. This already happens with only a single scroll jump - should the user then jump the scroll position to 36000px or even back to 26000px, the situation gets worse.

To compound matters, if there are two outstanding queries and the server responds to the lower one first, List.adjustRowIndices() is not aware that preload/loading nodes usually represent more than one row, and thus any rows after a preload/loading will get renumbered incorrectly. At a casual glance, the user may not necessarily notice these things other than the scrollbar jumping around a little, however it becomes a large issue when farOffRemoval is set to Infinity and the grid is rendering complex components connected to an external model.

This patch includes a plugin (tests/NodeChecker.js) that monitors the grid at key points and writes to the console (and optionally triggers a breakpoint) if it detects any errors. Copying tests/NodeChecker.js, tests/performance_slow_network.html and util/misc.js onto the current master will highlight the issues. Note that setting farOffRemoval = Infinity on the current master branch will make the browser very slow.

This is a rough list of issues this patch fixes:
* List.js:
* Make List.js aware of nodes that represent multiple rows (eg preload/loading) so that it doesn't incorrectly renumber subsequent rows
* Make List.js reduce the size of adjacent non-data (eg preload/loading) nodes if there is not enough "room" to render the returned rows. This is required for most renders when queryRowsOverlap != 0.
* OnDemandList.js:
* Enable preload nodes to be split as necessary to place loading nodes (and consequently data rows) in the correct position
* Move preload.count to preload.node.rowCount so that it's consistent with rowIndex and visible to List and other interested plugins.
* When maxEmptySpace is being used, the scrollbar will no longer jump around when the user scrolls back up; the grid remembers how far they scrolled down and stays at least that size.
* Fixed several race conditions, in particular between loading nodes and also if the grid is reset while it still has outstanding requests
* test/performance_slow_network.html:
* Allow the user to turn farOffRemoval on and off, as well as change maxEmptySpace
* util/misc.js
* Include a function to determine whether a row contains actual data or is some kind of placeholder