Geophile 20141104 openall

Geophile integration is complete and working for non-point as well as point spatial objects. System and performance testing is still needed.

Look at the individual commits for details, but here is an overview of how the integration works:

- As before, a spatial query is done as a nested loop, in which the query object is bound in the outer loop, and the inner loop is implemented as a spatial index lookup.

- But to accommodate non-point spatial objects (on both the outer and inner loops), IndexCursorSpatial_InBox has been modified to use a spatial join, instead of probing the index with z-values directly.

- A spatial join requires two geophile SpatialIndex inputs. The query object is decomposed, and the (z, query object) records are placed in a sorted array. GeophileIndex is an adapter between FDB indexes and geophile's Index abstraction. Access to the index is similarly mediated by a GeophileCursor.

- GeophileCursor manages a set of IndexCursorUnidirectionals, each probing a z-value or ancestor. GeophileCursor thus replaces MultiCursor in the previous implementation.

- But there are two significant differences. First, as mentioned, we probe each z-value resulting from the decomposition of the query object, and each ancestor thereof. This allows us to open early all of the FDB cursors that may be needed for the spatial query. Recent changes to geophile avoid "unpredictable" random accesses during the spatial join.

- Second, the IndexCursorUnidirectionals are wrapped by CachingCursors which cache the first item returned by each ICU. This allows the cursor to be probed during geophile's ancestor-finding, and then reopened and probed from the beginning again once a suitable ancestor has been found during spatial join.

Please pay particular attention to my changes to schema maintenance. If one column of a spatial index is dropped, the whole index must be dropped. It looks like ordinary indexes simply are reformed without the dropped column.

Also, note that IndexRow now implements com.geophile.z.Record. We talked about wrapping as a possible alternative. That is a bit more typing, (delegating most methods), and will of course result in the allocation of one wrapper object per row coming out of the index.
The calls come from geophile. E.g., geophile calls Index.newKeyRecord() to get a temp key used during the search. The z-value on this key needs to be set. com.foundationdb.server.spatial.GeophileIndex, which implements Index, returns an FDBIndexRow.

Record.z() is also called by SpatialIndex.add/remove, but because FDB is operating on the indexes directly for index maintenance, this path isn't used by FDB. (This was an area we discussed possibly revisiting.)