wiredtiger/wiredtiger.github.com — Issue #120: wt load doesn't match cursor semantics
Author: wiredtiger
Created: 2012-01-31 18:08:37+00:00
State: open

This Issue is extracted from #68.

There are two special semantics in wt load: -a to append to column-store tables, and -o to overwrite existing records in either row-store or column-store tables.

This matches existing cursor behavior for row-store (set "overwrite" on the cursor if o is set, and then use cursor>insert), but doesn't match existing cursor behavior for column-store.

Michael said:

For column stores, we don't have a single cursor operation that does an "insert with the specified recno or fail if it already exists", but that seems like the right semantic if we are trying to load using the record numbers in the dump.

Keith said:

It's easy enough to make cursor->insert fail if the record already exists: do you remember why we didn't do that in the first place?

Michael said:

The issue is that by default in a column store, WT_CURSOR::insert assigns the key, so collisions aren't possible.

When "overwrite" was added, we conflated two things:

(1) use the key from the application; (2) update an existing item rather than failing;

If we had a separate "use_application_key" flag for column store cursors (implied by "overwrite"), we could get the default row store behavior for column stores by setting use_application_key=true and overwrite=false.

Or we could turn this around and make the default behaviors the same (always use application keys, insert fails if they exist), and add an "append" flag for column store cursors that has WiredTiger assign keys and makes the overwrite setting irrelevant. That would actually match where I think you have ended up with load...

I agree with Michael's last suggestion, adding a special "append" flag to column-store cursors, but I'm not sure if I'm going to get to it today.