Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD.
Improve manual cleaning step performed by the multi-platform build tool for MSVC.
Candidate fix for a memory leak that occurs if a CREATE TABLE statement
contains two or more COLLATE clauses on the same column definition.
Add test cases to demonstrate the memory leak on the COLLATE clause.
Fix the memory leak in CREATE TABLE that occurs if there are two or more
COLLATE clauses on the same column.
Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option.
Add the "queryplanner" test permutation.  Continuing refinements to NGQP.
"make test" now passing.
Merge all changes from trunk.
Bug fixes in the handling of virtual tables.
Activate the one-pass optimization.  Update comments, especially the
descriptions of the various objects.