Adapt tools/ibus-table-query to work with the new database format
Merge pull request #37 from mike-fabian/adapt-ibus-table-query-to-the-new-database-format

Adapt ibus table query to the new database format
Add support for wildcards

The options

AUTO_WILDCARD, MULTI_WILDCARD_CHAR, and SINGLE_WILDCARD_CHAR
are now read from the table and used.

The defaults are:

    AUTO_WILDCARD = TRUE
    MULTI_WILDCARD_CHAR = *
    SINGLE_WILDCARD_CHAR = ?

Now one can use wildcards as in the following example for cangjie5:
“cangjie5.txt” contains:

    sqsf	馬	1000

Now typing something like “s*f” or “s?sf” will match 馬.

If AUTO_WILDCARD = TRUE, a multi wildcard character will always
be appended to the user input, for example typing “s” will
match like “s*”. Until now, ibus-table was always working like that.
Now it is possible to set AUTO_WILDCARD = FALSE to avoid that
automatic final wildcard and get only exact matches. This is
very rarely useful though, take care with that option, in most
cases this should be TRUE!
Add wildcard support also to pinyin mode

The wildcards used are the same as for table mode. I.e. if
the table defines:

    MULTI_WILDCARD_CHAR = *
    SINGLE_WILDCARD_CHAR = ?

the same wildcards are used for pinyin as well.

The AUTO_WILDCARD in the table is ignored for pinyin mode though, it
is always treated as AUTO_WILDCARD = TRUE for pinyin mode, FALSE
does not make any sense for pinyin mode.
Don’t show the prompt characters defined in the table in pinyin mode in the auxiliary text
Merge pull request #38 from mike-fabian/add-support-for-wildcards

Add support for wildcards
Release 1.8.1
Do not create useless indexes

Resolves: rhbz#1105465 - Please avoid to create sqlite table index in ibus-table
See: https://bugzilla.redhat.com/show_bug.cgi?id=1105465

Most of indexes which were created were not helpful at all because
most queries are done using the SQL “LIKE” operator which does
not benefit from indexes.

And the SELECT statements which do not use “LIKE” are not doing anything
where speed is critical. So we could not find a benefit in any
of the existing indexes. Benchmarking showed no significant difference
between having the indexes or not having them.

I changed the functions create_indexes() and drop_indexes() to do
nothing, if we ever find that some index is really useful, we can
easily add it again.

Not creating the indexes saves a lot of space:

Sizes with indexes:

    $ du -h -c /usr/share/ibus-table/tables/*.db
    8.6M	/usr/share/ibus-table/tables/array30-big.db
    3.7M	/usr/share/ibus-table/tables/array30.db
    3.7M	/usr/share/ibus-table/tables/cangjie-big.db
    2.1M	/usr/share/ibus-table/tables/cangjie3.db
    4.7M	/usr/share/ibus-table/tables/cangjie5.db
    572K	/usr/share/ibus-table/tables/cantonese.db
    700K	/usr/share/ibus-table/tables/cantonhk.db
    5.0M	/usr/share/ibus-table/tables/cns11643.db
    64K	/usr/share/ibus-table/tables/compose.db
    7.7M	/usr/share/ibus-table/tables/easy-big.db
    96K	/usr/share/ibus-table/tables/emoji-table.db
    6.6M	/usr/share/ibus-table/tables/erbi-qs.db
    3.0M	/usr/share/ibus-table/tables/erbi.db
    40K	/usr/share/ibus-table/tables/ipa-x-sampa.db
    640K	/usr/share/ibus-table/tables/jyutping.db
    60K	/usr/share/ibus-table/tables/latex.db
    312K	/usr/share/ibus-table/tables/mathwriter-ibus.db
    812K	/usr/share/ibus-table/tables/quick-classic.db
    2.0M	/usr/share/ibus-table/tables/quick3.db
    4.4M	/usr/share/ibus-table/tables/quick5.db
    40K	/usr/share/ibus-table/tables/rustrad.db
    5.0M	/usr/share/ibus-table/tables/scj6.db
    364K	/usr/share/ibus-table/tables/stroke5.db
    40K	/usr/share/ibus-table/tables/thai.db
    40K	/usr/share/ibus-table/tables/translit-ua.db
    40K	/usr/share/ibus-table/tables/translit.db
    40K	/usr/share/ibus-table/tables/viqr.db
    1000K	/usr/share/ibus-table/tables/wu.db
    11M	/usr/share/ibus-table/tables/wubi-haifeng86.db
    12M	/usr/share/ibus-table/tables/wubi-jidian86.db
    40K	/usr/share/ibus-table/tables/yawerty.db
    4.2M	/usr/share/ibus-table/tables/yong.db
    87M	合計

Sizes without indexes:

    $ du -h -c /usr/share/ibus-table/tables/*.db
    4.6M	/usr/share/ibus-table/tables/array30-big.db
    2.0M	/usr/share/ibus-table/tables/array30.db
    1.5M	/usr/share/ibus-table/tables/cangjie-big.db
    1020K	/usr/share/ibus-table/tables/cangjie3.db
    2.1M	/usr/share/ibus-table/tables/cangjie5.db
    236K	/usr/share/ibus-table/tables/cantonese.db
    292K	/usr/share/ibus-table/tables/cantonhk.db
    2.1M	/usr/share/ibus-table/tables/cns11643.db
    32K	/usr/share/ibus-table/tables/compose.db
    3.2M	/usr/share/ibus-table/tables/easy-big.db
    48K	/usr/share/ibus-table/tables/emoji-table.db
    3.2M	/usr/share/ibus-table/tables/erbi-qs.db
    1.3M	/usr/share/ibus-table/tables/erbi.db
    24K	/usr/share/ibus-table/tables/ipa-x-sampa.db
    268K	/usr/share/ibus-table/tables/jyutping.db
    36K	/usr/share/ibus-table/tables/latex.db
    152K	/usr/share/ibus-table/tables/mathwriter-ibus.db
    332K	/usr/share/ibus-table/tables/quick-classic.db
    980K	/usr/share/ibus-table/tables/quick3.db
    2.0M	/usr/share/ibus-table/tables/quick5.db
    24K	/usr/share/ibus-table/tables/rustrad.db
    2.2M	/usr/share/ibus-table/tables/scj6.db
    156K	/usr/share/ibus-table/tables/stroke5.db
    24K	/usr/share/ibus-table/tables/thai.db
    24K	/usr/share/ibus-table/tables/translit-ua.db
    24K	/usr/share/ibus-table/tables/translit.db
    24K	/usr/share/ibus-table/tables/viqr.db
    424K	/usr/share/ibus-table/tables/wu.db
    4.8M	/usr/share/ibus-table/tables/wubi-haifeng86.db
    5.7M	/usr/share/ibus-table/tables/wubi-jidian86.db
    24K	/usr/share/ibus-table/tables/yawerty.db
    1.8M	/usr/share/ibus-table/tables/yong.db
    40M	合計
Merge pull request #39 from mike-fabian/do-not-create-useless-indexes

Do not create useless indexes
Better sorting of the lookup table in the mixed Chinese modes

In the mixed Chinese modes 2 and 3 (simplified Chinese first and
traditional Chinese first), the filtering for simplified and traditional
was done as a last step filtering the candidate list already sorted
using the sort keys:

    - exact match?
    - user frequency
    - system frequency
    - length of input sequence
    - alphabetical order of input sequence
    - Unicode code point of first character of matched phrase

But this simplified and traditional filtering is basically just
another sort key. Doing this after the main sorting of the candidate
list is inefficient and does not give nice results either. It is
better to do all of the sorting in one place.

The sort key for the desired Chinese variant (traditional or simplified)
should have higher priority than the system frequency but lower
priority than the user frequency. If the user selects a certain variant
often, it should be preferred in future.

An example for the behaviour after this commit is applied:

wubi-jidian86.txt contains the following matches for 語
(as a single character):

    ygkg	語	204000000

It contains the following matches for 语 (as a single character):

    ygk	语	234000000
    ygkg	语	234000000

And the following matches for the input sequence “ygkg”:

    ygkg	語	204000000
    ygkg	语	234000000
    ygkg	𢌐	0
    ygkg	𣄟	0
    ygkg	𩐕	0

After this commit, typing “ygkg” gives the following results in the
lookup table (when all user frequencies are 0. For testing this
can be achieved by removing the user database):

Chinese mode 0, “simplified Chinese only”:
==========================================

ygkg (1/4)
1. 语 234000000 0
2. 𢌐 0 0
3. 𣄟 0 0
4. 𩐕 0 0

Chinese mode 1, “traditional Chinese only”:
===========================================

ygkg (1/4)
1. 語 204000000 0
2. 𢌐 0 0
3. 𣄟 0 0
4. 𩐕 0 0

Chinese mode 2, “simplified Chinese before traditional”:
========================================================

ygkg (1/5)
1. 语 234000000 0
2. 𢌐 0 0
3. 𣄟 0 0
4. 𩐕 0 0
5. 語 204000000 0

Chinese mode 3, “traditional Chinese before simplified”:
========================================================

ygkg (1/5)
1. 語 204000000 0
2. 𢌐 0 0
3. 𣄟 0 0
4. 𩐕 0 0
5. 语 234000000 0

Chinese mode 4, “All Chinese characters”:
=========================================

ygkg (1/5)
1. 语 234000000 0
2. 語 204000000 0
3. 𢌐 0 0
4. 𣄟 0 0
5. 𩐕 0 0
Merge pull request #40 from mike-fabian/better-sorting-in-the-mixed-chinese-modes

Better sorting in the mixed chinese modes
Release v1.8.2.
Insert a special candidate for the wildcard character itself if only a wildcard character has been typed

Resolves: rhbz#1110325 - Unable to input question marks in Wubi-jidian
See: https://bugzilla.redhat.com/show_bug.cgi?id=1110325

The same problem has been also reported here:

    https://code.google.com/p/ibus/issues/detail?id=1720

If only a wildcard character has been typed, insert a special
candidate at the first position for the wildcard character itself. For
example, if “?” is used as a wildcard character and this is the only
character typed, add a candidate ('?', '?', 0, 1000000000) in
halfwidth mode or a candidate ('?', '？', 0, 1000000000) in fullwidth
mode.  This is needed to make it possible to input the wildcard
characters themselves, if “?” acted only as a wildcard it would be
impossible to input a fullwidth question mark.
Make wildchard characters configurable

The options AUTO_WILDCARD, SINGLE_WILDCARD_CHAR, and
MULTI_WILDCARD_CHAR are now not only configurable in the table sources
but also in the setup tool and via dconf.

The default for AUTO_WILDCARD if neither specified in the table source
nor in the setup tool nor in dconf is TRUE, this is best for most
tables.

The defaults for SINGLE_WILDCARD_CHAR and MULTI_WILDCARD_CHAR
are the empty strings, this means no wildcards are used.

In some table sources default values for SINGLE_WILDCARD_CHAR and
MULTI_WILDCARD_CHAR are defined, in most cases these are

SINGLE_WILDCARD_CHAR = ?
MULTI_WILDCARD_CHAR = *
Update German translations
Update Chinese .po files

(Only to get the changed strings from the source synced with the .po
files, I did neither change nor add any translations).
Merge pull request #41 from mike-fabian/improve-wildcard-support

Improve wildcard support
Release v1.8.3.
Don’t add self._single_wildcard_char and self._multi_wildcard_char to self._valid_input_chars and self._pinyin_valid_input_chars

This is because the wildcard characters can be changed with the setup
tool at runtime. If they are added to the string of valid input
characters, the old wildcards would need to be removed from that
string if the wildcards are changed. But the wildcard character may
have been in that list already before it was added as a wildcard
character. In that case, only one instance of the old wildcard
character should be removed from the string of valid input characters
if the wildcard character is changed. Replacing the wildcard
characters in the string of valid input characters when they change
seems more confusing that not adding them to that string in the first
place.

Therefore, I do not add the wildcard characters to the string of
valid input characters anymore, but that means that when checking
whether a character is a valid input character

    if c in self._valid_input_chars:

is not enough anymore, now one needs

    if c in self._valid_input_chars + self._single_wildcard_char + self._multi_wildcard_char:

That seems less confusing than adding the wildcard characters to
self._valid_input_chars and replacing them there when they change.
Merge pull request #42 from mike-fabian/do-not-add-wildcards-to-string-of-valid-input-chars

Don’t add self._single_wildcard_char and self._multi_wildcard_char to se...