Remove tabdict

Do not translate the typed characters into integers with tabdict.
Use the characters themselves instead.

Translating characters to integers doesn’t make anything faster,
just more complicated.

Even if a translation to integers were desirable, why use a custom
table and not the Unicode code points? A custom table always
needs to be extended when more characters are allowed as valid
input characters.
Remove parse_phrase_to_tabkeys() and simplify parse_phrase()
Simplify goucima database table and remove goucima cache

The goucima cache is not really used anyway. Only in tabcreatedb.py
db.cache_goucima() is called, but then not used there either.
Don’t use try: and except: to decide whether to decode  from UTF-8 in get_no_check_chars()
Simplify pinyin table
Don’t filter candidates in pinyin mode

I may change that again later, but currently the format of the
candidate tuples in pinyin mode is different from the format in
non-pinyin mode which makes this filtering fail. Therefore, pinyin
mode would not work at all at the moment if Chinese mode 2 or 3 is
selected.
Remove useless “$Id: $” from comments in the file headers

This is not used by git, it was only useful for older version
control systems.
Change the design of the database

The old database used columns like

    id|mlen|clen|m0|m1|m2|m3|m4|category|phrase|freq|user_freq

where “m0”, “m1”, “m2”, “m3”, “m4” did hold individual
characters of the input string which was therefore limited to a fixed
maximum (could be different for each table). In this case the maximum
is 5 characters. “mlen” was the length of the input string actually
used, for example if the input string was “adlt”, then “mlen = 3”,
“m0 = a”, “m1 = d”, “m2 = l”, “m3 = t”, and “m4 = None”.
“clen” was the length of the Chinese phrase matching this input
string, for example, if “adlt” is the input string for the Chinese
phrase “苦力”, then “clen = 2”. “category” was a bitmask
indicating whether that Chinese phrase is simplified Chinese,
traditional Chinese, used in both variants of Chinese, or is a mixture
of simplified and traditional Chinese. Non-Chinese tables lacked that
“category” column.

I.e. in this example the database row in system database for the old
design would have looked like:

    345|4|2|a|d|l|t|None|1|苦力|3600000|0

(“345” is an abitrary row-id)

The new database now uses only:

    id|tabkeys|phrase|freq|user_freq

“tabkeys” is the combination of “m0” ... “m4”, it is of type TEXT
and has no artificial limit (except the limit imposed by sqlite, which
is huge). So the row in the above example would look like this in
the new database design:

    345|adlt|苦力|3600000|0

“mlen” and “clen” have been removed, they just duplicate
information, the lengths can be calculated easily when
needed. “category” has been removed as well, it can be calculated
fast and easy at runtime, there is no need to save this at build time
into the system database. Omitting “category” also makes the columns
used in Chinese and non-Chinese databases identical, avoiding lots of
special cases for Chinese and non-Chinese in the code.
Remove unused functions get_invalid_input_chars() and get_invalid_input_string()
Update copyright texts in headers
Remove unused function get_index()
Use strings instead of lists for the input characters typed by the user

Makes the code more readable, avoids many complicated statements
with join, list, map, ...

The only places where it looks slightly less readable with strings
is

    c = a[-1]
    a = a[:-1]

when a is a string instead of

    c = a.pop()

when a is a list.
Instead of an array self._chars = [u'',u'',u''] use three separate strings

Using self._chars_valid, self._chars_invalid, and self._chars_prevalid
instead of self._chars[0], self._chars[1], self._chars[2] makes the
code more readable because one does not have to remember what the
indices 0, 1, 2 mean.
Introduce an environment variable IBUS_TABLE_DEBUG_LEVEL for debugging

Currently, if this variable is set to a number > 0,
system frequency and user frequency data is shown in the lookup table.
Small code readability improvement
Fix remove_cand_from_userdb() and rename it to remove_candidate_from_user_database()

After the database redesign, tabsqlitedb.remove_phrase() takes
named arguments “tabkeys” and “phrase” instead of a complete candidate
as an array.

Also, instead of deleting a candidate from the user database only
if it is user defined and thus not in the system database (candidates
which have freq=-1 in the user database are user defined), delete
the candidate always, if it is in the user database at all.

This way, the user can not only use this function to remove user defined
candidates but also to lower the priority of candidates which are
already in the system database and have aquired a high priority by
being used. Removing these candidates from the user database lowers
their priority again.
Rename self._chars_valid to self._chars_valid_when_update_candidates_was_last_called

Long name, but this makes the purpose of that variable clear.
It is used *only* to make update_candidates() return fast when
nothing has really changed.

If the users input characters have not changed, usually no update_candidates()
is necessary, but there are some exceptional cases for example when
deleting a candidate from the user database when the candidate
list changes while the user input has not changed. In these exceptional
cases, an update of the lookup table needs to be forced.
Minor code cleanup
Don’t use self._strings.pop() in pop_input() if self._strings is already empty
Simplify get_select_keys()