fcitx/fcitx — pull request #48 by yuyichao (2012-11-10 08:25:11+00:00)
Title: codegen + desktop file parser + get rid of intltool, etc....................

### Changes:
1. Utils: (basically related to desktop parser)
* string manipulating functions
* atomic operations (for desktop-parser + python....)
* small things including fcitx_expect, likely, unlikely, icd's for UT_array etc.
2. Clipboard:
* sth about remove duplicated records etc.
3. Desktop Parser:
* read from and write to desktop files
* preserve comments
* preserve order (although duplicated entries and groups are removed, therefore cannot parse gitconfig lol :laughing: ....)
* being able to be embed in users' own ones (VTable)
* add/move/remove groups and entries, modify entry values
* (i18n support can be added but no one need it now....)
* reference counting, not really well designed for concurrent access, mainly for making it safe for keeping a pointer to a structure (e.g. a conf file may keep a pointer to the confdesc file or keeping it in a python object) without worrying about whether it may be freed by others. Atomic operations are used just because it is the only way to do ref counting and it is easy to implement.
4. Codegen:
* Basically all what it can do can be found in fcitx-x11's fxaddon(.in) file (it provides a lot of different kinds functions and is therefore a good place to try and demonstrate stuffs :tongue: .......)
* several helper macros are also defined in module.h both to make my life in fcitx-scanner (the binary that generate the code) easier and to make it easier for calling functions without including the right headers (e.g. in pinyin-enhance and cloudpinyin)
5. Build system:
* goal is making it easier to add new cmake api (e.g. codegen) and deal with po/pot files (extract po strings and merging from and to different types of files) and being able to embed in other functions and adding support for new files/targets types without touching it directly. (k python here again....)
* details explained below.

### Build system
* Backward compatibility:
All old macros should still work, although it is a little hard to make them benefit from new functions.

* Add Addon:
Almost the only fcitx cmake function to call in order to add a new addon is `fcitx_add_addon_full`. With the help of `cmake_parse_arguments` (which I have renamed to `fcitx_parse_arguments` for reason described below) it handles everything including c libraries, configdesc files, conf files, codegen and even po strings (described below) the reason is there may be generated headers and generated sources etc and it is easier to handle these relations correctly if they are in a single function (another reason is about po strings.)

* PO Strings:
1. intltool is not needed with new functions (hard, if not impossible, to get rid of it in old macros...). Actually my main reason to remove it is to get rid of the empty dirs in the parent dir when running `make pot`......
2. `POTFILES.in.in` is removed, it was only needed for intltool, so it is not necessary anymore. And even with intltool, it is better to generate this file from within `CMakeLists.txt` since it can take advantage of cmake's reconfigure and re-scan dependency hook and being able to have correct dependency and check whether these files exists (some of files in the `POTFILES.in.in` were out-of-date and doesn't exist anymore...) by cmake, which is another reason to remove it. It is removed also because it is trivial to synchronize files added in a deep subdirectory with a file list almost in the root directory.....
3. absolute paths in po/pot files are removed.... (goodbye saber, good luck :smile: )
4. the handler scripts (to extract po strings and merge translation) decide whether a file can be handled from mainly the file name (although it script can do fancier thing if it wants...) (the script can be written in any languages although all of them are in sh now.)
5. po files are parsed and cached in a format based on base64 (.... based on base64 ..... sounds weird......) which shouldn't be hard to parse in any languages but is extremely easy to parse in sh.... (simply source it).
6. on `make pot`, all .po's and the main .pot is updated according to the source (so after changing/adding conf/desc files, it is not necessary to first clear the po string cache and run `make`); on `make`, mo files are generated from po files and conf is generated from conf.in and po files (Hopefully more natural than the old way.....)

### Use of shell script and dependencies
* using sh scripts
The main reason to use shell script is to merge po strings back into conf/desktop files. The most straight forward way should be using the mo files (since it is already compiled with all complex syntax and useless information removed) however, there isn't a way (AFAIK) to access that in any languages without being affected by the locale.gen.... :cry: . So parsing po files is necessary and therefore gettext is needed.
Gettext provides a set of cli tools to manipulate po files and also a c library to do similar things. It is possible (with these tools/libraries) to parse po file in both c and shell script. Shell script is used since:
1. It's shorter and simpler
2. C is tooooo overkill. There is an almost line-to-line corresponding between conf.in and conf files (rather than an almost random relation for the codegen case, where a c program is necessary) so a shell script can easily do the job.
3. Although parsing in shell script is a lot slower (basically because the execution of a large number of processes imo (~5 for each entry...).....the ja translation takes <1s to parse while the zh_CN one taks ~5s (longer with make -jn )), the result can be cached in a easy-to-parse format (as mentioned above) so the time isn't a big deal (and it is build time anyway.....).

* cmake
The only obvious cmake version dependency is CMakeParseArguments.cmake provided in cmake 2.8.3 (or .4...) but luckily, this can easily be removed by simply copying the definition of the function. Therefore, the cmake version dependency shouldn't have changed (although it is hard for me to find a cmake 2.6 to test...)

* dependencies except sh (including dev time dependencies)
1. `coreutils`:

`rm` `cat` `dirname` `realpath` `base64`
2. `sed`:
3. `gettext`:

`xgettext` `msgattrib` `msgconv` `msgexec` `msgcat` `msgmerge` `msgfmt`

* sh dependency (non-posix):
* build time: (tested on bash and dash)
* function local variables
in bash/dash although it is not POSIX maybe not impossible to remove but a little tricky and annoy
* ~mapfile~
~in bash >= 4, replaced by a funny POSIX approach~
* dev time: (basically for `make pot`)
* arrays type:
in bash (almost impossible to remove this dependency)
* bash regex and BASH_REMATCH
in bash 3+ (hard to remove)
* plus some subtle different that can only be noticed when really executing the script on a POSIX-only shell.....

So the build time dependency now is bash (hopefully any versions) or dash. Dev time dependencies is bash>=3 (probably 3.3)