static site search backend

A full text search using compressed `suffix trees`_ (CST).  In comparison to
a single index file like in Sphinx_ this has several advantages:

  - O(1/27 * n log n) instead of O(n) space efficency (one character prefix versus index)
  - full text search with no arbitrary limitation of the character set
  - finds all exact matches and substrings in O(log n)

For the record (index for aroun 170 posts):

  - JSON index with no special characters allowed (such as dash): 375k (132k gzipped)
  - CST  index with special characters (except punctuation): 42k (12k gzipped) per prefix

So, what is a prefix?  The idea is, that a user does mainly search for a
single term and a single prefix such as `python` where `p` is a prefix of the
term and in a suffix tree we can construct in O(n * log n) for a constant
size alphabet.  As alphabet, we use 26 lowercase ascii characters and a tree
for everything else, hence O(1/27 * n log n) space efficency per sub tree,
which is better that O(n) in practise (see the numbers above).

When the user now enters a search term, the browser only needs to load the
sub tree that contains all suffixes for this term, in average 42k (12
gzipped) with much more search features \o/.

.. _suffix trees: https://en.wikipedia.org/wiki/Suffix_tree
.. _Sphinx: http://sphinx-doc.org/
add misc/search.py to query CST

    $ acrylamid compile --search
    $ python misc/search.py blog/output/search/p.js python
    exact match: [0, 2, 3, 133, 8, 14, 15, 16, 19, 20, 22 ... ]
    partial match: [19, 85, 158, 58, 39, 22, 23]
add JavaScript API to query CST
document search API
change to directory of conf.py
remove :func:`log.once` and include unidecode egg for Python 3
rethink priority values for summarize and hyphenation filter

the current settings are document in :doc:`filters` and should
avoid multiple calculation for different paths in most environments.
fix blockdiag fontsize
Merge branch 'search'

A space-efficient and fast full text search engine for Acrylamid using
compressed `suffix trees`_ (CST).  In comparison to a single index file
like in Sphinx_ this has several advantages:

  - :math:`O(\frac{1}{27} n \log n)` instead of :math:`O(n)` space efficiency
  - full text search with no arbitrary character set limitation
  - exact and partial matches in :math:`O(m) + O(k \log n)`

For the record (index for around 170 posts):

  - JSON index with no special characters allowed (such as dash): 375k (132k gz)
  - CST  index with special characters (except punctuation): 42k (12k gz) per prefix

So, what is a prefix?  The idea is, that a user does mostly search for a
single keyword a single time and maybe with a refinement afterwards. The user
does not need to load the whole site index just to query for "python" or
"python project". With (compressed) suffix trees, it is possible to split
the index to one-character prefixes. In the example, `p` for `python` and
fortunately even for `project`. Acrylamid can construct in :math:`O(n \log n)`
for a constant size alphabet. The alphabet use 26 lowercase ascii characters
and a tree for everything else, hence :math:`O(\frac{1}{27} n \log n)` space
efficiency per sub tree. In practice (due tree compression) this is more
space-efficient than a global index (42k versus 375k in average for 170 posts).

Like Sphinx_ the index only links to the article containing the keyword and
does not provide any context.  Hence, the search view renders a plain text
version of all posts and the API provides a method to get up to N paragraphs
containing the keyword (loaded asynchronously in background).

.. _suffix trees: https://en.wikipedia.org/wiki/Suffix_tree
.. _Sphinx: http://sphinx-doc.org/