cihaidata-unihan 0.4.2
Tool to build UNIHAN dataset into datapackage / simple data format.
License: MIT
Keywords: cihaidata-unihan

*cihaidata-unihan* - tool to build `unihan`_ into `simple data format`
CSV format. Part of the `cihai`_ project.

|pypi| |docs| |build-status| |coverage| |license|

Unihan's data is disperved across multiple files in the format of::

    U+3400	kCantonese	jau1
    U+3400	kDefinition	(same as U+4E18 丘) hillock or mound
    U+3400	kMandarin	qiū
    U+3401	kCantonese	tim2
    U+3401	kDefinition	to lick; to taste, a mat, bamboo bark
    U+3401	kHanyuPinyin	10019.020:tiàn
    U+3401	kMandarin	tiàn

``cihaidata_unihan/process.py`` will download Unihan.zip and build all files into a
single tabular CSV (default output: ``./data/unihan.csv``)::

    char,ucn,kCantonese,kDefinition,kHanyuPinyin,kMandarin
    丘,U+3400,jau1,(same as U+4E18 丘) hillock or mound,,qiū
    㐁,U+3401,tim2,"to lock; to taste, a mat, bamboo bark",10019.020:"tiàn,tiàn"

``process.py`` supports command line arguments. See `cihaidata_unihan/process.py CLI
arguments`_ for information on how you can specify custom columns, files,
download URL's and output destinations.

Being built against unit tests. See the `Travis Builds`_ and
`Revision History`_.

.. _cihai: https://cihai.git-pull.com
.. _cihai-handbook: https://github.com/cihai/cihai-handbook
.. _cihai team: https://github.com/cihai?tab=members
.. _cihai-python: https://github.com/cihai/cihai-python
.. _cihaidata-unihan on github: https://github.com/cihai/cihaidata-unihan

Usage
-----

To download and build your own ``unihan.csv``:

.. code-block:: bash

    $ ./cihaidata_unihan/process.py

Creates ``data/unihan.csv``.

See `cihaidata_unihan/process.py CLI arguments`_ for advanced usage examples.

.. _cihaidata_unihan/process.py CLI arguments: http://cihaidata-unihan.readthedocs.org/en/latest/cli.html

Structure
---------

.. code-block:: bash

    # dataset metadata, schema information.
    datapackage.json

    # (future) when this package is stable, unihan.csv will be provided
    data/unihan.csv

    # stores downloaded Unihan.zip and it's txt file contents (.gitignore'd)
    data/build_files/

    # script to download + build a SDF csv of unihan.
    cihaidata_unihan/process.py

    # unit tests to verify behavior / consistency of builder
    tests/*

    # python 2/3 compatibility modules
    cihaidata_unihan/_compat.py
    cihaidata_unihan/unicodecsv.py

    # python module, public-facing python API.
    __init__.py
    cihaidata_unihan/__init__.py

    # utility / helper functions
    cihaidata_unihan/util.py


Cihai is *not* required for:

- ``data/unihan.csv`` - `simple data format`_ compatible csv file.
- ``cihaidata_unihan/process.py`` - create a ``data/unihan.csv``.

When this module is stable, ``data/unihan.csv`` will have prepared
releases, without requires using ``cihaidata_unihan/process.py``. ``process.py``
will not require external libraries.

Examples
--------

- https://github.com/datasets/gdp
- https://github.com/datasets/country-codes

Related links:

- CSV *Simple Data Format* (SDF): http://data.okfn.org/standards/simple-data-format
- Tools: http://data.okfn.org/tools


.. _Travis Builds: https://travis-ci.org/cihai/cihaidata-unihan/builds
.. _Revision History: https://github.com/cihai/cihaidata-unihan/commits/master
.. _cjklib: http://cjklib.org/0.3/
.. _current datasets: http://cihai.readthedocs.org/en/latest/api.html#datasets
.. _permissively licensing your dataset: http://cihai.readthedocs.org/en/latest/information_liberation.html

==============  ==========================================================
Python support  Python 2.7, >= 3.3, pypy/pypy3
Source          https://github.com/cihai/cihaidata-unihan
Docs            https://cihaidata-unihan.git-pull.com
Changelog       https://cihaidata-unihan.git-pull.com/en/latest/history.html
API             https://cihaidata-unihan.git-pull.com/en/latest/api.html
Issues          https://github.com/cihai/cihaidata-unihan/issues
Travis          https://travis-ci.org/cihai/cihaidata-unihan
Test coverage   https://codecov.io/gh/cihai/cihaidata-unihan
pypi            https://pypi.python.org/pypi/cihaidata-unihan
OpenHub         https://www.openhub.net/p/cihaidata-unihan
License         `MIT`_.
git repo        .. code-block:: bash

                    $ git clone https://github.com/cihai/cihaidata-unihan.git
install dev     .. code-block:: bash

                    $ git clone https://github.com/cihai/cihaidata-unihan.git cihai
                    $ cd ./cihai
                    $ virtualenv .env
                    $ source .env/bin/activate
                    $ pip install -e .
tests           .. code-block:: bash

                    $ python setup.py test
==============  ==========================================================

.. _MIT: http://opensource.org/licenses/MIT
.. _Documentation: http://cihai.readthedocs.org/en/latest/
.. _API: http://cihai.readthedocs.org/en/latest/api.html
.. _Unihan: http://www.unicode.org/charts/unihan.html
.. _datapackages: http://dataprotocols.org/data-packages/
.. _datapackage.json format: https://github.com/datasets/gdp/blob/master/datapackage.json
.. _json table schema: http://dataprotocols.org/json-table-schema/
.. _simple data format: http://data.okfn.org/standards/simple-data-format
.. _cihai dataset API: http://cihai.readthedocs.org/en/latest/extending.html
.. _PEP 301\: python package format: http://www.python.org/dev/peps/pep-0301/

.. |pypi| image:: https://img.shields.io/pypi/v/cihaidata-unihan.svg
    :alt: Python Package
    :target: http://badge.fury.io/py/cihaidata-unihan

.. |build-status| image:: https://img.shields.io/travis/cihai/cihaidata-unihan.svg
   :alt: Build Status
   :target: https://travis-ci.org/cihai/cihaidata-unihan

.. |coverage| image:: https://codecov.io/gh/cihai/cihaidata-unihan/branch/master/graph/badge.svg
    :alt: Code Coverage
    :target: https://codecov.io/gh/cihai/cihaidata-unihan

.. |license| image:: https://img.shields.io/github/license/cihai/cihaidata-unihan.svg
    :alt: License 

.. |docs| image:: https://readthedocs.org/projects/cihaidata-unihan/badge/?version=latest
    :alt: Documentation Status
    :scale: 100%
    :target: https://readthedocs.org/projects/cihaidata-unihan/