DomainThesaurus 1.2.3
extract domain thesaurus automatically
License: MIT

DomainThesaurus
================

Introduction
------------

**DomainThesaurus** is a python package offering a technique of extracting domain-specific
thesaurus which is commonly used in *Natural Language Processing*. Here is one item of generated
thesaurus::

    { "internet explorer":
    {"abbreviation":["ie"],
    "synonym":["internet explorers", "internet explorere", "internetexplorer"],
    "other":["firefox","chrome","opera"]}
    }

Except for domain-specific thesaurus, the package also provides several useful modules.
For example, **DomainTerm** for extracting domain-specific term and **WordDiscrimination**
for discriminate words (e.g. abbreviation, synonyms).
Details of the implemented approaches can be found in our publication:
`SEthesaurus: WordNet in Software Engineering
<https://ieeexplore.ieee.org/document/8827962>`_. (IEEE Transactions on Software Engineering 2019)


Domain-Specific term
::::::::::::::::::::::::::::::

**DomainTerm** can automatically extract domain-specific terms from domain corpus.
For example, *Javascript* in the domain of  computer science and technology and *karush kuhn tucker* in
domain of mathematics.

Abbreviations and Synonyms
:::::::::::::::::::::::::::

The module **WordDiscrimination** can divide semantic-related words into different types.
The default module can recognize semantic-related words as `abbreviation` and `synonym`. Note that,
in our module, the `synonym` means that two words are semantic-related word and they are morphologically similar.
For example, *ie* is the abbreviation of *internet explorer* and *javascripts* is
the synonym of *javascript*.

Installation
------------

**DomainThesaurus** is tested to work under `Python 3.x`. Please use it in `Python 3.x`.
We will try to support *Python 2.x*.

Dependency requirements:

* gensim(>=3.6.0)
* networkx(>=2.1)

**DomainThesaurus** is currently available on the PyPi's repository and you can
install it via `pip`::

  pip install DomainThesaurus

If you prefer, you can clone it and run the setup.py file. Use the following
command to get a copy from GitHub::

 git clone https://github.com/DunZhang/DomainSpecificThesaurus.git


Usage
----------

A simple example::
    >>> dst = DomainThesaurus(domain_specific_corpus_path="your domain specific corpus path",
    >>>                       general_vocab_path="your general vocab path",
    >>>                       outputDir="path of output")
    >>> # extract domain thesauruss
    >>> your_thesaurus = dst.extract()

If you don't have any datasets, you can copy and run this code:
https://github.com/DunZhang/DomainSpecificThesaurus/blob/master/docs/notebooks/domain_thesaurus.ipynb .
This code will automatically download datasets for you.
The code design is flexible, you can replace the default `function class` with your own `function class` to get better
performance.
You can find more usage in https://github.com/DunZhang/DomainSpecificThesaurus/blob/master/docs/notebooks

Acknowledgements
-----------------

In this project, we use `Levenshtein Distance` and `GoogleDriveDownloader` from https://pypi.org/project/jellyfish/
and  https://github.com/ndrplz/google-drive-downloader. Thanks for their code.