spacy-lookup 0.1.0
spaCy pipeline component for Named Entity Recognition based on dictionaries.
License: MIT

spacy-lookup: Named Entity Recognition based on dictionaries
************************************************************

`spaCy v2.0 <https://spacy.io/usage/v2>`_ extension and pipeline component
for adding Named Entities metadata to ``Doc`` objects. Detects Named Entities
using dictionaries. The extension sets the custom ``Doc``,
``Token`` and ``Span`` attributes ``._.is_entity``, ``._.entity_type``,
``._.has_entities`` and ``._.entities``.

Named Entities are matched using the python module ``flashtext``, and
looks up in the data provided by different dictionaries.

Installation
===============

``spacy-lookup`` requires ``spacy`` v2.0.16 or higher.

.. code:: bash

    pip install spacy-lookup

Usage
=====
First, you need to download a language model.

.. code:: bash

    python -m spacy download en

Import the component and initialise it with the shared ``nlp`` object (i.e. an
instance of ``Language``), which is used to initialise ``flashtext``
with the shared vocab, and create the match patterns. Then add the component
anywhere in your pipeline.

.. code:: python

    import spacy
    from spacy_lookup import Entity

    nlp = spacy.load('en')
    entity = Entity(keywords_list=['python', 'product manager', 'java platform'])
    nlp.add_pipe(entity, last=True)

    doc = nlp(u"I am a product manager for a java and python.")
    assert doc._.has_entities == True
    assert doc[0]._.is_entity == False
    assert doc[3]._.entity_desc == 'product manager'
    assert doc[3]._.is_entity == True

    print([(token.text, token._.canonical) for token in doc if token._.is_entity])


``spacy-lookup`` only cares about the token text, so you can use it on a blank
``Language`` instance (it should work for all
`available languages <https://spacy.io/usage/models#languages>`_!), or in
a pipeline with a loaded model. If you're loading a model and your pipeline
includes a tagger, parser and entity recognizer, make sure to add  the entity
component as ``last=True``, so the spans are merged at the end of the pipeline.

Available attributes
--------------------

The extension sets attributes on the ``Doc``, ``Span`` and ``Token``. You can
change the attribute names on initialisation of the extension. For more details
on custom components and attributes, see the
`processing pipelines documentation <https://spacy.io/usage/processing-pipelines#custom-components>`_.

====================== ======= ===
``Token._.is_entity``   bool    Whether the token is an entity.
``Token._.entity_type`` unicode A human-readable description of the entity.
``Doc._.has_entities``    bool    Whether the document contains entity.
``Doc._.entities``        list    ``(entity, index, description)`` tuples of the document's entities.
``Span._.has_entities``   bool    Whether the span contains entity.
``Span._.entities``       list    ``(entity, index, description)`` tuples of the span's entities.
====================== ======= ===

Settings
--------

On initialisation of ``Entity``, you can define the following settings:

=============== ============ ===
``nlp``         ``Language`` The shared ``nlp`` object. Used to initialise the matcher with the shared ``Vocab``, and create ``Doc`` match patterns.
``attrs``       tuple        Attributes to set on the ._ property. Defaults to ``('has_entities', 'is_entity', 'entity_type', 'entity')``.
``keywords_list``      list         Optional lookup table with the list of terms to look for.
``keywords_dict``      dict         Optional lookup table with the list of terms to look for.
``keywords_file``      string         Optional filename with the list of terms to look for.
=============== ============ ===

.. code:: python

    entity = Entity(nlp, keywords_list=['python', 'java platform'], label='ACME')
    nlp.add_pipe(entity)
    doc = nlp(u"I am a product manager for a java platform and python.")
    assert doc[3]._.is_entity