Entity extraction
Entity extraction The Watson Natural Language Processing Entity extraction models extract entities from input text. For details, on available extraction types, refer to these sections: * [Machine-learning-based extraction for general entities](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enmachine-learning-general) * [Machine-learning-based extraction for PII entities](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enmachine-learning-pii) * [Rule-based extraction for general entities](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enrule-based-general) * [Rule-based extraction for PII entities](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enrule-based-pii) Machine-learning-based extraction for general entities The machine-learning-based extraction models are trained on labeled data for the more complex entity types such as person, organization and location. Capabilities The entity models extract entities from the input text. The following types of entities are recognized: * Date * Duration * Facility * Geographic feature * Job title * Location * Measure * Money * Ordinal * Organization * Person * Time Capabilities of machine-learning-based extraction based on an example Capabilities Examples Extracts entities from the input text. IBM's CEO Arvind Krishna is based in the US -> IBMOrganization , CEOJobTitle, Arvind KrishnaPerson, USLocation Available workflows and blocks differ, depending on the runtime used. Blocks and workflows for handling general entities with their corresponding runtimes Block or workflow name Available in runtime entity-mentions_transformer-workflow_multilingual_slate.153m.distilled [Runtime 23.1](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enruntime-231) entity-mentions_transformer-workflow_multilingual_slate.153m.distilled-cpu [Runtime 23.1](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enruntime-231) entity-mentions_bert_multi_stock [Runtime 22.2](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-entity-enhanced.html?context=cdpaas&locale=enruntime-222) Machine-learning-based workflows for general entities in Runtime 23.1 Workflow names * entity-mentions_transformer-workflow_multilingual_slate.153m.distilled: this workflow can be used on both CPUs and GPUs. * entity-mentions_transformer-workflow_multilingual_slate.153m.distilled-cpu: this workflow is optimized for CPU-based runtimes. Supported languages Entity extraction is available for the following languages. For a list of the language codes and the corresponding language, see [Language codes](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-catalog.htmllang-codes): ar, cs, da, de, en, es, fi, fr, he, hi, it, ja, ko, nb, nl, nn, pt, ro, ru, sk, sv, tr, zh-cn Code sample import watson_nlp Load the workflow model entities_workflow = watson_nlp.load('entity-mentions_transformer-workflow_multilingual_slate.153m.distilled') Run the entity extraction workflow on the input text entities = entities_workflow.run('IBM's CEO Arvind Krishna is based in the US', language_code="en") print(entities.get_mention_pairs()) Output of the code sample: [('IBM', 'Organization'), ('CEO', 'JobTitle'), ('Arvind Krishna', 'Person'), ('US', 'Location')] Machine-learning-based blocks for general entities in Runtime 22.2 Block namesentity-mentions_bert_multi_stock Supported languages Entity extraction is available for the following languages. For a list of the language codes and the corresponding language, see [Language codes](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/watson-nlp-block-catalog.htmllang-codes). ar, cs, da, de, en, es, fi, fr, he, hi, it, ja, ko, nb, nl, nn, pt, ro, ru, sk, sv, tr, zh-cn Dependencies on other blocks The following block must run before you can run the Entity extraction block: * syntax_izumo_<language>_stock Code sample import watson_nlp Load Syntax Model for English, and the multilingual BERT Entity model syntax_model = watson_nlp.load('syntax_izumo_en_stock') bert_entity_model = watson_nlp.load('entity-mentions_bert_multi_stock') Run the syntax model on the input text syntax_prediction = syntax_model.run('IBM's CEO Arvind Krishna is based in the US') Run the entity mention model on the result of syntax model bert_entity_mentions = bert_entity_model.run(syntax_prediction) print(bert_entity_mentions.get_mention_pairs()) Output of the code sample: [('IBM', 'Organization'), ('CEO', 'JobTitle'), ('Arvind Krishna', 'Person'), ('US', 'Location')] Machine-learning-based extraction for PII entities Block namesentity-mentions_bilstm_en_pii Blocks for handling Personal Identifiable Information (PII) entities with their corresponding runtimes Block name Available in runtime entity-mentions_bilstm_en_pii Runtime 22.2, Runtime 23.1 The entity-mentions_bilstm_en_pii machine-learning based extraction model is trained on labeled data for types person and location. Capabilities The entity-mentions_bilstm_en_pii block recognizes the following types of entities: Entities extracted by the entity-mentions_bilstm_en_pii block Entity type name Description Supported languages Location All geo-political regions, continents, countries, and street names, states, provinces, cities, towns or islands. en Person Any being; living, nonliving, fictional or real. en Dependencies on other blocks The following block must run before you can run the entity-mentions_bilstm_en_pii block: * syntax_izumo_en_stock Code sample import os import watson_nlp Load Syntax and a Entity Mention BiLSTM model for English syntax_model = watson_nlp.load('syntax_izumo_en_stock') entity_model = watson_nlp.load('entity-mentions_bilstm_en_pii') text = 'Denver is the capital of Colorado. The total estimated government spending in Colorado in fiscal year 2016 was $36.0 billion. IBM office is located in downtown Denver. Michael Hancock is the mayor of Denver.' Run the syntax model on the input text syntax_prediction = syntax_model.run(text) Run the entity mention model on the result of the syntax analysis entity_mentions = entity_model.run(syntax_prediction) print(entity_mentions) Output of the code sample: { "mentions": [ { "span": { "begin": 0, "end": 6, "text": "Denver" }, "type": "Location", "producer_id": { "name": "BiLSTM Entity Mentions", "version": "1.0.0" }, "confidence": 0.6885626912117004, "mention_type": "MENTT_UNSET", "mention_class": "MENTC_UNSET", "role": "" }, { "span": { "begin": 25, "end": 33, "text": "Colorado" }, "type": "Location", "producer_id": { "name": "BiLSTM Entity Mentions", "version": "1.0.0" }, "confidence": 0.8509215116500854, "mention_type": "MENTT_UNSET", "mention_class": "MENTC_UNSET", "role": "" }, { "span": { "begin": 78, "end": 86, "text": "Colorado" }, "type": "Location", "producer_id": { "name": "BiLSTM Entity Mentions", "version": "1.0.0" }, "confidence": 0.9928259253501892, "mention_type": "MENTT_UNSET", "mention_class": "MENTC_UNSET", "role": "" }, { "span": { "begin": 151, "end": 166, "text": "downtown Denver" }, "type": "Location", "producer_id": { "name": "BiLSTM Entity Mentions", "version": "1.0.0" }, "confidence": 0.48378944396972656, "mention_type": "MENTT_UNSET", "mention_class": "MENTC_UNSET", "role": "" }, { "span": { "begin": 168, "end": 183, "text": "Michael Hancock" }, "type": "Person", "producer_id": { "name": "BiLSTM Entity Mentions",