Package: expertai-nlapi
expert.ai Natural Language API client for PythonPython client for the expert.ai Natural Language API adds Natural Language understanding capabilities to your Python apps.
The client can use either the Cloud basedNatural Language APIorStudio Local Deployment Agent.APIs capabilitesMake reference to theNatural Language APIandStudio Local Deployment Agentdocumentation to know more about the APIs capabilities.Here is a side-by-side comparison of the two APIs:CapabilityNatural Language APIStudio Local Deployment AgentWhere does it run?In the Cloud, shared by all usersOn user's PCDocument analysis:YESYESDocument classification with ready-to-use taxonomies (IPTC Media topics, GeoTax, Emotional traits, Behavioral traits)YESNOCustom document classificationNOYES*Information detection with ready-to-use detectors (Personally Identifiable Information (PII), Writeprint, Temporal information, ESG Sentiment, Hate speech)YESNOCustom information extractionNOYES*Document size limit?YES (<= 10KB)NODocument number limit?NOSee theService Level AgreementCharacters limit?YES (<= 10 million characters per month when using the free tier)NO* Based onStudioproject.InstallationTo install the client library withpip:pipinstallexpertai-nlapiTo install usingconda:condainstall-cconda-forgeexpertai-nlapiUsageAuthorizationYou need an expert.ai developer account to use the APIs and you can get one for free registering on the expert.aideveloper portal.Your developer account credentials must be specified as environment variables:Linux:exportEAI_USERNAME=YOUR_USERNAMEexportEAI_PASSWORD=YOUR_PASSWORDWindows:SETEAI_USERNAME=YOUR_USERNAME
SETEAI_PASSWORD=YOUR_PASSWORDReplaceYOUR_USERNAMEwith the email address you specyfied during registration andYOUR_PASSWORDwith the actual account password.You can also define credentials inside your code:importosos.environ["EAI_USERNAME"]='YOUR_USERNAME'os.environ["EAI_PASSWORD"]='YOUR_PASSWORD'Create the clientThe next thing to do inside your code is importing the client section of the library and instantiating the client based on the API you want to use.To use Natural Language API:fromexpertai.nlapi.cloud.clientimportExpertAiClientclient=ExpertAiClient()To use Studio Local Deployment Agent:fromexpertai.nlapi.edge.clientimportExpertAiClientclient=ExpertAiClient()Access capabilities and self-documentation resourcesIn thesamplesdirectory of this repository you can find ready-to-run scripts showing how to access all API capabilities for English texts and API self-documentation resources.Scripts are listed and described in the tables below.Natural Language APIYou can find these scripts under the/samples/natural language apifolder.CapabilitySampleDocument analysis,standardcontext,full analysisdocument analysis/full.pyDocument analysis,standardcontext, sub-analysis:Deep linguistic analysisdocument analysis/deep-linguistic-analysis.pyDocument analysis,standardcontext, sub-analysis:Keyphrase extractiondocument analysis/keyphrase-extraction.pyDocument analysis,standardcontext, sub-analysis:Named entity recognitiondocument analysis/named-entity-recognition.pyDocument analysis,standardcontext, sub-analysis:Relation extractiondocument analysis/relation-extraction.pyDocument analysis,standardcontext, sub-analysis:Sentiment analysisdocument analysis/sentiment-analysis.pyDocument analysis, self-documemtation esources: list of the availablecontextsdocument analysis/contexts.pyDocument classification withiptctaxonomydocument classification/iptc.pyDocument classification withgeotaxtaxonomydocument classification/geotax.pyDocument classification withemotional-traitstaxonomydocument classification/emotional-traits.pyDocument classification withbehavioral-traitstaxonomydocument classification/behavioral-traits.pyDocument classification, self-documentation resources: list of available taxonomiesdocument classification/taxonomies.pyDocument classification, self-documentation resources: category tree of theiptctaxonomydocument classification/category-tree-iptc.pyDocument classification, self-documentation resources: category tree of thegeotaxtaxonomydocument classification/category-tree-geotax.pyDocument classification, self-documentation resources: category tree of theemotional-traitstaxonomydocument classification/category-tree-emotional-traits.pyDocument classification, self-documentation resources: category tree of thebehavioral-traitstaxonomydocument classification/category-tree-behavioral-traits.pyInformation detection withpiidetectorinformation detection/pii.pyInformation detection withwriteprintdetectorinformation detection/writeprint.pyInformation detection withtemporal-informationdetectorinformation detection/temporal-information.pyInformation detection withesg-sentimentdetectorinformation detection/esg-sentiment.pyInformation detection withhate-speechdetectorinformation detection/hate-speech.pyInformation detection, self-documentation resources: list of available detectorsinformation detection/detectors.pyStudio Local Deployment AgentYou can find these scripts under the/samples/studio ldafolder.CapabilitySampleDocument analysis,full analysisdocument analysis/full.pyDocument analysis, sub-analysis:Deep linguistic analysisdocument analysis/deep-linguistic-analysis.pyDocument analysis, sub-analysis:Keyphrase extractiondocument analysis/keyphrase-extraction.pyDocument analysis, sub-analysis:Named entity recognitiondocument analysis/named-entity-recognition.pyDocument analysis, sub-analysis:Relation extractiondocument analysis/relation-extraction.pyDocument analysis, sub-analysis:Sentiment analysisdocument analysis/sentiment-analysis.pyChanging the languageNatural Language API is capable of processing texts of various languages.
You can use self-documentations resources likecontextsandtaxonomies—look for the corresponding ready-to-run scripts in the first of the two tables above—to determine if a functionality is available for a given language.If it's available, specify thelanguageparameter in your code, for example:text="Michael Jordan was one of the best basketball players of all time. Scoring was Jordan's stand-out skill, but he still holds a defensive NBA record, with eight steals in a half."language='en'output=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'disambiguation'})See also the documentation pages aboutcontextsandtaxonomies.Studio Local Deployment Agent supports the language of your Studio project.More examples of document analysisStart with deep linguistic analysis...To perform thedeep linguistic analysisof a text:Natural Language API:output=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'disambiguation'})Studio Local Deployment Agent:output=client.deep_linguistic_analysis(text)...then examine tokenization and lemmatization...Lemmatization looks beyond word reduction, using a language's full vocabulary to apply amorphological analysisto words. The lemma of 'was' is 'be' and the lemma of 'mice' is 'mouse'. Further, the lemma of 'meeting' might be 'meet' or 'meeting' depending on its use in a sentence.print(f'{"TOKEN":{20}}{"LEMMA":{8}}')fortokeninoutput.tokens:print(f'{text[token.start:token.end]:{20}}{token.lemma:{8}}')TOKEN                LEMMA   
Facebook             Facebook Inc.
is                   is      
looking at           look at 
buying               buy     
an                   an      
American             American
startup              startup 
for                  for     
$6 million           6,000,000 dollar
based                base    
in                   in      
Springfield, IL      Springfield
.                    ....Part of Speech (PoS) tagging...Analysis determines the part-of-speech of tokens. PoS labels are from theUniversal Dependenciesframework.print(f'{"TOKEN":{18}}{"PoS":{4}}')fortokeninoutput.tokens:print(f'{text[token.start:token.end]:{18}}{token.pos:{4}}')TOKEN              PoS   
Facebook           PROPN  
is                 AUX    
looking at         VERB   
buying             VERB   
an                 DET    
American           ADJ    
startup            NOUN   
for                ADP    
$6 million         NOUN   
based              VERB   
in                 ADP    
Springfield, IL    PROPN  
.                  PUNCT...and dependency parsing informationThe analysis returns the dependency parsing information assigned to each token, using the Universal Dependencies framework as well.print(f'{"TOKEN":{18}}{"Dependency label":{8}}')fortokeninoutput.tokens:print(f'{text[token.start:token.end]:{18}}{token.dependency.label:{4}}')TOKEN              Dependency label
Facebook           nsubj 
is                 aux  
looking at         root 
buying             advcl 
an                 det  
American           amod 
startup            obj  
for                case 
$6 million         obl  
based              acl  
in                 case 
Springfield, IL    obl  
.                  punctNamed entity recognitionGoing a step beyond linguistic analysis,named entitiesadd another layer of context.  Named entities are recognized by theentitiessub-analysis.Natural Language API:output=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'entities'})Studio Local Depolyment Agent:output=client.named_entity_recognition(text)Printing results:print(f'{"ENTITY":{40}}{"TYPE":{10}}')forentityinoutput.entities:print(f'{entity.lemma:{40}}{entity.type_:{10}}')ENTITY               TYPE
6,000,000 dollar     MON        
Springfield          GEO        
Facebook Inc.        COMIn addition to the entity type, the API provides some metadata from Linked Open Data sources such as WikiData and GeoNames.
For example, you can get the open data connected with the entitySpringfield, ILprint(output.entities[1].lemma)Springfieldforentryinoutput.knowledge:if(entry.syncon==document.entities[1].syncon):forpropinentry.properties:print(f'{prop.type_:{12}}{prop.value:{30}}')Coordinate   Lat:39.47.58N/39.799446;Long:89.39.18W/-89.654999
DBpediaId    dbpedia.org/page/Springfield  
GeoNamesId   4250542                       
WikiDataId   Q28515Springfield has been recognized asQ28515on Wikidata, that is the Q-id for Springfield, IL (i.e.not for Springfield in Vermont o in California)Key elementsKey elements, the result of keyphrase extraction, are obtained with therelevantssub-analysis and identified from the document as main sentences, main concepts (called "syncons"), main lemmas and relevant Knowledge Graph topics.Let's focus on the main lemmas of the document; each lemma is provided with a relevance score.Natural Language API:document=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'relevants'})Studio Local Deployment Agent:document=client.keyphrase_extraction(text)print(f'{"LEMMA":{20}}{"SCORE":{5}}')formainlemmaindocument.main_lemmas:print(f'{mainlemma.value:{20}}{mainlemma.score:{5}}')LEMMA                SCORE 
Facebook Inc.         43.5
startup               40.4
Springfield             15Sentiment analysisSentimentis obtained with thesentimentsub-analysis and expresses thepolarity—positive or negative—and the intensity of the tone of the text is.text='Today is a good day. I love to go to mountain.'Natural Language API:document=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'sentiment'})Studio Local Deployment Agent:document=client.sentiment(text)Printing results:print("sentiment:",document.sentiment.overall)RelationsRelationsare obtained with therelationssub-analysis that labels concepts expressed in the text with their semantic role with respect to the verb they are connected to.text='John sent a letter to Mary.'Natural Language API:document=client.specific_resource_analysis(body={"document":{"text":text}},params={'language':language,'resource':'relations'})Studio Local Deployment Agent:document=client.relations(text)Printing results:forrelindocument.relations:print("Verb:",rel.verb.lemma)forrinrel.related:print("Relation:",r.relation,"Lemma:",r.lemma)More examples of document classificationLet's see how to classify documents according to theIPTC Media Topics Taxonomyprovided by the Natural Language API; then we'll use the matplot lib to display categorization results as a bar chart.text="""Strategic acquisitions have been important to the growth of Facebook (FB).Mark Zuckerberg founded the company in 2004, and since then it has acquired scores of companies,ranging from tiny two-person start-ups to well-established businesses such as WhatsApp. For 2019,Facebook reported 2.5 billion monthly active users (MAU) and $70.69 billion in revenue."""importmatplotlib.pyplotasplt%matplotlibinlineplt.style.use('ggplot')taxonomy='iptc'document=client.classification(body={"document":{"text":text}},params={'taxonomy':taxonomy,'language':language})categories=[]scores=[]print(f'{"CATEGORY":{27}}{"IPTC ID":{10}}{"FREQUENCY":{8}}')forcategoryindocument.categories:categories.append(category.label)scores.append(category.frequency)print(f'{category.label:{27}}{category.id_:{10}}{category.frequency:{8}}')CATEGORY                    ID           FREQUENCY
Earnings                    20000178     29.63
Social networking           20000769     21.95plt.bar(categories,scores,color='#17a2b8')plt.xlabel("Categories")plt.ylabel("Frequency")plt.title("Media Topics Classification")plt.show()If yourStudioproject performs document classification, to request it to Studio Local Deployment Agent use:document=client.classification(text)Results structure is the same as for the Natural Language API.Information detection exampleInformation detectionleverages deep linguistic analysis to extract particular types of information from the text.For example, the Personal Identifiable Informtion (PII) detector of the Natural Language API extract personal information such as people names, dates, addresses, telephone numbers, etc. that could be considered "sensitive".text='Longtime TFS News reporter Marcus Smith died unexpectedly Monday at the age of 60'Natural Language API:document=client.detection(body={"document":{"text":text}},params={'language':language,'detector':'pii'})forextractionindocument.extractions:print("Template:",extraction.template)forfieldinextraction.fields:print("field: ",field.name," value: ",field.value)forpositioninfield.positions:print("start: ",position.start,"end: ",position.end)If yourStudioproject performs information extraction, to request it to Studio Local Deployment Agent use:document=client.extraction(text)Results structure is the same as for the Natural Language API....and if you made it this far...Good job! You're an expert in the expert.ai community! :clap: :tada:Check out other language SDKs available on ourGithub page.Contribute!Clone this GitHub repository and run the following script:$cdnlapi-python
$pipinstall-rrequirements-dev.txtAs good practice it's recommended to work in an isolated Python environment, creating a virtual environment withvirtualenv packagebefore building the package. You can create your environment with the command:$virtualenvexpertai
$sourceexpertai/bin/activate