Package: ner-kit
Named Entity Recognition ToolkitProvide a toolkit for rapidly extracting useful entities from text using various Python packages, includingStanza.FeaturesWe try to bring the complicated use of existing NLP toolkits down to earth by keeping APIs as simple as possible with best practice.Installationpip install ner-kitExamplesExample 1: Word segmentionfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()sw.download(lang="en")text='This is a test sentence for stanza. This is another sentence.'result1=sw.tokenize(text)sw.print_result(result1)Example 2: Chinese word segmentationfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()sw.download(lang="zh")text='我在北京吃苹果！'result1=sw.tokenize(text,lang='zh')sw.print_result(result1)Example 3: Multi-Word Token (MWT) Expansionfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()sw.download(lang="fr")text='Nous avons atteint la fin du sentier.'result1=sw.mwt_expand(text,lang='fr')sw.print_result(result1)Example 4: POS taggingfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()sw.download(lang='en')text='I like apple'result1=sw.tag(text)sw.print_result(result1)sw.download_chinese_model()text='我喜欢苹果'result2=sw.tag_chinese(text,lang='zh')sw.print_result(result2)Example 5: Named Entity Recognitionfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()sw.download(lang='en')sw.download_chinese_model()text_en='I like Beijing!'result1=sw.ner(text_en)sw.print_result(result1)text='我喜欢北京！'result2=sw.ner_chinese(text)sw.print_result(result2)Example 6: Sentiment Analysisfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()text_en='I like Beijing!'result1=sw.sentiment(text_en)sw.print_result(result1)text_zh='我讨厌苹果！'result2=sw.sentiment_chinese(text_zh)sw.print_result(result2)Example 7: Language detection from textfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()list_text=['I like Beijing!','我喜欢北京！',"Bonjour le monde!"]result1=sw.lang(list_text)sw.print_result(result1)Example 8: Language detection from text with a user-defined processing functionfromnerkit.StanzaApiimportStanzaWrapperif__name__=="__main__":sw=StanzaWrapper()list_text=['I like Beijing!','我喜欢北京！',"Bonjour le monde!"]defprocess(model):# do your own businessdoc=model["doc"]print(f"{doc.sentences[0].dependencies_string()}")result1=sw.lang_multi(list_text,func_process=process,download_lang='en,zh,fr')print(result1)sw.print_result(result1)Example 9: Stanza's NER (Legacy use for Java-based Stanford CoreNLP)fromnerkit.StanzaApiimportStanfordCoreNLPClientcorenlp_root_path=f"stanfordcorenlp/stanford-corenlp-latest/stanford-corenlp-4.3.2"corenlp=StanfordCoreNLPClient(corenlp_root_path=corenlp_root_path,language='zh')text="我喜欢游览广东孙中山故居景点！"list_token=corenlp.get_entity_list(text)fortokeninlist_token:print(f"{token['value']}\t{token['pos']}\t{token['ner']}")Example 10: Stanford CoreNLP (Not official version)importosfromnerkit.StanfordCoreNLPimportget_entity_listtext="我喜欢游览广东孙中山故居景点！"current_path=os.path.dirname(os.path.realpath(__file__))res=get_entity_list(text,resource_path=f"{current_path}/stanfordcorenlp/stanford-corenlp-latest/stanford-corenlp-4.3.2")print(res)forw,taginres:iftagin['PERSON','ORGANIZATION','LOCATION']:print(w,tag)Example 11: Open IEfromnerkit.StanzaApiimportStanfordCoreNLPClientcorenlp_root_path=f"stanfordcorenlp/stanford-corenlp-latest/stanford-corenlp-4.3.2"text="Barack Obama was born in Hawaii. Richard Manning wrote this sentence."corenlp=StanfordCoreNLPClient(corenlp_root_path=corenlp_root_path)list_result=corenlp.open_ie(text)formodelinlist_result:print(model["subject"],'--',model['relation'],'-->',model["object"])out=corenlp.force_close_server()# support force closing port in Windowsprint(out)Example 12: Generate triples from filesfromnerkit.triples.textimportgenerate_triples_from_filesinput_folder='data'output_folder='output'list_result_all=generate_triples_from_files(input_folder=input_folder,output_folder=output_folder,return_all_results=True,ltp_data_folder='../ltp_data')print(list_result_all)Example 13: Generate a list of triplsfromnerkit.triples.ltpimport*text=open("data/test.txt",'r',encoding='utf-8').read()extractor=get_ltp_triple_instance(ltp_data_folder='D:/UIBEResearch/ltp_data')list_event=get_ltp_triple_list(extractor=extractor,text=text)foreventinlist_event:print(event)Credits & ReferencesStanzaStanford CoreNLPLicenseThener-kitproject is provided byDonghua Chen.