Package: pynsett
Pynsett: A programmable relation extraction toolInstallationBefore installing the package you need to install the tools for compiling python-igraphsudoapt-getinstallbuild-essentialpython-devpython3-devThe basic version can be installed by typingvirtualenv--python=/usr/bin/python3.env
pipinstallpynsettThe system is now installed, however the parser requires an additional module from Spacy and AllenNLP. You will need to typepython3-mspacydownloaden_core_web_lg
python3-mpynsettdownloadWhat is PynsettPynsett is a programmable relation extractor.
The user sets up a set of rules which are used to parse any English text.
As a result, Pynsett returns a list of triplets as defined in the rules.Example usageLet's assume we want to extract wikidata relations from a file named 'test.txt'.
An example code would befrompynsett.discourseimportDiscoursefrompynsett.extractorimportExtractorfrompynsett.auxiliary.prior_knowedgeimportget_wikidata_knowledgetext=open('test.txt').read()discourse=Discourse(text)extractor=Extractor(discourse,get_wikidata_knowledge())triplets=extractor.extract()fortripletintriplets:print(triplet)The distribution comes with two sets of rules: The generic knowledge, accessible using
pynsett.auxiliary.prior_knowledge.get_generic_knowledge(), and the wikidata knowledge, which
can be loaded using pynsett.auxiliary.prior_knowledge.get_wikidata_knowledge()Create new rules for extractionLet's assume we are writing a new file called "my_own_rules.rules".
An example of a new set of rules can be the following:MATCH"Jane#1 is an engineer#2"CREATE(HAS_ROLE12);Here the symbol #1 gives a label to 'Jane' and #2 gives a label to 'engineer'.
These labels can be used when creating the relation '(IS_A 1 2)'.A more generic rule uses the entity types (Jane is a PERSON)MATCH"{PERSON}#1 is an engineer#2"CREATE(HAS_ROLE12);This rule matches all the sentences where the subject is a person (compatibly with the internal
NER). The name of the person is associated to the node.There are 18 entity types that you can type within brackets:
CARDINAL, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, NORP, ORDINAL,
ORG, PERCENT, PERSON, PRODUCT, QUANTITY, TIME, WORK_OF_ARTThere you go, a person is now connected with a role: Node 1 is whoever matches for node 1 and
the profession is "engineer". The properties of the words are put into node 1 and 2.This seems a little bit limiting, because the previous relations only works for engineers.
Let us define aword cloudand call it "ROLE".DEFINEROLEAS[engineer,architect,physicist,doctor];MATCH"{PERSON}#1 is a ROLE#2"CREATE(HAS_ROLE12);As a final touch let us make the text a little bit nicer to the eyes: Let's use PERSON instead
of {PERSON}DEFINEPERSONAS{PERSON};DEFINEROLEAS[engineer,architect,physicist,doctor];MATCH"PERSON#1 is a ROLE#2"CREATE(HAS_ROLE12);A working example of pynsett's rules is inthis file.Use the extraction rulesIf you have a specific file with the extraction rules, you can load it by creating a new
Knowledge object:frompynsett.discourseimportDiscoursefrompynsett.extractorimportExtractorfrompynsett.knowledgeimportKnowledgetext=open('test.txt').read()discourse=Discourse(text)knowledge=Knowledge()knowledge.add_rules(open('./my_own_rules.rules').read())extractor=Extractor(discourse,knowledge)triplets=extractor.extract()fortripletintriplets:print(triplet)Import the triplets into Neo4JThe triplets can be imported into a proper graph database. As an example, let us do it for Neo4j.You would need to install the system onto your machine, as well as installing the python package
'py2neo'. After everything is set up, you can run the following script.frompy2neoimportGraphfrompynsett.discourseimportDiscoursefrompynsett.extractorimportExtractorfrompynsett.auxiliary.prior_knowedgeimportget_wikidata_knowledgeknowledge=get_wikidata_knowledge()text=open('sample_wikipedia.txt').read()discourse=Discourse(text)extractor=Extractor(discourse,knowledge)triplets=extractor.extract()graph=Graph('http://localhost:7474/db/data/')fortripletintriplets:graph.run('MERGE (a {text: "%s"}) MERGE (b {text: "%s"}) CREATE (a)-[:%s]->(b)'%(triplet[0],triplet[2],triplet[1]))This script works on an example page called 'sample_wikipedia.txt' that you will have to provide.Using the internal Web ServerTo start the internal web server you can write the followingfrompynsett.serverimportpynsett_apppynsett_app.run(debug=True,port=4001,host='0.0.0.0',use_reloader=False)which will open a flask app atlocalhost:4001.Web interfaceThe server provides three web interfaces:A Wikidata relation extractor athttp://localhost:4001/wikidataA Programmable relation extractor athttp://localhost:4001/relationsA Neo-Davidsonian representation of a text athttp://localhost:4001APIThe wikidata relation extractor API can be called withimportjsonimportrequeststext="John is a writer."triplets=json.loads(requests.post('http://localhost:4001/api/wikidata',json={'text':text}).text)print(triplets)with output:[['John','JOB_TITLE','writer']]The rules can programmed by posting as in the followingimportjsonimportrequestsrules="""DEFINE PERSON AS{PERSON};DEFINE ORG AS{ORG};DEFINE ROLE AS [engineer, author, doctor, researcher];MATCH "PERSON#1 was ROLE at ORG#2"CREATE (WORKED_AT 1 2);"""triplets=json.loads(requests.post('http://localhost:4001/api/set_rules',json={'text':rules}).text)These rules are then used at the following API endpointimportjsonimportrequeststext="Isaac Asimov was an American writer and professor of biochemistry at Boston University."triplets=json.loads(requests.post('http://localhost:4001/api/relations',json={'text':text}).text)print(triplets)The Neo-Davidsonian representation API can be called withimportjsonimportrequeststext="John is tall."graph=json.loads(requests.post('http://localhost:4001/api/drt',json={'text':text}).text)print(graph)with output:{'edges':[{'arrows':'to','from':'v1','label':'AGENT','to':'v0'},{'arrows':'to','from':'v1','label':'ADJECTIVE','to':'v2'}],'nodes':[{'id':'v1','label':'is'},{'id':'v0','label':'John'},{'id':'v2','label':'tall'}]}Pre-Formatting of the TextThe text must be submitted respecting the following rules:No parenthesis (...) nor brackets [...]. The parser is confused by those.The paragraphs must be separated by1 empty line. Dividing a text into paragraphs helps with anaphora.Thisisparagraph1.

Thisisparagraph2.Known issues and shortcomingsSpeed! Parsing is done one sentence at a timeAnaphora only works inside paragraphsAnaphora is done through AllenNLP, with can be slow-ish without a GPUThe text needs to be cleaned and pre-formatted. This is not an issueper sebut it must be kept in mind