Package: breds
BREDSBREDS extracts relationships using a bootstrapping/semi-supervised approach, it relies on an initial set of
seeds, i.e. paris of named-entities representing relationship type to be extracted.The algorithm expands the initial set of seeds using distributional semantics to generalize the relationship while
limiting the semantic drift.Extracting companies headquarters:The input text needs to have the named-entities tagged, like show in the example bellow:The tech company <ORG>Soundcloud</ORG> is based in <LOC>Berlin</LOC>, capital of Germany.<ORG>Pfizer</ORG> says it has hired <ORG>Morgan Stanley</ORG> to conduct the review.<ORG>Allianz</ORG>, based in <LOC>Munich</LOC>, said net income rose to EUR 1.32 billion.<LOC>Switzerland</LOC> and <LOC>South Africa</LOC> are co-chairing the meeting.<LOC>Ireland</LOC> beat <LOC>Italy</LOC> , then lost 43-31 to <LOC>France</LOC>.<ORG>Pfizer</ORG>, based in <LOC>New York City</LOC> , employs about 90,000 workers.<PER>Burton</PER> 's engine passed <ORG>NASCAR</ORG> inspection following the qualifying session.We need to give seeds to boostrap the extraction process, specifying the type of each named-entity and
relationships examples that should also be present in the input text:e1:ORGe2:LOCLufthansa;CologneNokia;EspooGoogle;Mountain ViewDoubleClick;New YorkSAP;WalldorfTo run a simple example,downloadthe following files- afp_apw_xin_embeddings.bin
- sentences_short.txt.bz2
- seeds_positive.txtInstall BREDS using pippipinstallbreadsRun the following command:breds--word2vec=afp_apw_xin_embeddings.bin--sentences=sentences_short.txt--positive_seeds=seeds_positive.txt--similarity=0.6--confidence=0.6After the  process is terminated an output filerelationships.jsonlis generated containing the extracted  relationships.You can pretty print it's content to the terminal with:jq '.' < relationships.jsonl:{"entity_1":"Medtronic","entity_2":"Minneapolis","confidence":0.9982486865148862,"sentence":"<ORG>Medtronic</ORG> , based in <LOC>Minneapolis</LOC> , is the nation 's largest independent medical device maker . ","bef_words":"","bet_words":", based in","aft_words":", is","passive_voice":false}{"entity_1":"DynCorp","entity_2":"Reston","confidence":0.9982486865148862,"sentence":"Because <ORG>DynCorp</ORG> , headquartered in <LOC>Reston</LOC> , <LOC>Va.</LOC> , gets 98 percent of its revenue from government work .","bef_words":"Because","bet_words":", headquartered in","aft_words":", Va.","passive_voice":false}{"entity_1":"Handspring","entity_2":"Silicon Valley","confidence":0.893486865148862,"sentence":"There will be more firms like <ORG>Handspring</ORG> , a company based in <LOC>Silicon Valley</LOC> that looks as if it is about to become a force in handheld computers , despite its lack of machinery .","bef_words":"firms like","bet_words":", a company based in","aft_words":"that looks","passive_voice":false}BREDS has several parameters to tune the extraction process, in the example above it uses the default values, but these
can be set in the configuration file:parameters.cfgmax_tokens_away=6# maximum number of tokens between the two entitiesmin_tokens_away=1# minimum number of tokens between the two entitiescontext_window_size=2# number of tokens to the left and right of each entityalpha=0.2# weight of the BEF context in the similarity functionbeta=0.6# weight of the BET context in the similarity functiongamma=0.2# weight of the AFT context in the similarity functionwUpdt=0.5# < 0.5 trusts new examples less on each iterationnumber_iterations=4# number of bootstrap iterationswUnk=0.1# weight given to unknown extracted relationship instanceswNeg=2# weight given to extracted relationship instancesmin_pattern_support=2# minimum number of instances in a cluster to be considered a patternand passed with the argument--config=parameters.cfg.The full command line parameters are:-h,--helpshowthishelpmessageandexit--configCONFIGfilewithbootstrappingconfigurationparameters--word2vecWORD2VECanembeddingmodelbasedonword2vec,intheformatofa.binfile--sentencesSENTENCESatextfilewithasentenceperline,andwithatleasttwoentitiespersentence--positive_seedsPOSITIVE_SEEDSatextfilewithaseedperline,intheformat,e.g.:'Nokia;Espoo'--negative_seedsNEGATIVE_SEEDSatextfilewithaseedperline,intheformat,e.g.:'Microsoft;San Francisco'--similaritySIMILARITYtheminimumsimilaritybetweentuplesandpatternstobeconsideredamatch--confidenceCONFIDENCEtheminimumconfidencescoreforamatchtobeconsideredatruepositive--number_iterationsNUMBER_ITERATIONSthenumberofiterationstherunPlease, refer to the sectionReferences and Citationsfor details on the parameters.In the first step BREDS pre-processes the input filesentences.txtgenerating word vector representations ofrelationships (i.e.:processed_tuples.pkl).This is done so that then you can experiment with different seed examples without having to repeat the process of
generating word vectors representations. Just pass the argument--sentences=processed_tuples.pklinstead to skip
this generation step.References and CitationsSemi-Supervised Bootstrapping of Relationship Extractors with Distributional Semantics, EMNLP'15@inproceedings{batista-etal-2015-semi,title="Semi-Supervised Bootstrapping of Relationship Extractors with Distributional Semantics",author="Batista, David S.  and Martins, Bruno  and Silva, M{\'a}rio J.",booktitle="Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing",month=sep,year="2015",address="Lisbon, Portugal",publisher="Association for Computational Linguistics",url="https://aclanthology.org/D15-1056",doi="10.18653/v1/D15-1056",pages="499--504",}"Large-Scale Semantic Relationship Extraction for Information Discovery" - Chapter 5, David S Batista, Ph.D. Thesis@incollection{phd-dsbatista2016title={Large-ScaleSemanticRelationshipExtractionforInformationDiscovery},author = {Batista, David S.},school = {Instituto Superior Técnico, Universidade de Lisboa},year = {2016}}Presenting BREDS at PyData Berlin 2017Contributing to BREDSImprovements, adding new features and bug fixes are welcome. If you wish to participate in the development of BREDS,
please read the following guidelines.The contribution process at a glancePreparing the development environmentCode away!Continuous IntegrationSubmit your changes by opening a pull requestSmall fixes and additions can be submitted directly as pull requests, but larger changes should be discussed in
an issue first. You can expect a reply within a few days, but please be patient if it takes a bit longer.Preparing the development environmentMake sure you have Python3.9 installed on your systemmacOsbrewinstallpython@3.9
python3.9-mpipinstall--user--upgradepip
python3.9-mpipinstallvirtualenvClone the repository and prepare the development environment:gitclonegit@github.com:davidsbatista/BREDS.gitcdBREDSpython3.9-mvirtualenvvenv# create a new virtual environment for development using python3.9sourcevenv/bin/activate# activate the virtual environmentpipinstall-rrequirements_dev.txt# install the development requirementspipinstall-e.# install BREDS in edit modeContinuous IntegrationBREDS runs a continuous integration (CI) on all pull requests. This means that if you open a pull request (PR), a full
test suite is run on your PR:The code is formatted usingblackandisortUnused imports are auto-removed usingpyclnLinting is done usingpylingandflake8Type checking is done usingmypyTests are run usingpytestNevertheless, if you prefer to run the tests & formatting locally, it's possible too.makeallOpening a Pull RequestEvery PR should be accompanied by short description of the changes, including:Impact and  motivation for the changesAny open issues that are closed by this PRGive a ⭐️ if this project helped you!