Package: buzzepar
Berkeley Neural ParserA high-accuracy parser with models for 11 languages, implemented in Python. Based onConstituency Parsing with a Self-Attentive Encoderfrom ACL 2018, with additional changes described inMultilingual Constituency Parsing with Self-Attention and Pre-Training.ContentsInstallationUsageAvailable ModelsTrainingReproducing ExperimentsCitationCreditsIf you are primarily interested in training your own parsing models, skip to theTrainingsection of this README.InstallationTo install the parser, run the commands:$pipinstallcythonnumpy
$pipinstallbenepar[cpu]Cython and numpy should be installed separately prior to installing benepar. Note thatpip install benepar[cpu]has a dependency on thetensorflowpip package, which is a CPU-only version of tensorflow. Usepip install benepar[gpu]to instead introduce a dependency ontensorflow-gpu. Installing a GPU-enabled version of TensorFlow will likely require additional steps; see theofficial TensorFlow installation instructionsfor details.Benepar integrates with one of two NLP libraries for Python:NLTKorspaCy.If using NLTK, you should install the NLTK sentence and word tokenizers:>>>importnltk>>>nltk.download('punkt')If using spaCy, you should install a spaCy model for your language. For English, the installation command is:$python-mspacydownloadenParsing models need to be downloaded separately, using the commands:>>>importbenepar>>>benepar.download('benepar_en2')See theAvailable Modelssection below for a full list of models.UsageUsage with NLTK>>>importbenepar>>>parser=benepar.Parser("benepar_en2")>>>tree=parser.parse("Short cuts make long delays.")>>>print(tree)(S(NP(JJShort)(NNScuts))(VP(VBPmake)(NP(JJlong)(NNSdelays)))(..))Speed note: the first call toparsewill take much longer that subsequent calls, as caches are being warmed up.The parser can also parse pre-tokenized text. For some languages (including Chinese), this is required due to the lack of a built-in tokenizer.>>>parser.parse(['Short','cuts','make','long','delays','.'])Useparse_sentsto parse multiple sentences. It accepts an entire document as a string, or a list of sentences.>>>parser.parse_sents("The time for action is now. It's never too late to do something.")>>>parser.parse_sents(["The time for action is now.","It's never too late to do something."])>>>parser.parse_sents([['The','time','for','action','is','now','.'],['It',"'s",'never','too','late','to','do','something','.']])All parse trees returned are represented usingnltk.Treeobjects.Usage with spaCyBenepar also ships with a component that integrates with spaCy:>>>importspacy>>>frombenepar.spacy_pluginimportBeneparComponent>>>nlp=spacy.load('en')>>>nlp.add_pipe(BeneparComponent("benepar_en2"))>>>doc=nlp(u"The time for action is now. It's never too late to do something.")>>>sent=list(doc.sents)[0]>>>print(sent._.parse_string)(S(NP(NP(DTThe)(NNtime))(PP(INfor)(NP(NNaction))))(VP(VBZis)(ADVP(RBnow)))(..))>>>sent._.labels('S',)>>>list(sent._.children)[0]ThetimeforactionSince spaCy does not provide an official constituency parsing API, all methods are accessible through the extension namespacesSpan._andToken._.The following extension properties are available:Span._.labels: a tuple of labels for the given span. A span may have multiple labels when there are unary chains in the parse tree.Span._.parse_string: a string representation of the parse tree for a given span.Span._.constituents: an iterator overSpanobjects for sub-constituents in a pre-order traversal of the parse tree.Span._.parent: the parentSpanin the parse tree.Span._.children: an iterator over childSpans in the parse tree.Token._.labels,Token._.parse_string,Token._.parent: these behave the same as calling the corresponding method on the length-one Span containing the token.These methods will raise an exception when called on a span that is not a constituent in the parse tree. Such errors can be avoided by traversing the parse tree starting at either sentence level (by iterating overdoc.sents) or with an individualTokenobject.Available ModelsThe following trained parser models are available:ModelLanguageInfobenepar_en2English95.17 F1 on WSJ test set, 94 MB on disk.benepar_en2_largeEnglish95.52 F1 on WSJ test set, 274 MB on disk. This model is up to 3x slower thanbenepar_en2when running on CPU; we recommend running it on a GPU instead.benepar_zhChinese91.69 F1 on CTB 5.1 test set. Usage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Use a package such asjiebafor tokenization. Usage with spaCy first requires implementing Chinese support in spaCy. There is no official Chinese support in spaCy at the time of writing, but unofficial packages such asthis onemay work.benepar_arArabicUsage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Usage with spaCy first requires implementing Arabic support in spaCy. Accepts Unicode as input, but was trained on transliterated text (seesrc/transliterate.py); please let us know if there are any bugs.benepar_deGermanFull support for NLTK and spaCy; usepython -m spacy download deto download spaCy model for German.benepar_euBasqueUsage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Usage with spaCy first requires implementing Basque support in spaCy.benepar_frFrenchFull support for NLTK and spaCy; usepython -m spacy download frto download spaCy model for French.benepar_heHebrewUsage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Usage with spaCy first requires implementing Hebrew support in spaCy. Accepts Unicode as input, but was trained on transliterated text (seesrc/transliterate.py); please let us know if there are any bugs.benepar_huHungarianUsage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Usage with spaCy requires aHungarian model for spaCy.benepar_koKoreanUsage with NLTK requires tokenized sentences (untokenized raw text is not supported.) Usage with spaCy first requires implementing Korean support in spaCy.benepar_plPolishFull support for NLTK (including parsing from raw text.) Usage with spaCy first requires implementing Polish support in spaCy.benepar_svSwedishFull support for NLTK (including parsing from raw text.) Usage with spaCy first requires implementing Swedish support in spaCy.benepar_enEnglishNo part-of-speech tagging capabilities: we recommend usingbenepar_en2instead. When using this model, part-of-speech tags will be inherited from either NLTK (requiresnltk.download('averaged_perceptron_tagger')) or spaCy; however, we've found that our own tagger in models such asbenepar_en2gives better results. This model was released to accompany our ACL 2018 paper, and is retained for compatibility. 95.07 F1 on WSJ test set.benepar_en_smallEnglishNo part-of-speech tagging capabilities: we recommend usingbenepar_en2instead. This model was released to accompany our ACL 2018 paper, and is retained for compatibility. A smaller model that is 3-4x faster than thebenepar_enwhen running on CPU because it uses a smaller version of ELMo. 94.65 F1 on WSJ test set.benepar_en_ensembleEnglishNo part-of-speech tagging capabilities: we recommend usingbenepar_en2_largeinstead. This model was released to accompany our ACL 2018 paper, and is retained for compatibility. An ensemble of two parsers: one that uses the original ELMo embeddings and one that uses the 5.5B ELMo embeddings. A GPU is highly recommended for running the ensemble. 95.43 F1 on WSJ test set.TrainingThe code used to train our parsing models is currently different from the code used to parse sentences in the release version described above, though both are stored in this repository. The training code uses PyTorch and can be obtained by cloning this repository from GitHub. The release version uses TensorFlow instead, because it allows serializing the parsing model into a single file on disk in a way that minimizes software dependencies and reduces file size on disk.Software Requirements for TrainingPython 3.6 or higher.Cython 0.25.2 or any compatible version.PyTorch0.4.1, 1.0/1.1, or any compatible version.EVALB. Before starting, runmakeinside theEVALB/directory to compile anevalbexecutable. This will be called from Python for evaluation. If training on the SPMRL datasets, you will need to runmakeinside theEVALB_SPMRL/directory instead.AllenNLP0.7.0 or any compatible version (only required when using ELMo word representations)pytorch-pretrained-bert0.4.0 or any compatible version (only required when using BERT word representations)Pre-trained Models (PyTorch)The following pre-trained parser models are available for download:en_charlstm_dev.93.61.pt: Our best English single-system parser that does not rely on external word representationsen_elmo_dev.95.21.pt: The best English single-system parser from our ACL 2018 paper. Using this parser requires ELMo weights, which must be downloaded separately.To use ELMo embeddings, download the following files into thedata/folder (preserving their names):elmo_2x4096_512_2048cnn_2xhighway_options.jsonelmo_2x4096_512_2048cnn_2xhighway_weights.hdf5There is currently no command-line option for configuring the locations/names of the ELMo files.Pre-trained BERT weights will be automatically downloaded as needed by thepytorch-pretrained-bertpackage.Training InstructionsA new model can be trained using the commandpython src/main.py train .... Some of the available arguments are:ArgumentDescriptionDefault--model-path-basePath base to use for saving modelsN/A--evalb-dirPath to EVALB directoryEVALB/--train-pathPath to training treesdata/02-21.10way.clean--dev-pathPath to development treesdata/22.auto.clean--batch-sizeNumber of examples per training update250--checks-per-epochNumber of development evaluations per epoch4--subbatch-max-tokensMaximum number of words to process in parallel while training (a full batch may not fit in GPU memory)2000--eval-batch-sizeNumber of examples to process in parallel when evaluating on the development set100--numpy-seedNumPy random seedRandom--use-wordsUse learned word embeddingsDo not use word embeddings--use-tagsUse predicted part-of-speech tags as inputDo not use predicted tags--use-chars-lstmUse learned CharLSTM word representationsDo not use CharLSTM--use-elmoUse pre-trained ELMo word representationsDo not use ELMo--use-bertUse pre-trained BERT word representationsDo not use BERT--bert-modelPre-trained BERT model to use if--use-bertis passedbert-base-uncased--no-bert-do-lower-caseInstructs the BERT tokenizer to retain case information (setting should match the BERT model in use)Perform lowercasing--predict-tagsAdds a part-of-speech tagging component and auxiliary loss to the parserDo not predict tagsAdditional arguments are available for other hyperparameters; seemake_hparams()insrc/main.py. These can be specified on the command line, such as--num-layers 2(for numerical parameters),--use-tags(for boolean parameters that default to False), or--no-partitioned(for boolean parameters that default to True).If--use-tagsis passed, the training and development trees are assumed to have predicted part-of-speech tags. If--predict-tagsis passed, the data is assumed to have ground-truth tags instead. As a result, these two options can't be used simultaneously. Note that the files we provide in thedata/folder have predicted tags, and that data with gold tags must be obtained separately.For each development evaluation, the F-score on the development set is computed and compared to the previous best. If the current model is better, the previous model will be deleted and the current model will be saved. The new filename will be derived from the provided model path base and the development F-score.As an example, to train an English parser using the default hyperparameters, you can use the command:python src/main.py train --use-words --use-chars-lstm --model-path-base models/en_charlstm --d-char-emb 64To train an English parser that uses ELMo embeddings, the command is:python src/main.py train --use-elmo --model-path-base models/en_elmo --num-layers 4To train an English parser that uses BERT, the command is:python src/main.py train --use-bert --model-path-base models/en_bert --bert-model "bert-large-uncased" --num-layers 2 --learning-rate 0.00005 --batch-size 32 --eval-batch-size 16 --subbatch-max-tokens 500Evaluation InstructionsA saved model can be evaluated on a test corpus using the commandpython src/main.py test ...with the following arguments:ArgumentDescriptionDefault--model-path-basePath base of saved modelN/A--evalb-dirPath to EVALB directoryEVALB/--test-pathPath to test treesdata/23.auto.clean--test-path-rawAlternative path to test trees that is used for evalb only (used to double-check that evaluation against pre-processed trees does not contain any bugs)Compare to trees from--test-path--eval-batch-sizeNumber of examples to process in parallel when evaluating on the test set100If the parser was trained to have predicted part-of-speech tags as input (via the--use-tagsflag) the test trees are assumed to have predicted part-of-speech tags. Otherwise, the tags in the test trees are not used as input to the parser.As an example, after extracting the pre-trained model, you can evaluate it on the test set using the following command:python src/main.py test --model-path-base models/nk_base6_lstm_dev.93.61.ptThe pre-trained model with CharLSTM embeddings obtains F-scores of 93.61 on the development set and 93.55 on the test set. The pre-trained model with ELMo embeddings obtains F-scores of 95.21 on the development set and 95.13 on the test set.Using the Trained ModelsSee therun_parsefunction insrc/main.pyfor an example of how a parser can be loaded from disk and used to parse sentences using the PyTorch codebase.Theexport/export.pyfile contains the code we used to convert our ELMo-based parser to a TensorFlow graph (for use in the release version of the parser). For our BERT-based parsers, consultexport/export_bert.pyinstead. This exporting code hard-codes certain hyperparameter choices, so you will likely need to tweak it to export your own models. Exporting the model to TensorFlow allows it to be stored in a single file, including all ELMo/BERT weights. We also use TensorFlow's graph transforms to shrink the model size on disk with only a tiny impact on parsing accuracy: the compressed ELMo model obtains an F1-score of 95.07 on the test set, compared to 95.13 for the uncompressed model.Reproducing ExperimentsThe code used for our ACL 2018 paper is taggedacl2018in git. TheEXPERIMENTS.mdfile in that version of the code contains additional notes about the command-line arguments we used to perform the experiments reported in our ACL 2018 paper.The version of the code currently in this repository has added new features (such as BERT support and part-of-speech tag prediction), eliminated some of the less-performant parser variations (e.g. the CharConcat word representation), and has updated to a newer version of PyTorch. TheEXPERIMENTS.mdfile now describes the commands used to train our best-performing single-system parser for each language that we evaluate on.CitationIf you use this software for research, please cite our paper as follows:@InProceedings{Kitaev-2018-SelfAttentive,
  author    = {Kitaev, Nikita and Klein, Dan},
  title     = {Constituency Parsing with a Self-Attentive Encoder},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  month     = {July},
  year      = {2018},
  address   = {Melbourne, Australia},
  publisher = {Association for Computational Linguistics},
}CreditsThe code in this repository and portions of this README are based onhttps://github.com/mitchellstern/minimal-span-parser