Package: vnerrant
VNERRANT v1.0.0OverviewThe main aim of VNERRANT is to automatically annotate parallel English sentences with error type information. Specifically, given an original and corrected sentence pair, VNERRANT will extract the edits that transform the former to the latter and classify them according to a rule-based error type framework. This can be used to standardise parallel datasets or facilitate detailed error type evaluation. Annotated output files are in M2 format and an evaluation script is provided.ExampleOriginal: This are gramamtical sentence .Corrected: This is a grammatical sentence .Output M2:S This are gramamtical sentence .
A 1 2|||R:VERB:SVA|||is|||REQUIRED|||-NONE-|||0
A 2 2|||M:DET|||a|||REQUIRED|||-NONE-|||0
A 2 3|||R:SPELL|||grammatical|||REQUIRED|||-NONE-|||0
A -1 -1|||noop|||-NONE-|||REQUIRED|||-NONE-|||1InstallationPip Installcondacreate-nvnerrantpython=3.9
condaactivatevnerrantYou have two options for installing VNERRANT:Option 1: Install VNERRANT using pip with the following commands:pipinstall-Upipsetuptoolswheel
pipinstallvnerrantOption 2: Alternatively, if you want to install ERRANT from the source, you can follow these steps:gitclonehttps://gitlab.testsprep.online/nlp/research/errantcdvnerrant
pipinstall-Upipsetuptoolswheel
pipinstall-e.Please obtain a Spacy model by using the following command:python-mspacydownloaden_core_web_smYou can verify the available models atthislocation.UsageCLITwo main commands are provided with VNERRANT:convertandevaluate. You can run them from anywhere on the command line without having to invoke a specific python script.1.vnerrant convert parallel-to-m2This is the main annotation command that takes an original text file and at least one parallel corrected text file as input, and outputs an annotated M2 file. By default, it is assumed that the original and corrected text files are word tokenised with one sentence per line.
Example:vnerrant convert parallel-to-m2 -o <orig_file> -c <cor_file1> [<cor_file2> ...] -out <out_m2>2.vnerrant convert m2-to-m2This is a variant ofparallel-to-m2that operates on an M2 file instead of parallel text files. This makes it easier to reprocess existing M2 files. You must also specify whether you want to use gold or auto edits; i.e.-goldwill only classify the existing edits, while-autowill extract and classify automatic edits. In both settings, uncorrected edits and noops are preserved.
Example:vnerrant convert m2-to-m2  -i <in_m2> -o <out_m2> {-auto|-gold}3.vnerrant evaluate m2This is the evaluation command that compares a hypothesis M2 file against a reference M2 file. The default behaviour evaluates the hypothesis overall in terms of span-based correction. The-cat {1,2,3}flag can be used to evaluate error types at increasing levels of granularity, while the-dsor-dtflag can be used to evaluate in terms of span-based or token-based detection (i.e. ignoring the correction). All scores are presented in terms of Precision, Recall and F-score (default: F0.5), and counts for True Positives (TP), False Positives (FP) and False Negatives (FN) are also shown.
Examples:vnerrant evaluate m2 -hyp <hyp_m2> -ref <ref_m2>
vnerrant evaluate m2 -hyp <hyp_m2> -ref <ref_m2> -cat {1,2,3}
vnerrant evaluate m2 -hyp <hyp_m2> -ref <ref_m2> -ds
vnerrant evaluate m2 -hyp <hyp_m2> -ref <ref_m2> -ds -cat {1,2,3}All these scripts also have additional advanced command line options which can be displayed using the-hflag.APIAs of v3.0.0, ERRANT now also comes with an API.Quick Startimportvnerrantannotator=vnerrant.load('en')orig='My    name    is   the     John'cor='My name is John'edits=annotator.annotate_with_pre_and_post_processing(orig,cor)foreinedits:print(e.o_start,e.o_end,e.o_str,e.c_start,e.c_end,e.c_str,e.type)print(e.o_toks.start_char,e.o_toks.end_char)# assert e.o_str == orig[e.o_toks.start_char:e.o_toks.end_char]Loadingvnerrant.load(lang, model_name)Instantiate an ERRANT Annotator object. Presently, the lang parameter exclusively accepts 'en' for English, though we aspire to broaden its language support in future iterations. The model_name corresponds to the name of the SpaCy model being utilized. Optionally, you can provide the nlp parameter if you've previously loaded SpaCy and wish to prevent ERRANT from loading it redundantly.Annotator ObjectsAn Annotator object is the main interface for ERRANT.Methodsannotator.parseannotator.parse(string, tokenise=False)Lemmatise, POS tag, and parse a text string with spacy. Settokeniseto True to also word tokenise with spacy. Returns a spacy Doc object.annotator.alignannotator.align(orig, cor, lev=False)Align spacy-parsed original and corrected text. The default uses a linguistically-enhanced Damerau-Levenshtein alignment, but thelevflag can be used for a standard Levenshtein alignment. Returns an Alignment object.annotator.mergeannotator.merge(alignment, merging='rules')Extract edits from the optimum alignment in an Alignment object. Four different merging strategies are available:rules: Use a rule-based merging strategy (default)all-split: Merge nothing: MSSDI -> M, S, S, D, Iall-merge: Merge adjacent non-matches: MSSDI -> M, SSDIall-equal: Merge adjacent same-type non-matches: MSSDI -> M, SS, D, IReturns a list of Edit objects.annotator.classifyannotator.classify(edit)Classify an edit. Sets theedit.typeattribute in an Edit object and returns the same Edit object.annotator.annotateannotator.annotate(orig, cor, lev=False, merging='rules')Run the full annotation pipeline to align two sequences and extract and classify the edits. Equivalent to runningannotator.align,annotator.mergeandannotator.classifyin sequence. Returns a list of Edit objects.importerrantannotator=errant.load(lang="en",model_name="en_core_web_sm")orig=annotator.parse('This are gramamtical sentence .')cor=annotator.parse('This is a grammatical sentence .')alignment=annotator.align(orig,cor)edits=annotator.merge(alignment)foreinedits:e=annotator.classify(e)annotator.import_editannotator.import_edit(orig, cor, edit, min=True, old_cat=False)Load an Edit object from a list.origandcormust be spacy-parsed Doc objects and the edit must be of the form:[o_start, o_end, c_start, c_end(, type)]. The values must be integers that correspond to the token start and end offsets in the original and corrected Doc objects. Thetypevalue is an optional string that denotes the error type of the edit (if known). Setminto True to minimise the edit (e.g. [a b -> a c] = [b -> c]) andold_catto True to preserve the old error type category (i.e. turn off the classifier).importvnerrantannotator=vnerrant.load('en')orig=annotator.parse('This are gramamtical sentence .')cor=annotator.parse('This is a grammatical sentence .')edit=[1,2,1,2,'SVA']# are -> isedit=annotator.import_edit(orig,cor,edit)print(edit.to_m2())Alignment ObjectsAn Alignment object is created from two spacy-parsed text sequences.Attributesalignment.origalignment.corThe spacy-parsed original and corrected text sequences.alignment.cost_matrixalignment.op_matrixThe cost matrix and operation matrix produced by the alignment.alignment.align_seqThe first cheapest alignment between the two sequences.Edit ObjectsAn Edit object represents a transformation between two text sequences.Attributesedit.o_startedit.o_endedit.o_toksedit.o_strThe start and end offsets, the spacy tokens, and the string for the edit in theoriginaltext.edit.c_startedit.c_endedit.c_toksedit.c_strThe start and end offsets, the spacy tokens, and the string for the edit in thecorrectedtext.edit.typeThe error type string.Methodedit.to_m2(id=0)
Format the edit for an output M2 file.idis the annotator id.