Package: sense2vec
sense2vec: Contextually-keyed word vectorssense2vec (Trask et. al, 2015) is a nice
twist onword2vecthat lets you learn
more interesting and detailed word vectors. This library is a simple Python
implementation for loading, querying and training sense2vec models. For more
details, check outour blog post. To explore the
semantic similarities across all Reddit comments of 2015 and 2019, see theinteractive demo.🦆Version 2.0 (for spaCy v3) out now!Read the release notes here.✨ FeaturesQueryvectors for multi-word phrasesbased on part-of-speech tags and
entity labels.spaCypipeline componentandextension attributes.Fullyserializableso you can easily ship your sense2vec vectors with your
spaCy model packages.Optionalcaching of nearest neighborsfor super fast "most similar"
queries.Train your own vectorsusing a pretrained spaCy model, raw text andGloVeor Word2Vec viafastText(details).Prodigyannotation recipesfor evaluating models,
creating lists of similar multi-word phrases and converting them to match
patterns, e.g. for rule-based NER or to bootstrap NER annotation
(details & examples).🚀 QuickstartStandalone usagefromsense2vecimportSense2Vecs2v=Sense2Vec().from_disk("/path/to/s2v_reddit_2015_md")query="natural_language_processing|NOUN"assertqueryins2vvector=s2v[query]freq=s2v.get_freq(query)most_similar=s2v.most_similar(query,n=3)# [('machine_learning|NOUN', 0.8986967),#  ('computer_vision|NOUN', 0.8636297),#  ('deep_learning|NOUN', 0.8573361)]Usage as a spaCy pipeline component⚠️ Note that this example describes usage withspaCy v3. For usage with spaCy v2, downloadsense2vec==1.0.3and check out thev1.xbranch of this repo.importspacynlp=spacy.load("en_core_web_sm")s2v=nlp.add_pipe("sense2vec")s2v.from_disk("/path/to/s2v_reddit_2015_md")doc=nlp("A sentence about natural language processing.")assertdoc[3:6].text=="natural language processing"freq=doc[3:6]._.s2v_freqvector=doc[3:6]._.s2v_vecmost_similar=doc[3:6]._.s2v_most_similar(3)# [(('machine learning', 'NOUN'), 0.8986967),#  (('computer vision', 'NOUN'), 0.8636297),#  (('deep learning', 'NOUN'), 0.8573361)]Interactive demosTo try out our pretrained vectors trained on Reddit comments, check out theinteractive sense2vec demo.This repo also includes aStreamlitdemo script for
exploring vectors and the most similar phrases. After installingstreamlit,
you can run the script withstreamlit runandone or more paths to
pretrained vectorsaspositional argumentson the command line. For
example:pipinstallstreamlit
streamlitrunhttps://raw.githubusercontent.com/explosion/sense2vec/master/scripts/streamlit_sense2vec.py/path/to/vectorsPretrained vectorsTo use the vectors, download the archive(s) and pass the extracted directory toSense2Vec.from_diskorSense2VecComponent.from_disk. The vector files areattached to the GitHub release. Large files have been split into multi-part
downloads.VectorsSizeDescription📥 Download (zipped)s2v_reddit_2019_lg4 GBReddit comments 2019 (01-07)part 1,part 2,part 3s2v_reddit_2015_md573 MBReddit comments 2015part 1To merge the multi-part archives, you can run the following:cats2v_reddit_2019_lg.tar.gz.*>s2v_reddit_2019_lg.tar.gz⏳ Installation & Setupsense2vec releases are available on pip:pipinstallsense2vecTo use pretrained vectors, downloadone of the vector packages, unpack the.tar.gzarchive
and pointfrom_diskto the extracted data directory:fromsense2vecimportSense2Vecs2v=Sense2Vec().from_disk("/path/to/s2v_reddit_2015_md")👩‍💻 UsageUsage with spaCy v3The easiest way to use the library and vectors is to plug it into your spaCy
pipeline. Thesense2vecpackage exposes aSense2VecComponent, which can be
initialised with the shared vocab and added to your spaCy pipeline as acustom pipeline component.
By default, components are added to theend of the pipeline, which is the
recommended position for this component, since it needs access to the dependency
parse and, if available, named entities.importspacyfromsense2vecimportSense2VecComponentnlp=spacy.load("en_core_web_sm")s2v=nlp.add_pipe("sense2vec")s2v.from_disk("/path/to/s2v_reddit_2015_md")The component will add severalextension attributes and methodsto spaCy'sTokenandSpanobjects that let you retrieve vectors and
frequencies, as well as most similar terms.doc=nlp("A sentence about natural language processing.")assertdoc[3:6].text=="natural language processing"freq=doc[3:6]._.s2v_freqvector=doc[3:6]._.s2v_vecmost_similar=doc[3:6]._.s2v_most_similar(3)For entities, the entity labels are used as the "sense" (instead of the token's
part-of-speech tag):doc=nlp("A sentence about Facebook and Google.")forentindoc.ents:assertent._.in_s2vmost_similar=ent._.s2v_most_similar(3)Available attributesThe following extension attributes are exposed on theDocobject via the._property:NameAttribute TypeTypeDescriptions2v_phrasespropertylistAll sense2vec-compatible phrases in the givenDoc(noun phrases, named entities).The following attributes are available via the._property ofTokenandSpanobjects – for exampletoken._.in_s2v:NameAttribute TypeReturn TypeDescriptionin_s2vpropertyboolWhether a key exists in the vector map.s2v_keypropertyunicodeThe sense2vec key of the given object, e.g. `"ducks2v_vecpropertyndarray[float32]The vector of the given key.s2v_freqpropertyintThe frequency of the given key.s2v_other_sensespropertylistAvailable other senses, e.g. `"ducks2v_most_similarmethodlistGet thenmost similar terms. Returns a list of((word, sense), score)tuples.s2v_similaritymethodfloatGet the similarity to anotherTokenorSpan.⚠️A note on span attributes:Under the hood, entities indoc.entsareSpanobjects. This is why the pipeline component also adds attributes and
methods to spans and not just tokens. However, it's not recommended to use the
sense2vec attributes on arbitrary slices of the document, since the model
likely won't have a key for the respective text.Spanobjects also don't
have a part-of-speech tag, so if no entity label is present, the "sense"
defaults to the root's part-of-speech tag.Adding sense2vec to a trained pipelineIf you're training and packaging a spaCy pipeline and want to include a sense2vec component in it, you can load in the data via the[initialize]blockof the training config:[initialize.components][initialize.components.sense2vec]data_path="/path/to/s2v_reddit_2015_md"Standalone usageYou can also use the underlyingSense2Vecclass directly and load in the
vectors using thefrom_diskmethod. See below for the available API methods.fromsense2vecimportSense2Vecs2v=Sense2Vec().from_disk("/path/to/reddit_vectors-1.1.0")most_similar=s2v.most_similar("natural_language_processing|NOUN",n=10)⚠️Important note:To look up entries in the vectors table, the keys need
to follow the scheme ofphrase_text|SENSE(note the_instead of spaces
and the|before the tag or label) – for example,machine_learning|NOUN.
Also note that the underlying vector table is case-sensitive.🎛 APIclassSense2VecThe standaloneSense2Vecobject that holds the vectors, strings and
frequencies.methodSense2Vec.__init__Initialize theSense2Vecobject.ArgumentTypeDescriptionshapetupleThe vector shape. Defaults to(1000, 128).stringsspacy.strings.StringStoreOptional string store. Will be created if it doesn't exist.senseslistOptional list of all available senses. Used in methods that generate the best sense or other senses.vectors_nameunicodeOptional name to assign to theVectorstable, to prevent clashes. Defaults to"sense2vec".overridesdictOptional custom functions to use, mapped to names registered via the registry, e.g.{"make_key": "custom_make_key"}.RETURNSSense2VecThe newly constructed object.s2v=Sense2Vec(shape=(300,128),senses=["VERB","NOUN"])methodSense2Vec.__len__The number of rows in the vectors table.ArgumentTypeDescriptionRETURNSintThe number of rows in the vectors table.s2v=Sense2Vec(shape=(300,128))assertlen(s2v)==300methodSense2Vec.__contains__Check if a key is in the vectors table.ArgumentTypeDescriptionkeyunicode / intThe key to look up.RETURNSboolWhether the key is in the table.s2v=Sense2Vec(shape=(10,4))s2v.add("avocado|NOUN",numpy.asarray([4,2,2,2],dtype=numpy.float32))assert"avocado|NOUN"ins2vassert"avocado|VERB"notins2vmethodSense2Vec.__getitem__Retrieve a vector for a given key. Returns None if the key is not in the table.ArgumentTypeDescriptionkeyunicode / intThe key to look up.RETURNSnumpy.ndarrayThe vector orNone.vec=s2v["avocado|NOUN"]methodSense2Vec.__setitem__Set a vector for a given key. Will raise an error if the key doesn't exist. To
add a new entry, useSense2Vec.add.ArgumentTypeDescriptionkeyunicode / intThe key.vectornumpy.ndarrayThe vector to set.vec=s2v["avocado|NOUN"]s2v["avacado|NOUN"]=vecmethodSense2Vec.addAdd a new vector to the table.ArgumentTypeDescriptionkeyunicode / intThe key to add.vectornumpy.ndarrayThe vector to add.freqintOptional frequency count. Used to find best matching senses.vec=s2v["avocado|NOUN"]s2v.add("🥑|NOUN",vec,1234)methodSense2Vec.get_freqGet the frequency count for a given key.ArgumentTypeDescriptionkeyunicode / intThe key to look up.default-Default value to return if no frequency is found.RETURNSintThe frequency count.vec=s2v["avocado|NOUN"]s2v.add("🥑|NOUN",vec,1234)asserts2v.get_freq("🥑|NOUN")==1234methodSense2Vec.set_freqSet a frequency count for a given key.ArgumentTypeDescriptionkeyunicode / intThe key to set the count for.freqintThe frequency count.s2v.set_freq("avocado|NOUN",104294)methodSense2Vec.__iter__,Sense2Vec.itemsIterate over the entries in the vectors table.ArgumentTypeDescriptionYIELDStupleString key and vector pairs in the table.forkey,vecins2v:print(key,vec)forkey,vecins2v.items():print(key,vec)methodSense2Vec.keysIterate over the keys in the table.ArgumentTypeDescriptionYIELDSunicodeThe string keys in the table.all_keys=list(s2v.keys())methodSense2Vec.valuesIterate over the vectors in the table.ArgumentTypeDescriptionYIELDSnumpy.ndarrayThe vectors in the table.all_vecs=list(s2v.values())propertySense2Vec.sensesThe available senses in the table, e.g."NOUN"or"VERB"(added at
initialization).ArgumentTypeDescriptionRETURNSlistThe available senses.s2v=Sense2Vec(senses=["VERB","NOUN"])assert"VERB"ins2v.sensespropertySense2vec.frequenciesThe frequencies of the keys in the table, in descending order.ArgumentTypeDescriptionRETURNSlistThe(key, freq)tuples by frequency, descending.most_frequent=s2v.frequencies[:10]key,score=s2v.frequencies[0]methodSense2vec.similarityMake a semantic similarity estimate of two keys or two sets of keys. The default
estimate is cosine similarity using an average of vectors.ArgumentTypeDescriptionkeys_aunicode / int / iterableThe string or integer key(s).keys_bunicode / int / iterableThe other string or integer key(s).RETURNSfloatThe similarity score.keys_a=["machine_learning|NOUN","natural_language_processing|NOUN"]keys_b=["computer_vision|NOUN","object_detection|NOUN"]print(s2v.similarity(keys_a,keys_b))asserts2v.similarity("machine_learning|NOUN","machine_learning|NOUN")==1.0methodSense2Vec.most_similarGet the most similar entries in the table. If more than one key is provided, the
average of the vectors is used. To make this method faster, see thescript for precomputing a cacheof the nearest
neighbors.ArgumentTypeDescriptionkeysunicode / int / iterableThe string or integer key(s) to compare to.nintThe number of similar keys to return. Defaults to10.batch_sizeintThe batch size to use. Defaults to16.RETURNSlistThe(key, score)tuples of the most similar vectors.most_similar=s2v.most_similar("natural_language_processing|NOUN",n=3)# [('machine_learning|NOUN', 0.8986967),#  ('computer_vision|NOUN', 0.8636297),#  ('deep_learning|NOUN', 0.8573361)]methodSense2Vec.get_other_sensesFind other entries for the same word with a different sense, e.g."duck|VERB"for"duck|NOUN".ArgumentTypeDescriptionkeyunicode / intThe key to check.ignore_caseboolCheck for uppercase, lowercase and titlecase. Defaults toTrue.RETURNSlistThe string keys of other entries with different senses.other_senses=s2v.get_other_senses("duck|NOUN")# ['duck|VERB', 'Duck|ORG', 'Duck|VERB', 'Duck|PERSON', 'Duck|ADJ']methodSense2Vec.get_best_senseFind the best-matching sense for a given word based on the available senses and
frequency counts. ReturnsNoneif no match is found.ArgumentTypeDescriptionwordunicodeThe word to check.senseslistOptional list of senses to limit the search to. If not set / empty, all senses in the vectors are used.ignore_caseboolCheck for uppercase, lowercase and titlecase. Defaults toTrue.RETURNSunicodeThe best-matching key or None.asserts2v.get_best_sense("duck")=="duck|NOUN"asserts2v.get_best_sense("duck",["VERB","ADJ"])=="duck|VERB"methodSense2Vec.to_bytesSerialize aSense2Vecobject to a bytestring.ArgumentTypeDescriptionexcludelistNames of serialization fields to exclude.RETURNSbytesThe serializedSense2Vecobject.s2v_bytes=s2v.to_bytes()methodSense2Vec.from_bytesLoad aSense2Vecobject from a bytestring.ArgumentTypeDescriptionbytes_databytesThe data to load.excludelistNames of serialization fields to exclude.RETURNSSense2VecThe loaded object.s2v_bytes=s2v.to_bytes()new_s2v=Sense2Vec().from_bytes(s2v_bytes)methodSense2Vec.to_diskSerialize aSense2Vecobject to a directory.ArgumentTypeDescriptionpathunicode /PathThe path.excludelistNames of serialization fields to exclude.s2v.to_disk("/path/to/sense2vec")methodSense2Vec.from_diskLoad aSense2Vecobject from a directory.ArgumentTypeDescriptionpathunicode /PathThe path to load fromexcludelistNames of serialization fields to exclude.RETURNSSense2VecThe loaded object.s2v.to_disk("/path/to/sense2vec")new_s2v=Sense2Vec().from_disk("/path/to/sense2vec")classSense2VecComponentThe pipeline component to add sense2vec to spaCy pipelines.methodSense2VecComponent.__init__Initialize the pipeline component.ArgumentTypeDescriptionvocabVocabThe sharedVocab. Mostly used for the sharedStringStore.shapetupleThe vector shape.merge_phrasesboolWhether to merge sense2vec phrases into one token. Defaults toFalse.lemmatizeboolAlways look up lemmas if available in the vectors, otherwise default to original word. Defaults toFalse.overridesOptional custom functions to use, mapped to names registred via the registry, e.g.{"make_key": "custom_make_key"}.RETURNSSense2VecComponentThe newly constructed object.s2v=Sense2VecComponent(nlp.vocab)classmethodSense2VecComponent.from_nlpInitialize the component from an nlp object. Mostly used as the component
factory for the entry point (see setup.cfg) and to auto-register via the@spacy.componentdecorator.ArgumentTypeDescriptionnlpLanguageThenlpobject.**cfg-Optional config parameters.RETURNSSense2VecComponentThe newly constructed object.s2v=Sense2VecComponent.from_nlp(nlp)methodSense2VecComponent.__call__Process aDocobject with the component. Typically only called as part of the
spaCy pipeline and not directly.ArgumentTypeDescriptiondocDocThe document to process.RETURNSDocthe processed document.methodSense2Vec.init_componentRegister the component-specific extension attributes here and only if the
component is added to the pipeline and used – otherwise, tokens will still get
the attributes even if the component is only created and not added.methodSense2VecComponent.to_bytesSerialize the component to a bytestring. Also called when the component is added
to the pipeline and you runnlp.to_bytes.ArgumentTypeDescriptionRETURNSbytesThe serialized component.methodSense2VecComponent.from_bytesLoad a component from a bytestring. Also called when you runnlp.from_bytes.ArgumentTypeDescriptionbytes_databytesThe data to load.RETURNSSense2VecComponentThe loaded object.methodSense2VecComponent.to_diskSerialize the component to a directory. Also called when the component is added
to the pipeline and you runnlp.to_disk.ArgumentTypeDescriptionpathunicode /PathThe path.methodSense2VecComponent.from_diskLoad aSense2Vecobject from a directory. Also called when you runnlp.from_disk.ArgumentTypeDescriptionpathunicode /PathThe path to load fromRETURNSSense2VecComponentThe loaded object.classregistryFunction registry (powered bycatalogue) to easily customize the
functions used to generate keys and phrases. Allows you to decorate and name
custom functions, swap them out and serialize the custom names when you save out
the model. The following registry options are available:NameDescriptionregistry.make_keyGiven awordandsense, return a string of the key, e.g. `"wordregistry.split_keyGiven a string key, return a(word, sense)tuple.registry.make_spacy_keyGiven a spaCy object (TokenorSpan) and a booleanprefer_entskeyword argument (whether to prefer the entity label for single tokens), return a(word, sense)tuple. Used in extension attributes to generate a key for tokens and spans.registry.get_phrasesGiven a spaCyDoc, return a list ofSpanobjects used for sense2vec phrases (typically noun phrases and named entities).registry.merge_phrasesGiven a spaCyDoc, get all sense2vec phrases and merge them into single tokens.Each registry has aregistermethod that can be used as a function decorator
and takes one argument, the name of the custom function.fromsense2vecimportregistry@registry.make_key.register("custom")defcustom_make_key(word,sense):returnf"{word}###{sense}"@registry.split_key.register("custom")defcustom_split_key(key):word,sense=key.split("###")returnword,senseWhen initializing theSense2Vecobject, you can now pass in a dictionary of
overrides with the names of your custom registered functions.overrides={"make_key":"custom","split_key":"custom"}s2v=Sense2Vec(overrides=overrides)This makes it easy to experiment with different strategies and serializing the
strategies as plain strings (instead of having to pass around and/or pickle the
functions themselves).🚂 Training your own sense2vec vectorsThe/scriptsdirectory contains command line utilities for
preprocessing text and training your own vectors.RequirementsTo train your own sense2vec vectors, you'll need the following:Avery largesource of raw text (ideally more than you'd use for word2vec,
since the senses make the vocabulary more sparse). We recommend at least 1
billion words.Apretrained spaCy modelthat assigns
part-of-speech tags, dependencies and named entities, and populates thedoc.noun_chunks. If the language you need doesn't provide a built insyntax iterator for noun phrases,
you'll need to write your own. (Thedoc.noun_chunksanddoc.entsare what
sense2vec uses to determine what's a phrase.)GloVeorfastTextinstalled and built.
You should be able to clone the repo and runmakein the respective
directory.Step-by-step processThe training process is split up into several steps to allow you to resume at
any given point. Processing scripts are designed to operate on single files,
making it easy to parallellize the work. The scripts in this repo require eitherGloveorfastTextwhich you need to
clone andmake.For Fasttext, the scripts will require the path to the created binary file.
If you're working on Windows, you can build withcmake, or alternatively
use the.exefile from thisunofficialrepo with FastText binary builds for Windows:https://github.com/xiamx/fastText/releases.ScriptDescription1.01_parse.pyUse spaCy to parse the raw text and output binary collections ofDocobjects (seeDocBin).2.02_preprocess.pyLoad a collection of parsedDocobjects produced in the previous step and output text files in the sense2vec format (one sentence per line and merged phrases with senses).3.03_glove_build_counts.pyUseGloVeto build the vocabulary and counts. Skip this step if you're using Word2Vec viaFastText.4.04_glove_train_vectors.py04_fasttext_train_vectors.pyUseGloVeorFastTextto train vectors.5.05_export.pyLoad the vectors and frequencies and output a sense2vec component that can be loaded viaSense2Vec.from_disk.6.06_precompute_cache.pyOptional:Precompute nearest-neighbor queries for every entry in the vocab to makeSense2Vec.most_similarfaster.For more detailed documentation of the scripts, check out the source or run them
with--help. For example,python scripts/01_parse.py --help.🍳 Prodigy recipesThis package also seamlessly integrates with theProdigyannotation tool and exposes recipes for using sense2vec vectors to quickly
generate lists of multi-word phrases and bootstrap NER annotations. To use a
recipe,sense2vecneeds to be installed in the same environment as Prodigy.
For an example of a real-world use case, check out thisNER projectwith downloadable datasets.The following recipes are available – see below for more detailed docs.RecipeDescriptionsense2vec.teachBootstrap a terminology list using sense2vec.sense2vec.to-patternsConvert phrases dataset to token-based match patterns.sense2vec.evalEvaluate a sense2vec model by asking about phrase triples.sense2vec.eval-most-similarEvaluate a sense2vec model by correcting the most similar entries.sense2vec.eval-abPerform an A/B evaluation of two pretrained sense2vec vector models.recipesense2vec.teachBootstrap a terminology list using sense2vec. Prodigy will suggest similar terms
based on the most similar phrases from sense2vec, and the suggestions will
be adjusted as you annotate and accept similar phrases. For each seed term, the
best matching sense according to the sense2vec vectors will be used.prodigysense2vec.teach[dataset][vectors_path][--seeds][--threshold][--n-similar][--batch-size][--resume]ArgumentTypeDescriptiondatasetpositionalDataset to save annotations to.vectors_pathpositionalPath to pretrained sense2vec vectors.--seeds,-soptionOne or more comma-separated seed phrases.--threshold,-toptionSimilarity threshold. Defaults to0.85.--n-similar,-noptionNumber of similar items to get at once.--batch-size,-boptionBatch size for submitting annotations.--resume,-RflagResume from an existing phrases dataset.Exampleprodigysense2vec.teachtech_phrases/path/to/s2v_reddit_2015_md
--seeds"natural language processing, machine learning, artificial intelligence"recipesense2vec.to-patternsConvert a dataset of phrases collected withsense2vec.teachto token-based
match patterns that can be used withspaCy'sEntityRuleror recipes likener.match. If no output file is specified, the patterns are
written to stdout. The examples are tokenized so that multi-token terms are
represented correctly, e.g.:{"label": "SHOE_BRAND", "pattern": [{ "LOWER": "new" }, { "LOWER": "balance" }]}.prodigysense2vec.to-patterns[dataset][spacy_model][label][--output-file][--case-sensitive][--dry]ArgumentTypeDescriptiondatasetpositionalPhrase dataset to convert.spacy_modelpositionalspaCy model for tokenization.labelpositionalLabel to apply to all patterns.--output-file,-ooptionOptional output file. Defaults to stdout.--case-sensitive,-CSflagMake patterns case-sensitive.--dry,-DflagPerform a dry run and don't output anything.Exampleprodigysense2vec.to-patternstech_phrasesen_core_web_smTECHNOLOGY
--output-file/path/to/patterns.jsonlrecipesense2vec.evalEvaluate a sense2vec model by asking about phrase triples: is word A more
similar to word B, or to word C? If the human mostly agrees with the model, the
vectors model is good. The recipe will only ask about vectors with the same
sense and supports different example selection strategies.prodigysense2vec.eval[dataset][vectors_path][--strategy][--senses][--exclude-senses][--n-freq][--threshold][--batch-size][--eval-whole][--eval-only][--show-scores]ArgumentTypeDescriptiondatasetpositionalDataset to save annotations to.vectors_pathpositionalPath to pretrained sense2vec vectors.--strategy,-stoptionExample selection strategy.most similar(default) orrandom.--senses,-soptionComma-separated list of senses to limit the selection to. If not set, all senses in the vectors will be used.--exclude-senses,-esoptionComma-separated list of senses to exclude. Seeprodigy_recipes.EVAL_EXCLUDE_SENSESfro the defaults.--n-freq,-foptionNumber of most frequent entries to limit to.--threshold,-toptionMinimum similarity threshold to consider examples.--batch-size,-boptionBatch size to use.--eval-whole,-EflagEvaluate the whole dataset instead of the current session.--eval-only,-OflagDon't annotate, only evaluate the current dataset.--show-scores,-SflagShow all scores for debugging.StrategiesNameDescriptionmost_similarPick a random word from a random sense and get its most similar entries of the same sense. Ask about the similarity to the last and middle entry from that selection.most_least_similarPick a random word from a random sense and get the least similar entry from its most similar entries, and then the last most similar entry of that.randomPick a random sample of 3 words from the same random sense.Exampleprodigysense2vec.evalvectors_eval/path/to/s2v_reddit_2015_md
--sensesNOUN,ORG,PRODUCT--threshold0.5recipesense2vec.eval-most-similarEvaluate a vectors model by looking at the most similar entries it returns for a
random phrase and unselecting the mistakes.prodigysense2vec.eval[dataset][vectors_path][--senses][--exclude-senses][--n-freq][--n-similar][--batch-size][--eval-whole][--eval-only][--show-scores]ArgumentTypeDescriptiondatasetpositionalDataset to save annotations to.vectors_pathpositionalPath to pretrained sense2vec vectors.--senses,-soptionComma-separated list of senses to limit the selection to. If not set, all senses in the vectors will be used.--exclude-senses,-esoptionComma-separated list of senses to exclude. Seeprodigy_recipes.EVAL_EXCLUDE_SENSESfro the defaults.--n-freq,-foptionNumber of most frequent entries to limit to.--n-similar,-noptionNumber of similar items to check. Defaults to10.--batch-size,-boptionBatch size to use.--eval-whole,-EflagEvaluate the whole dataset instead of the current session.--eval-only,-OflagDon't annotate, only evaluate the current dataset.--show-scores,-SflagShow all scores for debugging.prodigysense2vec.eval-most-similarvectors_eval_sim/path/to/s2v_reddit_2015_md
--sensesNOUN,ORG,PRODUCTrecipesense2vec.eval-abPerform an A/B evaluation of two pretrained sense2vec vector models by comparing
the most similar entries they return for a random phrase. The UI shows two
randomized options with the most similar entries of each model and highlights
the phrases that differ. At the end of the annotation session the overall stats
and preferred model are shown.prodigysense2vec.eval[dataset][vectors_path_a][vectors_path_b][--senses][--exclude-senses][--n-freq][--n-similar][--batch-size][--eval-whole][--eval-only][--show-mapping]ArgumentTypeDescriptiondatasetpositionalDataset to save annotations to.vectors_path_apositionalPath to pretrained sense2vec vectors.vectors_path_bpositionalPath to pretrained sense2vec vectors.--senses,-soptionComma-separated list of senses to limit the selection to. If not set, all senses in the vectors will be used.--exclude-senses,-esoptionComma-separated list of senses to exclude. Seeprodigy_recipes.EVAL_EXCLUDE_SENSESfro the defaults.--n-freq,-foptionNumber of most frequent entries to limit to.--n-similar,-noptionNumber of similar items to check. Defaults to10.--batch-size,-boptionBatch size to use.--eval-whole,-EflagEvaluate the whole dataset instead of the current session.--eval-only,-OflagDon't annotate, only evaluate the current dataset.--show-mapping,-SflagShow which models are option 1 and option 2 in the UI (for debugging).prodigysense2vec.eval-abvectors_eval_sim/path/to/s2v_reddit_2015_md/path/to/s2v_reddit_2019_md--sensesNOUN,ORG,PRODUCTPretrained vectorsThe pretrained Reddit vectors support the following "senses", either
part-of-speech tags or entity labels. For more details, see spaCy'sannotation scheme overview.TagDescriptionExamplesADJadjectivebig, old, greenADPadpositionin, to, duringADVadverbvery, tomorrow, down, whereAUXauxiliaryis, has (done), will (do)CONJconjunctionand, or, butDETdeterminera, an, theINTJinterjectionpsst, ouch, bravo, helloNOUNnoungirl, cat, tree, air, beautyNUMnumeral1, 2017, one, seventy-seven, MMXIVPARTparticle's, notPRONpronounI, you, he, she, myself, somebodyPROPNproper nounMary, John, London, NATO, HBOPUNCTpunctuation, ? ( )SCONJsubordinating conjunctionif, while, thatSYMsymbol$, %, =, :), 😝VERBverbrun, runs, running, eat, ate, eatingEntity LabelDescriptionPERSONPeople, including fictional.NORPNationalities or religious or political groups.FACILITYBuildings, airports, highways, bridges, etc.ORGCompanies, agencies, institutions, etc.GPECountries, cities, states.LOCNon-GPE locations, mountain ranges, bodies of water.PRODUCTObjects, vehicles, foods, etc. (Not services.)EVENTNamed hurricanes, battles, wars, sports events, etc.WORK_OF_ARTTitles of books, songs, etc.LANGUAGEAny named language.