Package: reach
reachA light-weight package for working with pre-trained word embeddings.
Useful for input into neural networks, or for doing compositional semantics.reachcan read in word vectors inword2vecorgloveformat without
any preprocessing.The assumption behindreachis a no-hassle approach to featurization. The
vectorization and bow approaches know how to deal with OOV words, removing
these problems from your code.reachalso includes nearest neighbor calculation for arbitrary vectors.DocumentationAPI referenceTutorial coming soon (see below for an example)InstallationIf you just wantreach:pip install reachIf you also wantAutoReach:pip install reach[auto]ExampleimportnumpyasnpfromreachimportReach# Load from a .vec or .txt file# unk_word specifies which token is the "unknown" token.# If this is token is not in your vector space, it is added as an extra word# and a corresponding zero vector.# If it is in your embedding space, it is used.r=Reach.load("path/to/embeddings",unk_word="UNK")# Alternatively, if you have a matrix, you can directly# input it.# Stand-in for word embeddingsmtr=np.random.randn(8,300)words=["UNK","cat","dog","best","creature","alive","span","prose"]r=Reach(mtr,words,unk_index=0)# Get vectors through indexing.# Throws a KeyError if a word is not present.vector=r['cat']# Compare two words.similarity=r.similarity('cat','dog')# Find most similar.similarities=r.most_similar('cat',2)sentence='a dog is the best creature alive'.split()corpus=[sentence,sentence,sentence]# bow representation consistent with word vectors,# for input into neural network.bow=r.bow(sentence)# vectorized representation.vectorized=r.vectorize(sentence)# can remove OOV words automatically.vectorized=r.vectorize(sentence,remove_oov=True)# Can mean pool out of the box.mean=r.mean_pool(sentence)# Automatically take care of incorrect sentences# these are set to the vector of the UNK word, or a vector of zeros.corpus_mean=r.mean_pool_corpus([sentence,sentence,["not_a_word"]],remove_oov=True,safeguard=False)# vectorize corpus.transformed=r.transform(corpus)# Get nearest words to arbitrary vectornearest=r.nearest_neighbor(np.random.randn(1,300))# Get every word within a certain thresholdthresholded=r.threshold("cat",threshold=.0)Loading and savingreachhas many options for saving and loading files, including custom separators, custom number of dimensions, loading a custom wordlist, custom number of words, and error recovery. One difference betweengensimandreachis thatreachloads both GloVe-style .vec files and regular word2vec files. Unlikegensim,reachdoes not support loading binary files.benchmarkOn my machine (a 2022 M1 macbook pro), we get the following times forCOW BIG, a file containing about 3 million rows and 320 dimensions.SystemTime (7 loops)Gensim3min 57s ± 344 msreach2min 14s ± 4.09 sFast formatreachhas a special fast format, which is useful if you want to reload your word vectors often. The fast format can be created using thesave_fast_formatfunction, and loaded using theload_fast_formatfunction. This is about equivalent to saving word vectors ingensim's own format in terms of loading speed.autoreachReach also has a way of automatically inferring words from strings without using a pre-defined tokenizer, i.e., without splitting the string into words. This is useful because there might be mismatches between the tokenizer you happen to have on hand, and the word vectors you use. For example, if your vector space contains an embedding for the word"it's", and your tokenizer splits this string into two tokens:["it", "'s"], the embedding for"it's"will never be found.autoreach solves this problem by only finding words from your pre-defined vocabulary in a string, this removing the need for any tokenization. We use theaho-corasick algorithm, which allows us to find substrings in linear time. The downside of using aho-corasick is that it also finds substrings of regular words. For example, the wordthewill be found as a substring ofthese. To circumvent this, we perform a regex-based clean-up step.Warning! The clean-up step involves checking for surrounding spaces and punctuation marks. Hence, if the language for which you use Reach does not actually use spaces and/or punctuation marks to designate word boundaries, the entire process might not work.ExampleimportnumpyasnpfromreachimportAutoReachwords=["dog","walked","home"]vectors=np.random.randn(3,32)r=AutoReach(vectors,words)sentence="The dog, walked, home"bow=r.bow(sentence)found_words=[r.indices[index]forindexinbow]benchmarkBecause we no longer need to tokenize,AutoReachcan be many times faster. In this benchmark, we compare to just splitting, andnltk'sword_tokenizefunction.We will use the entirety of Mary Shelley's Frankenstein, which you can findhere, and the glove.6b.50d vectors, which you can findhere.frompathlibimportPathfromnltkimportword_tokenizefromreachimportAutoReach,Reachtxt=Path("pg42324.txt").read_text().lower()normal_reach=Reach.load("glove.6B.100d.txt")auto_reach=AutoReach.load("glove.6B.100d.txt")# Ipython magic commands%timeitnormal_reach.vectorize(word_tokenize(txt),remove_oov=True)# 345 ms ± 3.42 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)%timeitnormal_reach.vectorize(txt.split(),remove_oov=True)# 25.4 ms ± 132 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)%timeitauto_reach.vectorize(txt)# 69.9 ms ± 237 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)As you can see, the tokenizer introduces significant overhead compared to just splitting, while using the aho-corasick algorithm to split is still reasonably fast.LicenseMITAuthorStéphan Tulkens