Module Expr Str Import alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias alias alias ImportFrom alias identifier alias alias identifier Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore NameLoad TryExcept body Import alias handlers ExceptHandler type NameLoad body Pass ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam defaults body Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr If CompareIn Str NameLoad body Delete SubscriptDel NameLoad Index Str Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str TupleLoad AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad NameLoad Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body If UnaryOpNot Call NameLoad NameLoad Str body Expr Call AttributeLoad NameLoad attr BinOpMod Str Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr keyword Str Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assert CompareLtELt Num NameLoad Call NameLoad NameLoad Str Return SubscriptLoad AttributeLoad Call AttributeLoad NameLoad attr attr Index NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr TryExcept body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr handlers ExceptHandler body Raise Call NameLoad Str Return SubscriptLoad NameLoad Index NameLoad decorator_list decorator_list FunctionDef arguments args NameParam defaults body Assign TupleStore NameStore NameStore NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad NameLoad AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore SubscriptLoad NameLoad Index NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad NameLoad Call AttributeLoad NameLoad attr Return NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad Num Num body Expr Str If CompareIs NameLoad NameLoad body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword Str orelse Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr ListLoad Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr TupleLoad ListLoad Num If CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return BinOpAdd Call NameLoad AttributeLoad NameLoad attr Call NameLoad ListComp Call NameLoad NameLoad comprehension NameStore AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str TupleLoad Call NameLoad NameLoad Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Num If BoolOpAnd AttributeLoad NameLoad attr CompareLt Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Num BinOpMult NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr For NameStore NameLoad body If Call NameLoad NameLoad AttributeLoad NameLoad attr body Assign NameStore Call NameLoad NameLoad orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore AttributeLoad NameLoad attr orelse Assign NameStore Call NameLoad NameLoad If CompareLt NameLoad BinOpMult Num AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr ListLoad NameLoad AttributeLoad NameLoad attr attr orelse Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AugAssignAdd AttributeStore NameLoad attr NameLoad If CompareGtE Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr If CompareEq BinOpMod Call NameLoad AttributeLoad NameLoad attr Num Num body Expr Call AttributeLoad NameLoad attr BinOpMod Str Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If Call AttributeLoad AttributeLoad NameLoad attr attr Str body Return BinOpMod Str TupleLoad AttributeLoad NameLoad attr NameLoad orelse Return BinOpMod Str TupleLoad AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str If UnaryOpNot AttributeLoad NameLoad attr body Return Assign NameStore Call NameLoad AttributeLoad NameLoad attr Assign NameStore CompareGt Num BinOpDiv BinOpMult Num AttributeLoad NameLoad attr BinOpMult Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr If NameLoad body Assign NameStore Call NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword Call NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr orelse Assign NameStore Call NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad IfExp NameLoad Str Str NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr NameLoad NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign TupleStore AttributeStore NameLoad attr AttributeStore NameLoad attr TupleLoad ListLoad Num decorator_list FunctionDef arguments args NameParam defaults body Assert AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Raise Call NameLoad Str Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index Num Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr BinOpMod Str Call NameLoad NameLoad Assign AttributeStore NameLoad attr Call NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Delete SubscriptDel AttributeLoad NameLoad attr Index Num Expr Call AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad BinOpMult ListLoad NameLoad Call NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr If BoolOpAnd NameLoad CompareGt NameLoad Num body Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad keyword BinOpAdd Num BinOpDiv Call NameLoad NameLoad NameLoad orelse Assign NameStore NameLoad Assign NameStore Call NameLoad NameLoad NameLoad Return TupleLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr For NameStore AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad If CompareIs AttributeLoad NameLoad attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad orelse Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd ListLoad Num ListComp Call NameLoad NameLoad comprehension NameStore AttributeLoad NameLoad attr Assign NameStore Lambda arguments args NameParam NameParam defaults ListComp TupleLoad BinOpAdd NameLoad SubscriptLoad NameLoad Index NameLoad NameLoad comprehension TupleStore NameStore NameStore NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore BoolOpOr NameLoad BoolOpAnd Call NameLoad NameLoad Str CompareGt AttributeLoad NameLoad attr Num CompareGt SubscriptLoad AttributeLoad NameLoad attr Index Num Num If UnaryOpNot NameLoad body Assign NameStore GeneratorExp Call NameLoad NameLoad NameLoad comprehension TupleStore NameStore NameStore Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad keyword Lambda arguments args NameParam defaults SubscriptLoad NameLoad Index Num orelse Assign NameStore ListLoad For TupleStore NameStore NameStore Call NameLoad NameLoad body Assign NameStore ListComp Call NameLoad NameLoad NameLoad comprehension NameStore NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore ListLoad For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad keyword Lambda arguments args NameParam defaults SubscriptLoad NameLoad Index Num Expr Call AttributeLoad NameLoad attr NameLoad If NameLoad body Expr Call AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr Assign NameStore Num For NameStore AttributeLoad NameLoad attr body AugAssignAdd NameStore Call NameLoad NameLoad If CompareLt NameLoad NameLoad body Break If BoolOpOr UnaryOpNot AttributeLoad NameLoad attr CompareLt NameLoad Num CompareGtE NameLoad NameLoad body Raise Call NameLoad BinOpMod Str TupleLoad NameLoad Call NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd BinOpSub NameLoad NameLoad Call NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign TupleStore NameStore AttributeStore NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign AttributeStore NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign TupleStore NameStore AttributeStore NameLoad attr TupleLoad AttributeLoad NameLoad attr NameLoad For NameStore Call AttributeLoad NameLoad attr body If CompareGt SubscriptLoad AttributeLoad NameLoad attr Index Num Num body For NameStore SubscriptLoad NameLoad Index NameLoad body Expr Yield NameLoad orelse Expr Yield SubscriptLoad NameLoad Index NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str Expr Call AttributeLoad NameLoad attr If CompareIs NameLoad NameLoad body Assign NameStore AttributeLoad NameLoad attr For NameStore AttributeLoad NameLoad attr body Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr attr For NameStore Call NameLoad Num SubscriptLoad AttributeLoad NameLoad attr Index Num NameLoad body Assign NameStore Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Num BinOpAdd NameLoad NameLoad Assign NameStore SubscriptLoad NameLoad Slice NameLoad NameLoad Expr Yield NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr For NameStore AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad vararg kwarg body Expr Str Expr Call AttributeLoad NameLoad attr If CompareIs NameLoad NameLoad body Assign NameStore AttributeLoad NameLoad attr Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Import alias For NameStore Call AttributeLoad NameLoad attr BinOpAdd AttributeLoad NameLoad attr Str body Expr Call AttributeLoad NameLoad attr BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad AttributeLoad NameLoad attr NameLoad Num body Expr Str If CompareIs NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str Assign NameStore BinOpAdd Num Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If CompareIsNot NameLoad NameLoad body If CompareLtE AttributeLoad NameLoad attr Num body Raise Call NameLoad Str Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad Call NameLoad NameLoad NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr keyword TupleLoad Call NameLoad NameLoad NameLoad keyword NameLoad For TupleStore NameStore NameStore Call NameLoad NameLoad body If CompareEq BinOpMod NameLoad Num Num body Expr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad NameLoad Call NameLoad NameLoad If Call NameLoad NameLoad AttributeLoad NameLoad attr body Pass orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr orelse Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr ListComp Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr comprehension NameStore NameLoad keyword AttributeLoad AttributeLoad NameLoad attr attr orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr orelse If Call NameLoad NameLoad AttributeLoad NameLoad attr body Pass orelse Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr attr Return NameLoad decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad Num AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad If CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Str TryExcept body Assign TupleStore NameStore NameStore NameStore TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str handlers ExceptHandler type NameLoad body Pass If CompareIsNot NameLoad NameLoad body Assign NameStore NameLoad If CompareIs NameLoad NameLoad body Raise Call NameLoad Str Assign NameStore GeneratorExp IfExp Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Call AttributeLoad NameLoad attr NameLoad IfExp Call NameLoad NameLoad AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr NameLoad comprehension NameStore NameLoad Assign AttributeStore NameLoad attr AttributeLoad Call AttributeLoad NameLoad attr NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword NameLoad keyword Num attr Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Expr Call AttributeLoad NameLoad attr BinOpMod Str AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Num keyword AttributeLoad AttributeLoad NameLoad attr attr orelse If Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad body Assign NameStore AttributeLoad NameLoad attr orelse If Call NameLoad NameLoad AttributeLoad NameLoad attr body If CompareEq AttributeLoad NameLoad attr Num body Assign AttributeStore NameLoad attr TupleLoad Num Call NameLoad NameLoad Assign NameStore AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad keyword AttributeLoad AttributeLoad NameLoad attr attr attr orelse Assign NameStore Call AttributeLoad NameLoad attr ListLoad NameLoad SubscriptLoad AttributeLoad AttributeLoad NameLoad attr attr Index Num keyword AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore BinOpMult AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr If BoolOpAnd CompareEq SubscriptLoad AttributeLoad NameLoad attr Index Num Num UnaryOpNot NameLoad body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr orelse Assign NameStore AttributeLoad Call AttributeLoad NameLoad attr attr Return NameLoad decorator_list decorator_list
N/A N/A 
This module contains functions and classes for computing similarities across
a collection of documents in the Vector Space Model.

The main class is `Similarity`, which builds an index for a given set of documents.
Once the index is built, you can perform efficient queries like "Tell me how similar
is this query document to each document in the index?". The result is a vector
of numbers as large as the size of the initial set of documents, that is, one float
for each index document. Alternatively, you can also request only the top-N most
similar index documents to the query.

You can later add new documents to the index via `Similarity.add_documents()`.

How It Works
------------

The `Similarity` class splits the index into several smaller sub-indexes ("shards"),
which are disk-based. If your entire index fits in memory (~hundreds of thousands
documents for 1GB of RAM), you can also use the `MatrixSimilarity` or `SparseMatrixSimilarity`
classes directly. These are more simple but do not scale well (the entire index is
kept in RAM).

Once the index has been initialized, you can query for document similarity simply by:

>>> index = Similarity('/tmp/tst', corpus, num_features=12) # build the index
>>> similarities = index[query] # get similarities between the query and all index documents

If you have more query documents, you can submit them all at once, in a batch:

>>> for similarities in index[batch_of_documents]: # the batch is simply an iterable of documents (=gensim corpus)
>>>     ...

The benefit of this batch (aka "chunked") querying is much better performance.
To see the speed-up on your machine, run ``python -m gensim.test.simspeed``
(compare to my results `here <http://groups.google.com/group/gensim/msg/4f6f171a869e4fca?>`_).

There is also a special syntax for when you need similarity of documents in the index
to the index itself (i.e. queries=indexed documents themselves). This special syntax
already uses the faster, batch queries internally:

>>> for similarities in index: # return all similarities of the 1st index document, then 2nd...
>>>     ...


 N/A logging N/A itertools N/A os N/A heapq N/A numpy N/A scipy.sparse gensim interfaces utils matutils six.moves map imap xrange zip izip N/A logger N/A N/A logging getLogger gensim.similarities.docsim N/A PARALLEL_SHARDS False N/A N/A N/A multiprocessing N/A N/A N/A ImportError N/A N/A Shard N/A N/A utils SaveLoad N/A N/A 
    A proxy class that represents a single shard instance within a Similarity
    index.

    Basically just wraps (Sparse)MatrixSimilarity so that it mmaps from disk on
    request (query).

     __init__ N/A N/A self fname index N/A N/A N/A N/A N/A self dirname N/A self fname N/A N/A N/A os path split fname N/A N/A self length N/A len index N/A N/A self cls N/A index __class__ N/A N/A N/A logger info N/A saving index shard to %s N/A N/A self fullname N/A N/A N/A index save N/A N/A self fullname N/A N/A self index N/A N/A self get_index N/A fullname N/A N/A self N/A N/A N/A N/A N/A N/A os path join N/A self dirname N/A self fname N/A __len__ N/A N/A self N/A N/A N/A N/A self length N/A __getstate__ N/A N/A self N/A N/A N/A result N/A N/A N/A self __dict__ copy N/A N/A index result N/A N/A N/A result N/A index N/A result N/A __str__ N/A N/A self N/A N/A N/A N/A %s Shard(%i documents in %s) N/A N/A N/A self cls __name__ N/A len self N/A N/A self fullname N/A get_index N/A N/A self N/A N/A N/A N/A N/A hasattr self index N/A N/A N/A N/A logger debug N/A mmaping index from %s N/A N/A self fullname N/A N/A self index N/A N/A N/A self cls load N/A N/A self fullname mmap r N/A N/A self index N/A get_document_id N/A N/A self pos N/A N/A N/A Return index vector at position `pos`.

        The vector is of the same type as the underlying index (ie., dense for
        MatrixSimilarity and scipy.sparse for SparseMatrixSimilarity.
         N/A N/A 0 pos N/A len self requested position out of range N/A N/A N/A N/A N/A self get_index index N/A pos N/A __getitem__ N/A N/A self query N/A N/A N/A index N/A N/A self get_index N/A N/A N/A N/A index num_best N/A self num_best N/A N/A index normalize N/A self normalize N/A N/A N/A N/A N/A ValueError num_best and normalize have to be set before querying a proxy Shard object N/A N/A index N/A query N/A N/A query_shard N/A N/A args N/A N/A N/A N/A query shard args N/A N/A N/A logger debug N/A querying shard %s num_best=%s in process %s N/A shard N/A shard num_best N/A N/A os getpid N/A result N/A shard N/A query N/A N/A N/A logger debug N/A finished querying shard %s in process %s N/A shard N/A N/A os getpid N/A result N/A Similarity N/A N/A interfaces SimilarityABC N/A N/A 
    Compute cosine similarity of a dynamic query against a static corpus of documents
    ("the index").

    Scalability is achieved by sharding the index into smaller pieces, each of which
    fits into core memory (see the `(Sparse)MatrixSimilarity` classes in this module).
    The shards themselves are simply stored as files to disk and mmap'ed back as needed.

     __init__ N/A N/A self output_prefix corpus num_features num_best chunksize shardsize N/A None 256 32768 N/A N/A 
        Construct the index from `corpus`. The index can be later extended by calling
        the `add_documents` method. **Note**: documents are split (internally, transparently)
        into shards of `shardsize` documents each, converted to a matrix, for faster BLAS calls.
        Each shard is stored to disk under `output_prefix.shard_number` (=you need write
        access to that location). If you don't specify an output prefix, a random
        filename in temp will be used.

        `shardsize` should be chosen so that a `shardsize x chunksize` matrix of floats
        fits comfortably into main memory.

        `num_features` is the number of features in the `corpus` (e.g. size of the
        dictionary, or the number of latent topics for latent semantic models).

        If `num_best` is left unspecified, similarity queries will return a full
        vector with one float for every document in the index:

        >>> index = Similarity('/path/to/index', corpus, num_features=400) # if corpus has 7 documents...
        >>> index[query] # ... then result will have 7 floats
        [0.0, 0.0, 0.2, 0.13, 0.8, 0.0, 0.1]

        If `num_best` is set, queries return only the `num_best` most similar documents,
        always leaving out documents for which the similarity is 0.
        If the input vector itself only has features with zero values (=the sparse
        representation is empty), the returned list will always be empty.

        >>> index.num_best = 3
        >>> index[query] # return at most "num_best" of `(index_of_document, similarity)` tuples
        [(4, 0.8), (2, 0.13), (3, 0.13)]

        You can also override `num_best` dynamically, simply by setting e.g.
        `self.num_best = 10` before doing a query.

         N/A N/A output_prefix None N/A N/A N/A self output_prefix N/A N/A utils randfname prefix simserver N/A N/A N/A self output_prefix output_prefix N/A N/A N/A logger info N/A starting similarity index under %s N/A self output_prefix N/A N/A self num_features num_features N/A N/A self num_best num_best N/A N/A self normalize True N/A N/A self chunksize N/A int chunksize N/A N/A self shardsize shardsize N/A N/A self shards N/A N/A N/A N/A self fresh_docs N/A self fresh_nnz N/A N/A 0 N/A N/A corpus None N/A N/A N/A N/A self add_documents corpus N/A __len__ N/A N/A self N/A N/A N/A N/A N/A len N/A self fresh_docs N/A sum N/A N/A len shard N/A shard N/A self shards N/A __str__ N/A N/A self N/A N/A N/A N/A Similarity index with %i documents in %i shards (stored under %s) N/A N/A len self N/A len N/A self shards N/A self output_prefix N/A add_documents N/A N/A self corpus N/A N/A N/A 
        Extend the index with new documents.

        Internally, documents are buffered and then spilled to disk when there's
        `self.shardsize` of them (or when a query is issued).
         N/A min_ratio 1.0 N/A N/A N/A self shards N/A N/A len N/A N/A self shards N/A -1 N/A min_ratio N/A self shardsize N/A N/A N/A N/A self reopen_shard N/A doc corpus N/A N/A N/A isinstance doc N/A numpy ndarray N/A N/A doclen N/A len doc N/A N/A N/A N/A N/A scipy sparse issparse doc N/A N/A doclen N/A doc nnz N/A N/A doclen N/A len doc N/A N/A doclen N/A 0.3 N/A self num_features N/A N/A doc N/A N/A matutils unitvec N/A N/A N/A matutils corpus2csc N/A doc N/A self num_features T N/A N/A doc N/A N/A matutils unitvec N/A N/A matutils sparse2full doc N/A self num_features N/A N/A N/A N/A self fresh_docs append doc N/A N/A self fresh_nnz doclen N/A N/A N/A len N/A self fresh_docs N/A self shardsize N/A N/A N/A N/A self close_shard N/A N/A N/A N/A len N/A self fresh_docs 10000 0 N/A N/A N/A N/A logger info N/A PROGRESS: fresh_shard size=%i N/A len N/A self fresh_docs N/A shardid2filename N/A N/A self shardid N/A N/A N/A N/A N/A N/A self output_prefix endswith . N/A N/A N/A %s%s N/A N/A self output_prefix shardid N/A N/A N/A %s.%s N/A N/A self output_prefix shardid N/A close_shard N/A N/A self N/A N/A N/A 
        Force the latest shard to close (be converted to a matrix and stored
        to disk). Do nothing if no new documents added since last call.

        **NOTE**: the shard is closed even if it is not full yet (its size is smaller
        than `self.shardsize`). If documents are added later via `add_documents()`,
        this incomplete shard will be loaded again and completed.
         N/A N/A N/A self fresh_docs N/A N/A N/A shardid N/A len N/A self shards N/A issparse N/A 0.3 N/A N/A 1.0 N/A self fresh_nnz N/A N/A len N/A self fresh_docs N/A self num_features N/A issparse N/A N/A index N/A SparseMatrixSimilarity N/A self fresh_docs num_terms N/A self num_features num_docs N/A len N/A self fresh_docs num_nnz N/A self fresh_nnz N/A N/A index N/A MatrixSimilarity N/A self fresh_docs num_features N/A self num_features N/A N/A N/A logger info N/A creating %s shard #%s N/A N/A issparse sparse dense shardid N/A shard N/A Shard N/A N/A self shardid2filename shardid index N/A N/A shard num_best N/A self num_best N/A N/A shard num_nnz N/A self fresh_nnz N/A N/A N/A N/A self shards append shard N/A N/A N/A self fresh_docs N/A self fresh_nnz N/A N/A 0 N/A reopen_shard N/A N/A self N/A N/A N/A N/A self shards N/A N/A self fresh_docs N/A N/A N/A ValueError cannot reopen a shard with fresh documents in index N/A last_shard N/A N/A self shards N/A -1 N/A last_index N/A N/A last_shard get_index N/A N/A N/A logger info N/A reopening an incomplete shard of %i documents N/A len last_shard N/A N/A self fresh_docs N/A list N/A last_index index N/A N/A self fresh_nnz N/A last_shard num_nnz N/A N/A N/A self shards N/A -1 N/A N/A N/A logger debug reopen complete N/A query_shards N/A N/A self query N/A N/A N/A 
        Return the result of applying shard[query] for each shard in self.shards,
        as a sequence.

        If PARALLEL_SHARDS is set, the shards are queried in parallel, using
        the multiprocessing module.
         N/A args N/A zip N/A N/A query N/A len N/A self shards N/A self shards N/A N/A PARALLEL_SHARDS N/A PARALLEL_SHARDS 1 N/A N/A N/A N/A logger debug N/A spawning %i query processes PARALLEL_SHARDS N/A pool N/A N/A multiprocessing Pool PARALLEL_SHARDS N/A result N/A N/A pool imap query_shard args chunksize N/A 1 N/A N/A len args PARALLEL_SHARDS N/A N/A pool None N/A result N/A imap query_shard args N/A N/A pool result N/A __getitem__ N/A N/A self query N/A N/A N/A Get similarities of document `query` to all documents in the corpus.

        **or**

        If `query` is a corpus (iterable of documents), return a matrix of similarities
        of all query documents vs. all corpus document. This batch query is more
        efficient than computing the similarities one document after another.
         N/A N/A N/A self close_shard N/A shard N/A self shards N/A N/A N/A shard num_best N/A self num_best N/A N/A shard normalize N/A self normalize N/A N/A pool shard_results N/A N/A self query_shards query N/A N/A N/A self num_best None N/A N/A result N/A N/A numpy hstack shard_results N/A N/A offsets N/A N/A numpy cumsum N/A N/A 0 N/A N/A len shard N/A shard N/A self shards N/A convert N/A N/A N/A doc shard_no N/A N/A N/A N/A doc_index N/A offsets N/A shard_no sim N/A N/A doc_index sim doc N/A N/A is_corpus query N/A N/A utils is_corpus query N/A is_corpus N/A is_corpus N/A N/A hasattr query ndim N/A N/A query ndim 1 N/A N/A N/A query shape N/A 0 1 N/A N/A is_corpus N/A N/A results N/A N/A convert result shard_no N/A N/A shard_no result N/A enumerate shard_results N/A result N/A N/A heapq nlargest N/A self num_best N/A N/A itertools chain results key N/A N/A N/A item N/A N/A item N/A 1 N/A N/A results N/A N/A N/A shard_no result N/A enumerate shard_results N/A N/A shard_result N/A N/A convert doc shard_no N/A doc result N/A N/A N/A results append shard_result N/A result N/A N/A parts N/A izip results N/A N/A merged N/A N/A heapq nlargest N/A self num_best N/A N/A itertools chain parts key N/A N/A N/A item N/A N/A item N/A 1 N/A N/A N/A result append merged N/A pool N/A N/A N/A N/A pool terminate N/A result N/A vector_by_id N/A N/A self docpos N/A N/A N/A 
        Return indexed vector corresponding to the document at position `docpos`.
         N/A N/A N/A self close_shard N/A pos 0 N/A shard N/A self shards N/A N/A pos N/A len shard N/A N/A docpos pos N/A N/A N/A N/A N/A N/A self shards N/A docpos 0 N/A docpos pos N/A N/A N/A ValueError N/A invalid document position: %s (must be 0 <= x < %s) N/A docpos N/A len self N/A result N/A N/A shard get_document_id N/A N/A docpos pos N/A len shard N/A result N/A similarity_by_id N/A N/A self docpos N/A N/A N/A 
        Return similarity of the given document only. `docpos` is the position
        of the query document within index.
         N/A query N/A N/A self vector_by_id docpos N/A N/A norm N/A self normalize N/A N/A self normalize False N/A result N/A self N/A query N/A N/A self normalize norm N/A result N/A __iter__ N/A N/A self N/A N/A N/A 
        For each index document, compute cosine similarity against all other
        documents in the index and yield the result.
         N/A N/A norm N/A self normalize N/A N/A self normalize False N/A chunk N/A N/A self iter_chunks N/A N/A N/A N/A N/A chunk shape N/A 0 1 N/A N/A sim N/A self N/A chunk N/A N/A N/A sim N/A N/A N/A N/A self N/A chunk N/A N/A self normalize norm N/A iter_chunks N/A N/A self chunksize N/A None N/A N/A 
        Iteratively yield the index as chunks of documents, each of size <= chunksize.

        The chunk is returned in its raw form (matrix or sparse matrix slice).
        The size of the chunk may be smaller than requested; it is up to the caller
        to check the result for real length, using `chunk.shape[0]`.
         N/A N/A N/A self close_shard N/A N/A chunksize None N/A N/A chunksize N/A self chunksize N/A shard N/A self shards N/A N/A query N/A N/A N/A shard get_index index N/A chunk_start N/A xrange 0 N/A N/A query shape N/A 0 chunksize N/A N/A chunk_end N/A min N/A N/A query shape N/A 0 N/A chunk_start chunksize N/A chunk N/A query N/A chunk_start chunk_end N/A N/A chunk N/A check_moved N/A N/A self N/A N/A N/A 
        Update shard locations, in case the server directory has moved on filesystem.
         N/A dirname N/A N/A N/A os path dirname N/A self output_prefix N/A shard N/A self shards N/A N/A N/A shard dirname dirname N/A save N/A N/A self fname N/A None args kwargs N/A N/A 
        Save the object via pickling (also see load) under filename specified in
        the constructor.

        Calls `close_shard` internally to spill any unfinished shards to disk first.

         N/A N/A N/A self close_shard N/A N/A fname None N/A N/A fname N/A self output_prefix N/A N/A N/A N/A super Similarity self save fname args kwargs N/A destroy N/A N/A self N/A N/A N/A 
        Delete all files under self.output_prefix. Object is not usable after calling
        this method anymore. Use with care!

         N/A glob N/A fname N/A N/A glob glob N/A N/A self output_prefix * N/A N/A N/A N/A logger info N/A deleting %s fname N/A N/A N/A os remove fname N/A N/A MatrixSimilarity N/A N/A interfaces SimilarityABC N/A N/A 
    Compute similarity against a corpus of documents by storing the index matrix
    in memory. The similarity measure used is cosine between two vectors.

    Use this if your input corpus contains dense vectors (such as documents in LSI
    space) and fits into RAM.

    The matrix is internally stored as a *dense* numpy array. Unless the entire matrix
    fits into main memory, use `Similarity` instead.

    See also `Similarity` and `SparseMatrixSimilarity` in this module.

     __init__ N/A N/A self corpus num_best dtype num_features chunksize N/A None N/A numpy float32 None 256 N/A N/A 
        `num_features` is the number of features in the corpus (will be determined
        automatically by scanning the corpus if not specified). See `Similarity`
        class for description of the other parameters.

         N/A N/A num_features None N/A N/A N/A N/A logger warning scanning corpus to determine the number of features (consider setting `num_features` explicitly) N/A num_features N/A 1 N/A N/A utils get_max_id corpus N/A N/A self num_features num_features N/A N/A self num_best num_best N/A N/A self normalize True N/A N/A self chunksize chunksize N/A N/A corpus None N/A N/A N/A N/A self num_features 0 N/A N/A N/A ValueError cannot index a corpus with zero features (you must specify either `num_features` or a non-empty corpus in the constructor) N/A N/A N/A logger info N/A creating matrix for %s documents and %i features N/A N/A len corpus num_features N/A N/A self index N/A N/A numpy empty shape N/A N/A len corpus num_features dtype dtype N/A N/A docno vector N/A enumerate corpus N/A N/A N/A N/A docno 1000 0 N/A N/A N/A N/A logger debug N/A PROGRESS: at document #%i/%i N/A docno N/A len corpus N/A N/A isinstance vector N/A numpy ndarray N/A N/A N/A N/A N/A N/A N/A scipy sparse issparse vector N/A N/A vector N/A N/A N/A N/A vector toarray flatten N/A N/A vector N/A N/A matutils unitvec N/A N/A matutils sparse2full vector num_features N/A N/A N/A self index N/A docno vector N/A __len__ N/A N/A self N/A N/A N/A N/A N/A N/A self index shape N/A 0 N/A get_similarities N/A N/A self query N/A N/A N/A 
        Return similarity of sparse vector `query` to all documents in the corpus,
        as a numpy array.

        If `query` is a collection of documents, return a 2D array of similarities
        of each document in `query` to all documents in the corpus (=batch query,
        faster than processing each document in turn).

        **Do not use this function directly; use the self[query] syntax instead.**

         N/A N/A is_corpus query N/A N/A utils is_corpus query N/A is_corpus N/A N/A query N/A N/A numpy asarray N/A N/A N/A matutils sparse2full vec N/A self num_features N/A vec query dtype N/A N/A self index dtype N/A N/A N/A N/A N/A scipy sparse issparse query N/A N/A query N/A N/A query toarray N/A N/A N/A isinstance query N/A numpy ndarray N/A N/A N/A N/A query N/A N/A matutils sparse2full query N/A self num_features N/A query N/A N/A numpy asarray query dtype N/A N/A self index dtype N/A result N/A N/A N/A numpy dot N/A self index N/A query T T N/A result N/A N/A SparseMatrixSimilarity N/A N/A interfaces SimilarityABC N/A N/A 
    Compute similarity against a corpus of documents by storing the sparse index
    matrix in memory. The similarity measure used is cosine between two vectors.

    Use this if your input corpus contains sparse vectors (such as documents in
    bag-of-words format) and fits into RAM.

    The matrix is internally stored as a `scipy.sparse.csr` matrix. Unless the entire
    matrix fits into main memory, use `Similarity` instead.

    See also `Similarity` and `MatrixSimilarity` in this module.
     __init__ N/A N/A self corpus num_features num_terms num_docs num_nnz num_best chunksize dtype N/A None None None None None 500 N/A numpy float32 N/A N/A N/A self num_best num_best N/A N/A self normalize True N/A N/A self chunksize chunksize N/A N/A corpus None N/A N/A N/A N/A logger info creating sparse index N/A N/A N/A N/A num_terms num_docs num_nnz N/A N/A corpus num_terms N/A corpus num_docs N/A corpus num_nnz N/A N/A N/A logger debug using efficient sparse index creation N/A N/A N/A AttributeError N/A N/A N/A N/A num_features None N/A N/A num_terms num_features N/A N/A num_terms None N/A N/A N/A ValueError refusing to guess the number of sparse features: specify num_features explicitly N/A corpus N/A N/A N/A N/A N/A scipy sparse issparse v N/A N/A matutils scipy2sparse v N/A N/A isinstance v N/A numpy ndarray N/A N/A matutils full2sparse v N/A N/A matutils unitvec v N/A v corpus N/A N/A self index N/A N/A N/A matutils corpus2csc corpus num_terms num_terms num_docs num_docs num_nnz num_nnz dtype dtype printprogress 10000 T N/A N/A self index N/A N/A N/A self index tocsr N/A N/A N/A logger info N/A created %r N/A self index N/A __len__ N/A N/A self N/A N/A N/A N/A N/A N/A self index shape N/A 0 N/A get_similarities N/A N/A self query N/A N/A N/A 
        Return similarity of sparse vector `query` to all documents in the corpus,
        as a numpy array.

        If `query` is a collection of documents, return a 2D array of similarities
        of each document in `query` to all documents in the corpus (=batch query,
        faster than processing each document in turn).

        **Do not use this function directly; use the self[query] syntax instead.**

         N/A N/A is_corpus query N/A N/A utils is_corpus query N/A is_corpus N/A N/A query N/A N/A matutils corpus2csc query N/A N/A N/A self index shape N/A 1 dtype N/A N/A self index dtype N/A N/A N/A N/A N/A scipy sparse issparse query N/A N/A query N/A query T N/A N/A N/A isinstance query N/A numpy ndarray N/A N/A N/A N/A query ndim 1 N/A N/A N/A query shape N/A 1 N/A len query N/A query N/A N/A N/A N/A scipy sparse csr_matrix query dtype N/A N/A self index dtype T N/A N/A query N/A N/A matutils corpus2csc N/A query N/A N/A N/A self index shape N/A 1 dtype N/A N/A self index dtype N/A result N/A N/A self index N/A N/A query tocsc N/A N/A N/A N/A N/A result shape N/A 1 1 N/A is_corpus N/A N/A result N/A N/A N/A N/A result toarray flatten N/A N/A result N/A N/A N/A result toarray T N/A result N/A N/A