Jaguar Vector Store
Source: llama_index

# Jaguar Vector StoreThis document demonstrates llama_index working with Jaguar vector store.- It is a distributed vector database that can store large number of vectors.- The ZeroMove feature enables instant horizontal scaling.- It supports embeddings, text, images, videos, PDFs, audio, time series, and spatial data. - The all-master architecture allows both parallel reads and writes.- Its anomaly detection capabilities can distinguish outliers in the dataset.- The RAG support can combine LLMs and proprietary and real-time data.- Sharing of metadata across multiple vector indexes improves data consistency.- Distance metrics include Euclidean, Cosine, InnerProduct, Manhatten, Chebyshev, Hamming, Jeccard, and Minkowski.- Similarity search can be performed with time cutoff and time decay effects.## PrerequisitesThere are two requirements for running the examples in this file.You must install and set up the JaguarDB server and its HTTP gateway server. Please follow the instructions in [Jaguar Setup](http://www.jaguardb.com/docsetup.html) as a reference.You must install packages llama-index and jaguardb-http-client.    docker pull jaguardb/jaguardb_with_http    docker run -d -p 8888:8888 -p 8080:8080 --name jaguardb_with_http jaguardb/jaguardb_with_http    pip install -U llama-index    pip install -U jaguardb-http-client    ```python%pip install llama-index-vector-stores-jaguar``````python!pip install -U jaguardb-http-client```    Collecting jaguardb-http-client      Using cached jaguardb_http_client-3.4.1-py2.py3-none-any.whl (15 kB)    Installing collected packages: jaguardb-http-client    Successfully installed jaguardb-http-client-3.4.1## ImportsThe following packages should be imported. We use the OpenAIEmbedding as an example. You could choose other embedding models in your application.```pythonfrom llama_index.core import VectorStoreIndex, SimpleDirectoryReaderfrom llama_index.core import StorageContextfrom llama_index.vector_stores.jaguar import JaguarVectorStorefrom jaguardb_http_client.JaguarHttpClient import JaguarHttpClient```## Client ObjectWe now instantiate a jaguar vector store client object. The url is the http endpoint of the gateway server. The url should be replaced with your environment settings. The pod is the Pod (or database) name. The store is the name of the vector store. A pod may have multiple stores. The vector_index is the name of the vector index in the store. A store may have multiple vector indexes. The store client object is, however, bound to one vector index only. The vector_type specifies the attributes of the vector index. In the string "cosine_fraction_short", cosine means that the distance between two vectors is computed with the cosine distance. Fraction means the vector components are fractional numbers. Short means the storage format of the vector components is a short integer of signed 16-bits integers. Storage format could be float of 32-bit floating point numbers. It can also be a byte of 8-bit signed integers. The vector_dimension is the dimension of the vector generated by the provided embedding model.```pythonurl = "http://127.0.0.1:8080/fwww/"pod = "vdb"store = "llamaindex_jaguar_store"vector_index = "v"vector_type = "cosine_fraction_float"# vector_type = "cosine_fraction_short"  # half of memory usage compared to float# vector_type = "cosine_fraction_byte" # quarter of memory usage compared to floatvector_dimension = 1536  # per OpenAIEmbedding modeljaguarstore = JaguarVectorStore(    pod,    store,    vector_index,    vector_type,    vector_dimension,    url,)```## AuthenticationThe client must login or connect to back-end jaguar server for system security and user authentication. Environment variable JAGUAR_API_KEY or file $HOME/.jagrc file must  contain the jaguar api ke issued by your system administrator. The login() method returns True or False. If it returns False, then it may mean that your jaguar api key is invalid, or the http gateway server is not running, or the jaguar server is not running properly.```pythontrue_or_false = jaguarstore.login()print(f"login result is {true_or_false}")```    login result is True## Create Vector StoreWe now create a vector store with a field 'v:text' of size 1024 bytesto hold text, and two additional metadata fields 'author' and 'category'.```pythonmetadata_str = "author char(32), category char(16)"text_size = 1024jaguarstore.create(metadata_str, text_size)```## Load DocumentsThe following code opens the example Paul Gram documents and read them into memory```pythondocuments = SimpleDirectoryReader("../data/paul_graham/").load_data()print(f"loading {len(documents)} doument(s)")```    loading 1 doument(s)## Make IndexPrepare storage context, service context, and make an index object. After the call of from_documents(), there will be 22 vectors saved in the vector store.```python### make a storage context using our vector storestorage_context = StorageContext.from_defaults(vector_store=jaguarstore)### clear all vectors in the vector storejaguarstore.clear()### make an index with the documents,storage contextindex = VectorStoreIndex.from_documents(    documents, storage_context=storage_context)### You could add more documents to the vector store:# jaguarstore.add_documents(some_docs)# jaguarstore.add_documents(more_docs, text_tag="tag to these documents")### print number of documents in jaguar vector storenum = jaguarstore.count()print(f"There are {num} vectors in jaguar vector store")```    There are 22 vectors in jaguar vector store## Ask QuestionsWe get a query engine and ask some questions to the engine.```pythonquery_engine = index.as_query_engine()q = "What did the author do growing up?"print(f"Question: {q}")response = query_engine.query(q)print(f"Answer: {str(response)}\n")q = "What did the author do after his time at Viaweb?"print(f"Question: {q}")response = query_engine.query(q)print(f"Answer: {str(response)}")```    Question: What did the author do growing up?    Answer: The author mentioned that growing up, they worked on two main things outside of school: writing and programming. They wrote short stories and tried writing programs on an IBM 1401 computer.        Question: What did the author do after his time at Viaweb?    Answer: After his time at Viaweb, the author started a company to put art galleries online. However, this idea did not turn out to be successful as art galleries did not want to be online.## Pass Query OptionsWe can pass extra arguments to the query engine to select only a subset of data from the jaguar vector store. This can be achieved by using the `vector_store_kwargs` argument. Parameter day_cutoff is number of days beyond which text will be ignored. day_decay_rate is rate of daily decay for similarity scores.  ```pythonqkwargs = {    "args": "day_cutoff=365,day_decay_rate=0.01",    "where": "category='startup' or category=''",}query_engine_filter = index.as_query_engine(vector_store_kwargs=qkwargs)q = "What was the author's life style?"print(f"Question: {q}")response = query_engine_filter.query(q)print(f"Answer: {str(response)}")```    Question: What was the author's life style?    Answer: The author's lifestyle involved attending the Accademia as a student and painting still lives in their bedroom at night. They also wrote essays and had a messy life, which they thought would be interesting and encouraging to others.## Cleanup and LogoutAll vectors and related data in the vector store can be deleted and the vector store can be removed completely to finish the test. Logout call makes sure resources used by the client are released.```python### remove all the data in the vector store if you wantjaguarstore.clear()### delete the whole vector in the database if you wantjaguarstore.drop()### disconnect from jaguar server and cleanup resourcesjaguarstore.logout()```