MyScale Vector Store
Source: llama_index

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/vector_stores/MyScaleIndexDemo.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a># MyScale Vector StoreIn this notebook we are going to show a quick demo of using the MyScaleVectorStore.If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.```python%pip install llama-index-vector-stores-myscale``````python!pip install llama-index```#### Creating a MyScale Client```pythonimport loggingimport syslogging.basicConfig(stream=sys.stdout, level=logging.INFO)logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))``````pythonfrom os import environimport clickhouse_connectenviron["OPENAI_API_KEY"] = "sk-*"# initialize clientclient = clickhouse_connect.get_client(    host="YOUR_CLUSTER_HOST",    port=8443,    username="YOUR_USERNAME",    password="YOUR_CLUSTER_PASSWORD",)```#### Load documents, build and store the VectorStoreIndex with MyScaleVectorStoreHere we will use a set of Paul Graham essays to provide the text to turn into embeddings, store in a ``MyScaleVectorStore`` and query to find context for our LLM QnA loop.```pythonfrom llama_index.core import VectorStoreIndex, SimpleDirectoryReaderfrom llama_index.vector_stores.myscale import MyScaleVectorStorefrom IPython.display import Markdown, display``````python# load documentsdocuments = SimpleDirectoryReader("../data/paul_graham").load_data()print("Document ID:", documents[0].doc_id)print("Number of Documents: ", len(documents))```    Document ID: a5f2737c-ed18-4e5d-ab9a-75955edb816d    Number of Documents:  1Download Data```python!mkdir -p 'data/paul_graham/'!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'```You can process your files individually using [SimpleDirectoryReader](/examples/data_connectors/simple_directory_reader.ipynb):```pythonloader = SimpleDirectoryReader("./data/paul_graham/")documents = loader.load_data()for file in loader.input_files:    print(file)    # Here is where you would do any preprocessing```    ../data/paul_graham/paul_graham_essay.txt```python# initialize with metadata filter and store indexesfrom llama_index.core import StorageContextfor document in documents:    document.metadata = {"user_id": "123", "favorite_color": "blue"}vector_store = MyScaleVectorStore(myscale_client=client)storage_context = StorageContext.from_defaults(vector_store=vector_store)index = VectorStoreIndex.from_documents(    documents, storage_context=storage_context)```#### Query IndexNow MyScale vector store supports filter search and hybrid searchYou can learn more about [query_engine](/module_guides/deploying/query_engine/index.md) and [retriever](/module_guides/querying/retriever/index.md).```pythonimport textwrapfrom llama_index.core.vector_stores import ExactMatchFilter, MetadataFilters# set Logging to DEBUG for more detailed outputsquery_engine = index.as_query_engine(    filters=MetadataFilters(        filters=[            ExactMatchFilter(key="user_id", value="123"),        ]    ),    similarity_top_k=2,    vector_store_query_mode="hybrid",)response = query_engine.query("What did the author learn?")print(textwrap.fill(str(response), 100))```#### Clear All Indexes```pythonfor document in documents:    index.delete_ref_doc(document.doc_id)```