lukawskikacper/qdrant-landing-page-2024-06-21 huggingface dataset:
qdrant-landing/content/documentation/frameworks/spring-ai.md

---
title: Spring AI
weight: 2200
---

# Spring AI

[Spring AI](https://docs.spring.io/spring-ai/reference/) is a Java framework that provides a [Spring-friendly](https://spring.io/) API and abstractions for developing AI applications.

Qdrant is available as supported vector database for use within your Spring AI projects.

## Installation

You can find the Spring AI installation instructions [here](https://docs.spring.io/spring-ai/reference/getting-started.html).

Add the Qdrant boot starter package.

```xml
<dependency>
 <groupId>org.springframework.ai</groupId>
 <artifactId>spring-ai-qdrant-store-spring-boot-starter</artifactId>
</dependency>
```

## Usage

Configure Qdrant with Spring Boot’s `application.properties`.

```
spring.ai.vectorstore.qdrant.host=<host of your qdrant instance>
spring.ai.vectorstore.qdrant.port=<the GRPC port of your qdrant instance>
spring.ai.vectorstore.qdrant.api-key=<your api key>
spring.ai.vectorstore.qdrant.collection-name=<The name of the collection to use in Qdrant>
```

Learn more about these options in the [configuration reference](https://docs.spring.io/spring-ai/reference/api/vectordbs/qdrant.html#qdrant-vectorstore-properties).

Or you can set up the Qdrant vector store with the `QdrantVectorStoreConfig` options.

```java
@Bean
public QdrantVectorStoreConfig qdrantVectorStoreConfig() {

    return QdrantVectorStoreConfig.builder()
        .withHost("<QDRANT_HOSTNAME>")
        .withPort(<QDRANT_GRPC_PORT>)
        .withCollectionName("<QDRANT_COLLECTION_NAME>")
        .withApiKey("<QDRANT_API_KEY>")
        .build();
}
```

Build the vector store using the config and any of the support [Spring AI embedding providers](https://docs.spring.io/spring-ai/reference/api/embeddings.html#available-implementations).

```java
@Bean
public VectorStore vectorStore(QdrantVectorStoreConfig config, EmbeddingClient embeddingClient) {
    return new QdrantVectorStore(config, embeddingClient);
}
```

You can now use the `VectorStore` instance backed by Qdrant as a vector store in the Spring AI APIs.

<aside role="status">If the collection is not <a href="/documentation/concepts/collections/#create-a-collection">created in advance</a>, <code>QdrantVectorStore</code> will attempt to create one using cosine similarity and the dimension of the configured <code>EmbeddingClient</code>.</aside>

## 📚 Further Reading

- Spring AI [Qdrant reference](https://docs.spring.io/spring-ai/reference/api/vectordbs/qdrant.html)
- Spring AI [API reference](https://docs.spring.io/spring-ai/reference/index.html)
- [Source Code](https://github.com/spring-projects/spring-ai/tree/main/vector-stores/spring-ai-qdrant)
qdrant-landing/content/documentation/frameworks/testcontainers.md

---
title: Testcontainers
weight: 2700
---

# Testcontainers

Qdrant is available as a [Testcontainers module](https://testcontainers.com/modules/qdrant/) in multiple languages. It  facilitates the spawning of a Qdrant instance for end-to-end testing.

As noted by [Testcontainers](https://testcontainers.com/), it "is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container."

## Usage

```java
import org.testcontainers.qdrant.QdrantContainer;

QdrantContainer qdrantContainer = new QdrantContainer("qdrant/qdrant");
```

```go
import (
    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/qdrant"
)

qdrantContainer, err := qdrant.RunContainer(ctx, testcontainers.WithImage("qdrant/qdrant"))
```

```typescript
import { QdrantContainer } from "@testcontainers/qdrant";

const qdrantContainer = await new QdrantContainer("qdrant/qdrant").start();
```

```python
from testcontainers.qdrant import QdrantContainer

qdrant_container = QdrantContainer("qdrant/qdrant").start()
```

Testcontainers modules provide options/methods to configure ENVs, volumes, and virtually everything you can configure in a Docker container.

## Further reading

- [Testcontainers Guides](https://testcontainers.com/guides/)
- [Testcontainers Qdrant Module](https://testcontainers.com/modules/qdrant/)
qdrant-landing/content/documentation/frameworks/txtai.md

---
title: txtai
weight: 500
aliases: [ ../integrations/txtai/ ]
---

# txtai

Qdrant might be also used as an embedding backend in [txtai](https://neuml.github.io/txtai/) semantic applications.

txtai simplifies building AI-powered semantic search applications using Transformers. It leverages the neural embeddings and their 
properties to encode high-dimensional data in a lower-dimensional space and allows to find similar objects based on their embeddings' 
proximity.

Qdrant is not built-in txtai backend and requires installing an additional dependency:

```bash
pip install qdrant-txtai
```

The examples and some more information might be found in [qdrant-txtai repository](https://github.com/qdrant/qdrant-txtai).
qdrant-landing/content/documentation/frameworks/unstructured.md

---
title: Unstructured
weight: 1900
---

# Unstructured

[Unstructured](https://unstructured.io/) is a library designed to help preprocess, structure unstructured text documents for downstream machine learning tasks.

Qdrant can be used as an ingestion destination in Unstructured.

## Setup

Install Unstructured with the `qdrant` extra.

```bash
pip install "unstructured[qdrant]"
```

## Usage


Depending on the use case you can prefer the command line or using it within your application.

### CLI

```bash
EMBEDDING_PROVIDER=${EMBEDDING_PROVIDER:-"langchain-huggingface"}

unstructured-ingest \
  local \
  --input-path example-docs/book-war-and-peace-1225p.txt \
  --output-dir local-output-to-qdrant \
  --strategy fast \
  --chunk-elements \
  --embedding-provider "$EMBEDDING_PROVIDER" \
  --num-processes 2 \
  --verbose \
  qdrant \
  --collection-name "test" \
  --url "http://localhost:6333" \
  --batch-size 80
```

For a full list of the options the CLI accepts, run `unstructured-ingest <upstream connector> qdrant --help`

### Programmatic usage

```python
from unstructured.ingest.connector.local import SimpleLocalConfig
from unstructured.ingest.connector.qdrant import (
    QdrantWriteConfig,
    SimpleQdrantConfig,
)
from unstructured.ingest.interfaces import (
    ChunkingConfig,
    EmbeddingConfig,
    PartitionConfig,
    ProcessorConfig,
    ReadConfig,
)
from unstructured.ingest.runner import LocalRunner
from unstructured.ingest.runner.writers.base_writer import Writer
from unstructured.ingest.runner.writers.qdrant import QdrantWriter

def get_writer() -> Writer:
    return QdrantWriter(
        connector_config=SimpleQdrantConfig(
            url="http://localhost:6333",
            collection_name="test",
        ),
        write_config=QdrantWriteConfig(batch_size=80),
    )

if __name__ == "__main__":
    writer = get_writer()
    runner = LocalRunner(
        processor_config=ProcessorConfig(
            verbose=True,
            output_dir="local-output-to-qdrant",
            num_processes=2,
        ),
        connector_config=SimpleLocalConfig(
            input_path="example-docs/book-war-and-peace-1225p.txt",
        ),
        read_config=ReadConfig(),
        partition_config=PartitionConfig(),
        chunking_config=ChunkingConfig(chunk_elements=True),
        embedding_config=EmbeddingConfig(provider="langchain-huggingface"),
        writer=writer,
        writer_kwargs={},
    )
    runner.run()
```

## Next steps

- Unstructured API [reference](https://unstructured-io.github.io/unstructured/api.html).
- Qdrant ingestion destination [reference](https://unstructured-io.github.io/unstructured/ingest/destination_connectors/qdrant.html).
- [Source Code](https://github.com/Unstructured-IO/unstructured/blob/main/unstructured/ingest/connector/qdrant.py)
qdrant-landing/content/documentation/frameworks/vanna-ai.md

---
title: Vanna.AI
weight: 3000
---

# Vanna.AI

[Vanna](https://vanna.ai/) is a Python package that uses retrieval augmentation to help you generate accurate SQL queries for your database using LLMs.

Vanna works in two easy steps - train a RAG "model" on your data, and then ask questions which will return SQL queries that can be set up to automatically run on your database.

Qdrant is available as a support vector store for ingesting and retrieving your RAG data.

## Installation

```console
pip install 'vanna[qdrant]'
```

## Setup

You can set up a Vanna agent using Qdrant as your vector store and any of the [LLMs supported by Vanna](https://vanna.ai/docs/postgres-openai-vanna-vannadb/).

We'll use OpenAI for demonstration.

```python
from vanna.openai import OpenAI_Chat
from vanna.qdrant import Qdrant_VectorStore
from qdrant_client import QdrantClient

class MyVanna(Qdrant, OpenAI_Chat):
    def __init__(self, config=None):
        Qdrant_VectorStore.__init__(self, config=config)
        OpenAI_Chat.__init__(self, config=config)

vn = MyVanna(config={
    'client': QdrantClient(...),
    'api_key': sk-...,
    'model': gpt-4-...,
})
```

## Usage

Once a Vanna agent is instantiated, you can connect it to [any SQL database](https://vanna.ai/docs/FAQ/#can-i-use-this-with-my-sql-database) of your choosing.

For example, Postgres.

```python
vn.connect_to_postgres(host='my-host', dbname='my-dbname', user='my-user', password='my-password', port='my-port')
```

You can now train and begin querying your database with SQL.

```python
# You can add DDL statements that specify table names, column names, types, and potentially relationships
vn.train(ddl="""
    CREATE TABLE IF NOT EXISTS my-table (
        id INT PRIMARY KEY,
        name VARCHAR(100),
        age INT
    )
""")

# You can add documentation about your business terminology or definitions.
vn.train(documentation="Our business defines OTIF score as the percentage of orders that are delivered on time and in full")

# You can also add SQL queries to your training data. This is useful if you have some queries already laying around.
vn.train(sql="SELECT * FROM my-table WHERE name = 'John Doe'")

# You can remove training data if there's obsolete/incorrect information. 
vn.remove_training_data(id='1-ddl')

# Whenever you ask a new question, Vanna will retrieve 10 most relevant pieces of training data and use it as part of the LLM prompt to generate the SQL.

vn.ask(question="<YOUR_QUESTION>")
```

## Further reading

- [Getting started with Vanna.AI](https://vanna.ai/docs/app/)
- [Vanna.AI documentation](https://vanna.ai/docs/)
- [Source Code](https://github.com/vanna-ai/vanna/tree/main/src/vanna/qdrant)
qdrant-landing/content/documentation/guides/administration.md

---
title: Administration
weight: 10
aliases:
  - ../administration
---

# Administration

Qdrant exposes administration tools which enable to modify at runtime the behavior of a qdrant instance without changing its configuration manually.

## Locking

A locking API enables users to restrict the possible operations on a qdrant process.
It is important to mention that:
- The configuration is not persistent therefore it is necessary to lock again following a restart.
- Locking applies to a single node only. It is necessary to call lock on all the desired nodes in a distributed deployment setup.

Lock request sample:

```http
POST /locks
{
    "error_message": "write is forbidden",
    "write": true
}
```

Write flags enables/disables write lock.
If the write lock is set to true, qdrant doesn't allow creating new collections or adding new data to the existing storage.
However, deletion operations or updates are not forbidden under the write lock.
This feature enables administrators to prevent a qdrant process from using more disk space while permitting users to search and delete unnecessary data.

You can optionally provide the error message that should be used for error responses to users.

## Recovery mode

*Available as of v1.2.0*

Recovery mode can help in situations where Qdrant fails to start repeatedly.
When starting in recovery mode, Qdrant only loads collection metadata to prevent
going out of memory. This allows you to resolve out of memory situations, for
example, by deleting a collection. After resolving Qdrant can be restarted
normally to continue operation.

In recovery mode, collection operations are limited to
[deleting](../../concepts/collections/#delete-collection) a
collection. That is because only collection metadata is loaded during recovery.

To enable recovery mode with the Qdrant Docker image you must set the
environment variable `QDRANT_ALLOW_RECOVERY_MODE=true`. The container will try
to start normally first, and restarts in recovery mode if initialisation fails
due to an out of memory error. This behavior is disabled by default.

If using a Qdrant binary, recovery mode can be enabled by setting a recovery
message in an environment variable, such as
`QDRANT__STORAGE__RECOVERY_MODE="My recovery message"`.
qdrant-landing/content/documentation/guides/common-errors.md

---
title: Troubleshooting
weight: 170
aliases:
  - ../tutorials/common-errors
  - /documentation/troubleshooting/
---

# Solving common errors

## Too many files open (OS error 24)

Each collection segment needs some files to be open. At some point you may encounter the following errors in your server log:

```text
Error: Too many files open (OS error 24)
```

In such a case you may need to increase the limit of the open files. It might be done, for example, while you launch the Docker container:

```bash
docker run --ulimit nofile=10000:10000 qdrant/qdrant:latest
```

The command above will set both soft and hard limits to `10000`.

If you are not using Docker, the following command will change the limit for the current user session:

```bash
ulimit -n 10000
```

Please note, the command should be executed before you run Qdrant server.

## Can't open Collections meta Wal

When starting a Qdrant instance as part of a distributed deployment, you may
come across an error message similar to this:

```bash
Can't open Collections meta Wal: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
```

It means that Qdrant cannot start because a collection cannot be loaded. Its
associated [WAL](../../concepts/storage/#versioning) files are currently
unavailable, likely because the same files are already being used by another
Qdrant instance.

Each node must have their own separate storage directory, volume or mount.

The formed cluster will take care of sharing all data with each node, putting it
all in the correct places for you. If using Kubernetes, each node must have
their own volume. If using Docker, each node must have their own storage mount
or volume. If using Qdrant directly, each node must have their own storage
directory.
qdrant-landing/content/documentation/hybrid-cloud/_index.md

---
title: Hybrid Cloud
weight: 15
---

# Qdrant Hybrid Cloud

Seamlessly deploy and manage your vector database across diverse environments, ensuring performance, security, and cost efficiency for AI-driven applications.

[Qdrant Hybrid Cloud](/hybrid-cloud/) integrates Kubernetes clusters from any setting - cloud, on-premises, or edge - into a unified, enterprise-grade managed service.

You can use [Qdrant Cloud's UI](/documentation/cloud/create-cluster/) to create and manage your database clusters, while they still remain within your infrastructure. **All Qdrant databases will operate solely within your network, using your storage and compute resources.**

Qdrant Hybrid Cloud ensures data privacy, deployment flexibility, low latency, and delivers cost savings, elevating standards for vector search and AI applications.

**How it works:** When you onboard a Kubernetes cluster as a Hybrid Cloud Environment, you can deploy the Qdrant Kubernetes Operator and Cloud Agent into this cluster. These will manage Qdrant databases within your Kubernetes cluster and establish an outgoing connection to Qdrant Cloud at `cloud.qdrant.io` on port `443`. You can then benefit from the same cloud management features and transport telemetry as is available with any managed Qdrant Cloud cluster.

<aside role="status">Qdrant Cloud does not connect to the API of your Kubernetes cluster, cloud provider, or any other platform APIs.</aside>

**Setup instructions:** To begin using Qdrant Hybrid Cloud, [read our installation guide](/documentation/hybrid-cloud/hybrid-cloud-setup/).

## Hybrid Cloud architecture

![hybrid-cloud-architecture](/blog/hybrid-cloud/hybrid-cloud-architecture.png)

## Upcoming roadmap items

We plan to introduce the following configuration options directly in the Qdrant Cloud Console in the future. If you need any of them beforehand, please contact our Support team.

* Node selectors
* Tolerations
* Affinities and anti-affinities
* Service types and annotations
* Ingresses
* Network policies
* Storage classes
* Volume snapshot classes
qdrant-landing/content/documentation/hybrid-cloud/networking-logging-monitoring.md

---
title: Networking, logging & monitoring
weight: 3
---
# Networking, logging & monitoring

## Configure network policies

For security reasons, each database cluster is secured with network policies. By default, database pods only allow egress traffic between each and allow ingress traffic to ports 6333 (rest) and 6334 (grpc) from within the Kubernetes cluster.

You can modify the default network policies in the Hybrid Cloud environment configuration:

```yaml
qdrant:
  networkPolicies:
    ingress:
    - from:
      - ipBlock:
          cidr: 192.168.0.0/22
      - podSelector:
          matchLabels:
            app: client-app
        namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: client-namespace
      - podSelector:
          matchLabels:
            app: traefik
        namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: kube-system
      ports:
      - port: 6333
        protocol: TCP
      - port: 6334
        protocol: TCP      
```

## Logging

You can access the logs with kubectl or the Kubernetes log management tool of your choice. For example:

```bash
kubectl -n qdrant-namespace logs -l app=qdrant,cluster-id=9a9f48c7-bb90-4fb2-816f-418a46a74b24
```

**Configuring log levels:** You can configure log levels for the databases individually in the configuration section of the Qdrant Cluster detail page. The log level for the **Qdrant Cloud Agent** and **Operator** can be set in the [Hybrid Cloud Environment configuration](/documentation/hybrid-cloud/operator-configuration/).

## Monitoring

The Qdrant Cloud console gives you access to basic metrics about CPU, memory and disk usage of your Qdrant clusters. You can also access Prometheus metrics endpoint of your Qdrant databases. Finally, you can use a Kubernetes workload monitoring tool of your choice to monitor your Qdrant clusters.
qdrant-landing/content/documentation/hybrid-cloud/operator-configuration.md

---
title: Advanced Qdrant Operator configuration
weight: 2
---

# Advanced Qdrant Operator configuration

The Qdrant Operator has several configuration options, which can be configured in the advanced section of your Hybrid Cloud Environment.

The following YAML shows all configuration options with their default values:

```yaml
# Retention for the backup history of Qdrant clusters
backupHistoryRetentionDays: 2
# Timeout configuration for the Qdrant operator operations
operationTimeout: 7200 # 2 hours
handlerTimeout: 21600 # 6 hours
backupTimeout: 12600 # 3.5 hours
# Incremental backoff configuration for the Qdrant operator operations
backOff:
  minDelay: 5
  maxDelay: 300
  increment: 5
# Cluster-manager configuration for a Qdrant cluster (experimental)
clusterManager:
  image:
    repository: qdrant/qdrant-cloud-cluster-manager
    tag: 0.1.2
  pullInterval: 10
  logSize: 10
  debug: false
#  node_selector: {}
#  tolerations: []
# Default ingress configuration for a Qdrant cluster
ingress:
  enabled: false
  provider: KubernetesIngress # or NginxIngress
#  kubernetesIngress:
#  ingressClassName: ""
# Default storage configuration for a Qdrant cluster
#storage:
# Default VolumeSnapshotClass for a Qdrant cluster
#  snapshot_class: "csi-snapclass"
# Default StorageClass for a Qdrant cluster, uses cluster default StorageClass if not set
#  default_storage_class_names:
# StorageClass for DB volumes
#    db: ""
# StorageClass for snapshot volumes
#    snapshots: ""
# Default scheduling configuration for a Qdrant cluster
#scheduling:
#  default_topology_spread_constraints: []
#  default_pod_disruption_budget: {}
qdrant:
# Default security context for Qdrant cluster
#  securityContext:
#    enabled: false
#    user: ""
#    fsGroup: ""
#    group: ""
# Default Qdrant image configuration
#  image:
#    pull_secret: ""
#    pull_policy: IfNotPresent
#    repository: qdrant/qdrant
# Default Qdrant log_level
#  log_level: INFO
# Default network policies to create for a qdrant cluster
  networkPolicies:
    ingress:
      - ports:
        - protocol: TCP
          port: 6333
        - protocol: TCP
          port: 6334
# Allow DNS resolution from qdrant pods at Kubernetes internal DNS server
    egress:
      - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
        ports:
        - protocol: UDP
          port: 53
```
qdrant-landing/content/documentation/interfaces/web-ui.md

---
title: Qdrant Web UI
weight: 1
aliases:
  - /documentation/web-ui/
---

# Qdrant Web UI

You can manage both local and cloud Qdrant deployments through the Web UI.

If you've set up a deployment locally with the Qdrant [Quickstart](/documentation/quick-start/),
navigate to http://localhost:6333/dashboard.

If you've set up a deployment in a cloud cluster, find your Cluster URL in your
cloud dashboard, at https://cloud.qdrant.io. Add `:6333/dashboard` to the end
of the URL. 

## Access the Web UI

Qdrant's Web UI is an intuitive and efficient graphic interface for your Qdrant Collections, REST API and data points.

In the **Console**, you may use the REST API to interact with Qdrant, while in **Collections**, you can manage all the collections and upload Snapshots. 

![Qdrant Web UI](/articles_data/qdrant-1.3.x/web-ui.png)

### Qdrant Web UI features

In the Qdrant Web UI, you can:

- Run HTTP-based calls from the console
- List and search existing [collections](/documentation/concepts/collections/)
- Learn from our interactive tutorial

You can navigate to these options directly. For example, if you used our 
[quick start](/documentation/quick-start/) to set up a cluster on localhost,
you can review our tutorial at http://localhost:6333/dashboard#/tutorial.