title_wip

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/use-cases/AI_ML/vector-search.md)

In this guide, you will:

- Get introduced briefly to vector search
- Learn about Approximate Nearest Neighbours (ANN) and Hierarchical Navigable Small World (HNSW)
- Learn about Quantised Bit (QBit)
- Use QBit to perform vector search using the DBPedia dataset

## Vector search primer[​](#vector-search-primer "Direct link to Vector search primer")

In mathematics and physics, a vector is formally defined as an object that has both a magnitude and direction.
This often takes the form of a line segment or an arrow through space and can be used to represent quantities such as velocity, force and acceleration.
In computer science, a vector is a finite sequence of numbers.
In other words, it’s a data structure that is used to store numeric values.

In machine learning, vectors are the same data structures we talk about in computer science, but the numerical values stored in them have a special meaning.
When we take a block of text or an image, and distill it down to the key concepts that it represents, this process is called encoding.
The resulting output is a machine’s representation of those key concepts in numerical form.
This is an embedding, and is stored in a vector.
Said differently, when this contextual meaning is embedded in a vector, we can refer to it as an embedding.

Vector search is everywhere now.
It powers music recommendations, retrieval-augmented generation (RAG) for large language models where external knowledge is fetched to improve answers, and even googling is powered by vector search to some extent.

Users often prefer regular databases with ad-hoc vector capabilities over fully specialized vector stores, despite specialized databases' advantages.
ClickHouse supports [brute-force vector search](/docs/engines/table-engines/mergetree-family/annindexes#exact-nearest-neighbor-search) as well as [methods for approximate nearest neighbour (ANN) search](/docs/engines/table-engines/mergetree-family/annindexes#approximate-nearest-neighbor-search), including HNSW – the current standard for fast vector retrieval.

### Understanding embeddings[​](#understanding-embeddings "Direct link to Understanding embeddings")

Let's look at a simple example to understand how vector search works.
Consider embeddings (vector representations) of words:

Create the table below with some sample embeddings:

```
CREATE TABLE fruit_animal
ENGINE = MergeTree
ORDER BY word
AS SELECT *
FROM VALUES(
  'word String, vec Array(Float64)',
  ('apple', [-0.99105519, 1.28887844, -0.43526649, -0.98520696, 0.66154391]),
  ('banana', [-0.69372815, 0.25587061, -0.88226235, -2.54593015, 0.05300475]),
  ('orange', [0.93338752, 2.06571317, -0.54612565, -1.51625717, 0.69775337]),
  ('dog', [0.72138876, 1.55757105, 2.10953259, -0.33961248, -0.62217325]),
  ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533])
);
```

You can search for the most similar words to a given embedding:

```
SELECT word, L2Distance(
  vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770]
) AS distance
FROM fruit_animal
ORDER BY distance
LIMIT 5;
```

```
┌─word───┬────────────distance─┐
│ apple  │ 0.14639757188169716 │
│ banana │  1.9989613690076786 │
│ orange │   2.039041552613732 │
│ horse  │  2.7555776805484813 │
│ dog    │   3.382295083120104 │
└────────┴─────────────────────┘
```

The query embedding is closest to "apple" (smallest distance), which makes sense if we look at the two embeddings side by side:

```
apple:           [-0.99105519,1.28887844,-0.43526649,-0.98520696,0.66154391]
query embedding: [-0.88693672,1.31532824,-0.51182908,-0.99652702,0.5990777]
```

## Approximate Nearest Neighbours (ANN)[​](#approximate-nearest-neighbours "Direct link to Approximate Nearest Neighbours (ANN)")

For large datasets, brute-force search becomes too slow.
This is where Approximate Nearest Neighbours methods come in.

### Quantisation[​](#quantisation "Direct link to Quantisation")

Quantisation involves downcasting to smaller numeric types.
Smaller numbers mean smaller data, and smaller data means faster distance calculations.
ClickHouse's vectorized query execution engine can fit more values into processor registers per operation, increasing throughput directly.

You have two options:

1. **Keep the quantised copy alongside the original column** - This doubles storage, but it's safe as we can always fall back to full precision
2. **Replace the original values entirely** (by downcasting on insertion) - This saves space and I/O, but it's a one-way door

### Hierarchical Navigable Small World (HNSW)[​](#hnsw "Direct link to Hierarchical Navigable Small World (HNSW)")

HNSW is built from multiple layers of nodes (vectors). Each node is randomly assigned to one or more layers, with the chance of appearing in higher layers decreasing exponentially.

When performing a search, we start from a node at the top layer and move greedily towards the closest neighbours. Once no closer node can be found, we descend to the next, denser layer.

Because of this layered design, HNSW achieves logarithmic search complexity with respect to the number of nodes.

HNSW limitation

The main bottleneck is memory. ClickHouse uses the [usearch](https://github.com/unum-cloud/usearch) implementation of HNSW, which is an in-memory data structure that doesn't support splitting.
As a result, larger datasets require proportionally more RAM.

### Comparison of approaches[​](#comparison-approaches "Direct link to Comparison of approaches")

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Category Brute-force HNSW QBit|  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | **Precision** Perfect Great Flexible|  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | | **Speed** Slow Fast Flexible|  |  |  |  | | --- | --- | --- | --- | | **Others** Quantized: more space or irreversible precision Index has to fit in memory and has to be built Still O(#records) | | | | | | | | | | | | | | | |

## QBit deep dive[​](#qbit-deepdive "Direct link to QBit deep dive")

### Quantised Bit (QBit)[​](#quantised-bit "Direct link to Quantised Bit (QBit)")

QBit is a new data structure that can store `BFloat16`, `Float32`, and `Float64` values by taking advantage of how floating-point numbers are represented – as bits.
Instead of storing each number as a whole, QBit splits the values into **bit planes**: every first bit, every second bit, every third bit, and so on.

This approach solves the main limitation of traditional quantisation. There's no need to store duplicated data or risk making values meaningless. It also avoids the RAM bottlenecks of HNSW, since QBit works directly with the stored data rather than maintaining an in-memory index.

Benefit

**Most importantly, no upfront decisions are required.**
Precision and performance can be adjusted dynamically at query time, allowing users to explore the balance between accuracy and speed with minimal friction.

Limitation

Although QBit speeds up vector search, its computational complexity remains O(n). In other words: if your dataset is small enough for an HNSW index to fit comfortably in RAM, that is still the fastest choice.

### The data type[​](#the-data-type "Direct link to The data type")

Here's how to create a QBit column:

```
SET allow_experimental_qbit_type = 1;
CREATE TABLE fruit_animal
(
  word String,
  vec QBit(Float64, 5)
)
ENGINE = MergeTree
ORDER BY word;

INSERT INTO fruit_animal VALUES
('apple',  [-0.99105519, 1.28887844, -0.43526649, -0.98520696, 0.66154391]),
('banana', [-0.69372815, 0.25587061, -0.88226235, -2.54593015, 0.05300475]),
('orange', [0.93338752, 2.06571317, -0.54612565, -1.51625717, 0.69775337]),
('dog',    [0.72138876, 1.55757105, 2.10953259, -0.33961248, -0.62217325]),
('horse',  [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]);
```