title_wip

The current size of the vector similarity index cache is shown in [system.metrics](/docs/operations/system-tables/metrics):

```
SELECT metric, value
FROM system.metrics
WHERE metric = 'VectorSimilarityIndexCacheBytes'
```

The cache hits and misses for a query with some query id can be obtained from [system.query\_log](/docs/operations/system-tables/query_log):

```
SYSTEM FLUSH LOGS query_log;

SELECT ProfileEvents['VectorSimilarityIndexCacheHits'], ProfileEvents['VectorSimilarityIndexCacheMisses']
FROM system.query_log
WHERE type = 'QueryFinish' AND query_id = '<...>'
ORDER BY event_time_microseconds;
```

For production use-cases, we recommend that the cache is sized large enough so that all vector indexes remain in memory at all times.

**Tuning quantization**

[Quantization](https://huggingface.co/blog/embedding-quantization) is a technique to reduce the memory footprint of vectors and the computational costs of building and traversing vector indexes.
ClickHouse vector indexes supports the following quantization options:

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Quantization Name Storage per dimension|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | f32 Single precision 4 bytes|  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | f16 Half precision 2 bytes|  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | bf16 (default) Half precision (brain float) 2 bytes|  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | | i8 Quarter precision 1 byte|  |  |  | | --- | --- | --- | | b1 Binary 1 bit | | | | | | | | | | | | | | | | | |

Quantization reduces the precision of vector searches compared to searching the original full-precision floating-point values (`f32`).
However, on most datasets, half-precision brain float quantization (`bf16`) results in a negligible precision loss, therefore vector similarity indexes use this quantization technique by default.
Quarter precision (`i8`) and binary (`b1`) quantization causes appreciable precision loss in vector searches.
We recommend both quantizations only if the the size of the vector similarity index is significantly larger than the available DRAM size.
In this case, we also suggest enabling rescoring ([vector\_search\_index\_fetch\_multiplier](/docs/operations/settings/settings#vector_search_index_fetch_multiplier), [vector\_search\_with\_rescoring](/docs/operations/settings/settings#vector_search_with_rescoring)) to improve accuracy.
Binary quantization is only recommended for 1) normalized embeddings (i.e. vector length = 1, OpenAI models are usually normalized), and 2) if the cosine distance is used as distance function.
Binary quantization internally uses the Hamming distance to construct and search the proximity graph.
The rescoring step uses the original full-precision vectors stored in the table to identify the nearest neighbours via cosine distance.

**Tuning data transfer**

The reference vector in a vector search query is provided by the user and generally retrieved by making a call to a Large Language Model (LLM).
Typical Python code which runs a vector search in ClickHouse might look like this

```
search_v = openai_client.embeddings.create(input = "[Good Books]", model='text-embedding-3-large', dimensions=1536).data[0].embedding

params = {'search_v': search_v}
result = chclient.query(
   "SELECT id FROM items
    ORDER BY cosineDistance(vector, %(search_v)s)
    LIMIT 10",
    parameters = params)
```

Embedding vectors (`search_v` in above snippet) could have a very large dimension.
For example, OpenAI provides models that generate embeddings vectors with 1536 or even 3072 dimensions.
In above code, the ClickHouse Python driver substitutes the embedding vector by a human readable string and subsequently send the SELECT query entirely as a string.
Assuming the embedding vector consists of 1536 single-precision floating point values, the sent string reaches a length of 20 kB.
This creates a high CPU usage for tokenizing, parsing and performing thousands of string-to-float conversions.
Also, significant space is required in the ClickHouse server log file, causing bloat in `system.query_log` as well.

Note that most LLM models return an embedding vector as a list or NumPy array of native floats.
We therefore recommend Python applications to bind the reference vector parameter in binary form by using the following style:

```
search_v = openai_client.embeddings.create(input = "[Good Books]", model='text-embedding-3-large', dimensions=1536).data[0].embedding

params = {'$search_v_binary$': np.array(search_v, dtype=np.float32).tobytes()}
result = chclient.query(
   "SELECT id FROM items
    ORDER BY cosineDistance(vector, reinterpret($search_v_binary$, 'Array(Float32)'))
    LIMIT 10"
    parameters = params)
```

In the example, the reference vector is sent as-is in binary form and reinterpreted as array of floats on the server.
This saves CPU time on the server side, and avoids bloat in the server logs and `system.query_log`.

#### Administration and monitoring[​](#administration "Direct link to Administration and monitoring")

The on-disk size of vector similarity indexes can be obtained from [system.data\_skipping\_indices](/docs/operations/system-tables/data_skipping_indices):

```
SELECT database, table, name, formatReadableSize(data_compressed_bytes)
FROM system.data_skipping_indices
WHERE type = 'vector_similarity';
```

Example output:

```
┌─database─┬─table─┬─name─┬─formatReadab⋯ssed_bytes)─┐
│ default  │ tab   │ idx  │ 348.00 MB                │
└──────────┴───────┴──────┴──────────────────────────┘
```

#### Differences to regular skipping indexes[​](#differences-to-regular-skipping-indexes "Direct link to Differences to regular skipping indexes")

As all regular [skipping indexes](/docs/optimize/skipping-indexes), vector similarity indexes are constructed over granules and each indexed block consists of `GRANULARITY = [N]`-many granules (`[N]` = 1 by default for normal skipping indexes).
For example, if the primary index granularity of the table is 8192 (setting `index_granularity = 8192`) and `GRANULARITY = 2`, then each indexed block will contain 16384 rows.
However, data structures and algorithms for approximate neighbor search are inherently row-oriented.
They store a compact representation of a set of rows and also return rows for vector search queries.
This causes some rather unintuitive differences in the way vector similarity indexes behave compared to normal skipping indexes.

When a user defines a vector similarity index on a column, ClickHouse internally creates a vector similarity "sub-index" for each index block.
The sub-index is "local" in the sense that it only knows about the rows of its containing index block.
In the previous example and assuming that a column has 65536 rows, we obtain four index blocks (spanning eight granules) and a vector similarity sub-index for each index block.
A sub-index is theoretically able to return the rows with the N closest points within its index block directly.
However, since ClickHouse loads data from disk to memory at the granularity of granules, sub-indexes extrapolate matching rows to granule granularity.
This is different from regular skipping indexes which skip data at the granularity of index blocks.