Equivalent concepts in ClickStack and Elastic

### Data modeling and flexibility[​](#data-modeling-and-flexibility "Direct link to Data modeling and flexibility")

Elasticsearch is known for its schema flexibility through [dynamic mappings](https://www.elastic.co/docs/manage-data/data-store/mapping/dynamic-mapping). Fields are created as documents are ingested, and types are inferred automatically - unless a schema is specified. ClickHouse is stricter by default — tables are defined with explicit schemas — but offers flexibility through [`Dynamic`](/docs/sql-reference/data-types/dynamic), [`Variant`](/docs/sql-reference/data-types/variant), and [`JSON`](/docs/integrations/data-formats/json/overview) types. These enable ingestion of semi-structured data, with dynamic column creation and type inference similar to Elasticsearch. Similarly, the [`Map`](/docs/sql-reference/data-types/map) type allows arbitrary key-value pairs to be stored - although a single type is enforced for both the key and value.

ClickHouse's approach to type flexibility is more transparent and controlled. Unlike Elasticsearch, where type conflicts can cause ingestion errors, ClickHouse allows mixed-type data in [`Variant`](/docs/sql-reference/data-types/variant) columns and supports schema evolution through the use of the [`JSON`](/docs/integrations/data-formats/json/overview) type.

If not using [`JSON`](/docs/integrations/data-formats/json/overview), the schema is statically-defined. If values aren't provided for a row, they will either be defined as [`Nullable`](/docs/sql-reference/data-types/nullable) (not used in ClickStack) or revert to the default value for the type e.g. empty value for `String`.

### Ingestion and transformation[​](#ingestion-and-transformation "Direct link to Ingestion and transformation")

Elasticsearch uses ingest pipelines with processors (e.g., `enrich`, `rename`, `grok`) to transform documents before indexing. In ClickHouse, similar functionality is achieved using [**incremental materialized views**](/docs/materialized-view/incremental-materialized-view), which can [filter, transform](/docs/materialized-view/incremental-materialized-view#filtering-and-transformation), or [enrich](/docs/materialized-view/incremental-materialized-view#lookup-table) incoming data and insert results into target tables. You can also insert data to a `Null` table engine if you only need the output of the materialized view to be stored. This means that only the results of any materialized views are preserved, but the original data is discarded - thus saving storage space.

For enrichment, Elasticsearch supports dedicated [enrich processors](https://www.elastic.co/docs/reference/enrich-processor/enrich-processor) to add context to documents. In ClickHouse, [**dictionaries**](/docs/dictionary) can be used at both [query time](/docs/dictionary#query-time-enrichment) and [ingest time](/docs/dictionary#index-time-enrichment) to enrich rows - for example, to [map IPs to locations](/docs/use-cases/observability/schema-design#using-ip-dictionaries) or apply [user agent lookups](/docs/use-cases/observability/schema-design#using-regex-dictionaries-user-agent-parsing) on insert.

### Query languages[​](#query-languages "Direct link to Query languages")

Elasticsearch supports a [number of query languages](https://www.elastic.co/docs/explore-analyze/query-filter/languages) including [DSL](https://www.elastic.co/docs/explore-analyze/query-filter/languages/querydsl), [ES|QL](https://www.elastic.co/docs/explore-analyze/query-filter/languages/esql), [EQL](https://www.elastic.co/docs/explore-analyze/query-filter/languages/eql) and [KQL](https://www.elastic.co/docs/explore-analyze/query-filter/languages/kql) (Lucene style) queries, but has limited support for joins — only **left outer joins** are available via [`ES|QL`](https://www.elastic.co/guide/en/elasticsearch/reference/8.x/esql-commands.html#esql-lookup-join). ClickHouse supports **full SQL syntax**, including [all join types](/docs/sql-reference/statements/select/join#supported-types-of-join), [window functions](/docs/sql-reference/window-functions), subqueries (and correlated), and CTEs. This is a major advantage if you need to correlate between observability signals and business or infrastructure data.

In ClickStack, [the UI provides a Lucene-compatible search interface](/docs/use-cases/observability/clickstack/search) for ease of transition, alongside full SQL support via the ClickHouse backend. This syntax is comparable to the [Elastic query string](https://www.elastic.co/docs/reference/query-languages/query-dsl/query-dsl-query-string-query#query-string-syntax) syntax. For an exact comparison of this syntax, see ["Searching in ClickStack and Elastic"](/docs/use-cases/observability/clickstack/migration/elastic/search).

### File formats and interfaces[​](#file-formats-and-interfaces "Direct link to File formats and interfaces")

Elasticsearch supports JSON (and [limited CSV](https://www.elastic.co/docs/reference/enrich-processor/csv-processor)) ingestion. ClickHouse supports over **70 file formats**, including Parquet, Protobuf, Arrow, CSV, and others — for both ingestion and export. This makes it easier to integrate with external pipelines and tools.

Both systems offer a REST API, but ClickHouse also provides a **native protocol** for low-latency, high-throughput interaction. The native interface supports query progress, compression, and streaming more efficiently than HTTP, and is the default for most production ingestion.

### Indexing and storage[​](#indexing-and-storage "Direct link to Indexing and storage")

The concept of sharding is fundamental to Elasticsearch's scalability model. Each ① [**index**](https://www.elastic.co/blog/what-is-an-elasticsearch-index) is broken into **shards**, each of which is a physical Lucene index stored as segments on disk. A shard can have one or more physical copies called replica shards for resilience. For scalability, shards and replicas can be distributed over several nodes. A single shard ② consists of one or more immutable segments. A segment is the basic indexing structure of Lucene, the Java library providing the indexing and search features on which Elasticsearch is based.

Insert processing in Elasticsearch

Ⓐ Newly inserted documents Ⓑ first go into an in-memory indexing buffer that is flushed by default once per second. A routing formula is used to determine the target shard for flushed documents, and a new segment is written for the shard on disk. To improve query efficiency and enable the physical deletion of deleted or updated documents, segments are continuously merged in the background into larger segments until they reach a max size of 5 GB. It is, however, possible to force a merge into larger segments.

Elasticsearch recommends sizing shards to around [50 GB or 200 million documents](https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/size-shards) due to [JVM heap and metadata overhead](https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/size-shards#each-shard-has-overhead). There's also a hard limit of [2 billion documents per shard](https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/size-shards#troubleshooting-max-docs-limit). Elasticsearch parallelizes queries across shards, but each shard is processed using a **single thread**, making over-sharding both costly and counterproductive. This inherently tightly couples sharding to scaling, with more shards (and nodes) required to scale performance.

Elasticsearch indexes all fields into [**inverted indices**](https://www.elastic.co/docs/manage-data/data-store/index-basics) for fast search, optionally using [**doc values**](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/doc-values) for aggregations, sorting and scripted field access. Numeric and geo fields use [Block K-D trees](https://users.cs.duke.edu/~pankaj/publications/papers/bkd-sstd.pdf) for searches on geospatial data and numeric and date ranges.