DataStore Factory Methods

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/chdb/datastore/factory-methods.md)

DataStore provides over 20 factory methods to create instances from various data sources including local files, databases, cloud storage, and data lakes.

## Universal URI Interface[​](#uri "Direct link to Universal URI Interface")

The `uri()` method is the recommended universal entry point that auto-detects the source type:

```
from chdb.datastore import DataStore

# Local files
ds = DataStore.uri("data.csv")
ds = DataStore.uri("/path/to/data.parquet")

# Cloud storage
ds = DataStore.uri("s3://bucket/data.parquet?nosign=true")
ds = DataStore.uri("https://example.com/data.csv")

# Databases
ds = DataStore.uri("mysql://user:pass@host:3306/db/table")
ds = DataStore.uri("postgresql://user:pass@host:5432/db/table")
```

### URI Syntax Reference[​](#uri-syntax "Direct link to URI Syntax Reference")

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Source Type URI Format Example|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Local file `path/to/file` `data.csv`, `/abs/path/data.parquet`| S3 `s3://bucket/path` `s3://mybucket/data.parquet?nosign=true`| GCS `gs://bucket/path` `gs://mybucket/data.csv`| Azure `az://container/path` `az://mycontainer/data.parquet`| HTTP/HTTPS `https://url` `https://example.com/data.csv`| MySQL `mysql://user:pass@host:port/db/table` `mysql://root:pass@localhost:3306/mydb/users`| PostgreSQL `postgresql://user:pass@host:port/db/table` `postgresql://postgres:pass@localhost:5432/mydb/users`| SQLite `sqlite:///path?table=name` `sqlite:///data.db?table=users`| ClickHouse `clickhouse://host:port/db/table` `clickhouse://localhost:9000/default/hits` | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

---

## File Sources[​](#file-sources "Direct link to File Sources")

### `from_file`[​](#from-file "Direct link to from-file")

Create DataStore from a local or remote file with automatic format detection.

```
DataStore.from_file(path, format=None, compression=None, **kwargs)
```

**Parameters:**

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Parameter Type Default Description|  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `path` str *required* File path (local or URL)|  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | | `format` str `None` File format (auto-detected if None)|  |  |  |  | | --- | --- | --- | --- | | `compression` str `None` Compression type (auto-detected if None) | | | | | | | | | | | | | | | |

**Supported formats:** CSV, TSV, Parquet, JSON, JSONLines, ORC, Avro, Arrow

**Examples:**

```
from chdb.datastore import DataStore

# Auto-detect format from extension
ds = DataStore.from_file("data.csv")
ds = DataStore.from_file("data.parquet")
ds = DataStore.from_file("data.json")

# Explicit format
ds = DataStore.from_file("data.txt", format="CSV")

# With compression
ds = DataStore.from_file("data.csv.gz", compression="gzip")
```

### Pandas-compatible read functions[​](#pandas-read "Direct link to Pandas-compatible read functions")

```
from chdb import datastore as pd

# CSV files
ds = pd.read_csv("data.csv")
ds = pd.read_csv("data.csv", sep=";", header=0, nrows=1000)

# Parquet files (recommended for large datasets)
ds = pd.read_parquet("data.parquet")
ds = pd.read_parquet("data.parquet", columns=['col1', 'col2'])

# JSON files
ds = pd.read_json("data.json")
ds = pd.read_json("data.jsonl", lines=True)

# Excel files
ds = pd.read_excel("data.xlsx", sheet_name="Sheet1")
```

---

## Cloud Storage[​](#cloud-storage "Direct link to Cloud Storage")

### `from_s3`[​](#from-s3 "Direct link to from-s3")

Create DataStore from Amazon S3.

```
DataStore.from_s3(url, access_key_id=None, secret_access_key=None, format=None, **kwargs)
```

**Parameters:**

|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Parameter Type Default Description|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `url` str *required* S3 URL (s3://bucket/path)|  |  |  |  |  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `access_key_id` str `None` AWS access key ID|  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | | `secret_access_key` str `None` AWS secret access key|  |  |  |  | | --- | --- | --- | --- | | `format` str `None` File format (auto-detected) | | | | | | | | | | | | | | | | | | | |

**Examples:**

```
from chdb.datastore import DataStore

# Anonymous access (public bucket)
ds = DataStore.from_s3("s3://bucket/data.parquet")

# With credentials
ds = DataStore.from_s3(
    "s3://bucket/data.parquet",
    access_key_id="AKIAIOSFODNN7EXAMPLE",
    secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
)

# Using URI with query parameters
ds = DataStore.uri("s3://bucket/data.parquet?nosign=true")
ds = DataStore.uri("s3://bucket/data.parquet?access_key_id=KEY&secret_access_key=SECRET")
```

### `from_gcs`[​](#from-gcs "Direct link to from-gcs")

Create DataStore from Google Cloud Storage.

```
DataStore.from_gcs(url, credentials_path=None, **kwargs)
```

**Examples:**

```
ds = DataStore.from_gcs("gs://bucket/data.parquet")
ds = DataStore.from_gcs("gs://bucket/data.parquet", credentials_path="/path/to/creds.json")
```

### `from_azure`[​](#from-azure "Direct link to from-azure")

Create DataStore from Azure Blob Storage.

```
DataStore.from_azure(url, account_name=None, account_key=None, **kwargs)
```

**Examples:**

```
ds = DataStore.from_azure(
    "az://container/data.parquet",
    account_name="myaccount",
    account_key="mykey"
)
```

### `from_hdfs`[​](#from-hdfs "Direct link to from-hdfs")

Create DataStore from HDFS.

```
DataStore.from_hdfs(url, **kwargs)
```

**Examples:**

```
ds = DataStore.from_hdfs("hdfs://namenode:8020/path/data.parquet")
```

### `from_url`[​](#from-url "Direct link to from-url")

Create DataStore from HTTP/HTTPS URL.

```
DataStore.from_url(url, format=None, **kwargs)
```

**Examples:**

```
ds = DataStore.from_url("https://example.com/data.csv")
ds = DataStore.from_url("https://raw.githubusercontent.com/user/repo/main/data.parquet")
```

---

## Databases[​](#databases "Direct link to Databases")

### `from_mysql`[​](#from-mysql "Direct link to from-mysql")

Create DataStore from MySQL database.

```
DataStore.from_mysql(host, database, table, user, password, port=3306, **kwargs)
```

**Parameters:**