DataStore Quickstart

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/chdb/datastore/quickstart.md)

Get up and running with DataStore in minutes. This guide covers installation, migrating from pandas, and basic usage patterns.

## Installation[​](#installation "Direct link to Installation")

Install chDB with pip:

```
pip install "chdb>=4.0"
```

For optional dependencies:

```
# For pandas DataFrame support
pip install "chdb[pandas]>=4.0"

# For PyArrow support
pip install "chdb[arrow]>=4.0"

# All optional dependencies
pip install "chdb[all]>=4.0"
```

### Verify Installation[​](#verify "Direct link to Verify Installation")

```
import chdb
print(chdb.__version__)  # Should print 4.x.x or higher

from chdb import datastore as pd
print("DataStore ready!")
```

## One-Line Migration from Pandas[​](#migration "Direct link to One-Line Migration from Pandas")

The simplest way to start using DataStore is to change your import statement:

```
# Before (pandas)
import pandas as pd

# After (DataStore)
from chdb import datastore as pd
```

That's it! Your existing pandas code will now use DataStore and benefit from SQL optimization.

### Migration Example[​](#migration-example "Direct link to Migration Example")

```
from pathlib import Path
Path("employees.csv").write_text("""\
name,age,city,salary,department,dept_id,status,email
Alice,28,NYC,75000,Engineering,1,active,[email protected]
Bob,35,LA,85000,Engineering,1,active,[email protected]
Charlie,52,NYC,95000,Product,2,active,[email protected]
Diana,32,SF,70000,Design,3,active,[email protected]
Eve,23,LA,48000,Product,2,inactive,[email protected]
""")

# Original pandas code
import pandas as pd

df = pd.read_csv("employees.csv")
result = (df[df['salary'] > 50000]
          .groupby('department')['salary']
          .agg(['mean', 'count'])
          .sort_values('mean', ascending=False))
print(result)

# DataStore version - just change the import!
from chdb import datastore as pd

df = pd.read_csv("employees.csv")
result = (df[df['salary'] > 50000]
          .groupby('department')['salary']
          .agg(['mean', 'count'])
          .sort_values('mean', ascending=False))
print(result)  # Same result, faster execution!
```

## Basic Usage[​](#basic-usage "Direct link to Basic Usage")

### Creating a DataStore[​](#creating "Direct link to Creating a DataStore")

```
from chdb import datastore as pd

# From a dictionary
ds = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [25, 30, 35],
    'city': ['NYC', 'LA', 'NYC']
})

# From a pandas DataFrame
import pandas
pdf = pandas.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})
ds = pd.DataFrame(pdf)

# From a CSV file
ds = pd.read_csv("data.csv")

# From a Parquet file (recommended for large datasets)
ds = pd.read_parquet("data.parquet")
```

### Filtering Data[​](#filtering "Direct link to Filtering Data")

```
from chdb import datastore as pd

ds = pd.read_csv("employees.csv")

# Single condition
senior = ds[ds['age'] > 30]

# Multiple conditions (AND)
senior_nyc = ds[(ds['age'] > 30) & (ds['city'] == 'NYC')]

# Multiple conditions (OR)
young_or_senior = ds[(ds['age'] < 25) | (ds['age'] > 50)]

# Using filter method (SQL-style)
result = ds.filter(ds['salary'] > 50000)
```

### Selecting Columns[​](#selecting "Direct link to Selecting Columns")

```
# Pandas style
subset = ds[['name', 'age']]

# SQL style
subset = ds.select('name', 'age')
```

### Sorting[​](#sorting "Direct link to Sorting")

```
# Pandas style
sorted_ds = ds.sort_values('salary', ascending=False)

# SQL style
sorted_ds = ds.sort('salary', ascending=False)
```

### Grouping and Aggregation[​](#groupby "Direct link to Grouping and Aggregation")

```
from pathlib import Path
Path("sales.csv").write_text("""\
region,product,category,amount,quantity,price,date,order_id
East,Widget,Electronics,5200,10,120,2024-01-15,1001
West,Gadget,Electronics,800,5,160,2024-02-20,1002
East,Gizmo,Home,6500,3,100,2024-03-10,1003
North,Widget,Electronics,4500,6,150,2024-06-18,1004
West,Gadget,Electronics,2000,8,250,2024-09-14,1005
""")

from chdb import datastore as pd

ds = pd.read_csv("sales.csv")

# Group by single column
by_region = ds.groupby('region')['amount'].sum()

# Group by multiple columns
by_region_product = ds.groupby(['region', 'product']).agg({
    'amount': ['sum', 'mean'],
    'quantity': 'sum'
})

# Multiple aggregations
summary = ds.groupby('category').agg({
    'price': ['min', 'max', 'mean'],
    'quantity': 'sum'
})
```

### Joining DataStores[​](#joining "Direct link to Joining DataStores")

```
from pathlib import Path
Path("departments.csv").write_text("""\
dept_id,department_name
1,Engineering
2,Product
3,Design
""")

from chdb import datastore as pd

employees = pd.read_csv("employees.csv")
departments = pd.read_csv("departments.csv")

# Inner join
result = employees.join(departments, on='dept_id', how='inner')

# Left join
result = employees.join(departments, on='dept_id', how='left')

# Using merge (pandas style)
result = pd.merge(employees, departments, on='dept_id')
```

## Getting Results[​](#results "Direct link to Getting Results")

DataStore uses lazy evaluation - operations are not executed until you need the results.

### Triggering Execution[​](#execution-triggers "Direct link to Triggering Execution")

```
# Automatic triggers
print(ds)           # Displaying results
len(ds)             # Getting row count
ds.columns          # Accessing properties
list(ds)            # Converting to list

# Explicit conversion
df = ds.to_df()     # Convert to pandas DataFrame
df = ds.to_pandas() # Same as to_df()
```

### Viewing Generated SQL[​](#view-sql "Direct link to Viewing Generated SQL")

```
# See what SQL DataStore will execute
query = ds.filter(ds['age'] > 25).groupby('city').agg({'salary': 'mean'})
print(query.to_sql())
```

Output:

```
SELECT city, AVG(salary) AS mean
FROM file('data.csv', 'CSVWithNames')
WHERE age > 25
GROUP BY city
```

## Working with Different Data Sources[​](#data-sources "Direct link to Working with Different Data Sources")

### Local Files[​](#local-files "Direct link to Local Files")

```
from chdb import datastore as pd

# CSV
ds = pd.read_csv("data.csv")

# Parquet (best performance)
ds = pd.read_parquet("data.parquet")

# JSON
ds = pd.read_json("data.json")
```

### Cloud Storage[​](#cloud-storage "Direct link to Cloud Storage")

```
from chdb.datastore import DataStore

# S3 (anonymous)
ds = DataStore.uri("s3://bucket/data.parquet?nosign=true")

# S3 (with credentials)
ds = DataStore.from_s3(
    "s3://bucket/data.parquet",
    access_key_id="KEY",
    secret_access_key="SECRET"
)

# HTTP/HTTPS
ds = DataStore.uri("https://example.com/data.csv")
```

### Databases[​](#databases "Direct link to Databases")

```
from chdb.datastore import DataStore

# MySQL
ds = DataStore.from_mysql(
    host="localhost",
    database="mydb",
    table="users",
    user="root",
    password="pass"
)

# PostgreSQL
ds = DataStore.from_postgresql(
    host="localhost",
    database="mydb",
    table="users",
    user="postgres",
    password="pass"
)

# Using URI
ds = DataStore.uri("mysql://user:pass@localhost:3306/mydb/users")
```

## String and DateTime Operations[​](#accessors "Direct link to String and DateTime Operations")

### String Operations[​](#string-ops "Direct link to String Operations")

```
# All pandas .str methods work
ds['name_upper'] = ds['name'].str.upper()
ds['name_len'] = ds['name'].str.len()
ds['has_a'] = ds['name'].str.contains('a')
```

### DateTime Operations[​](#datetime-ops "Direct link to DateTime Operations")

```
# All pandas .dt methods work
ds['year'] = ds['date'].dt.year
ds['month'] = ds['date'].dt.month
ds['day_of_week'] = ds['date'].dt.dayofweek
```

### ClickHouse Extensions[​](#extensions "Direct link to ClickHouse Extensions")

```
# URL parsing (not available in pandas!)
ds['domain'] = ds['url'].url.domain()

# JSON extraction
ds['user_name'] = ds['json_data'].json.get_string('name')

# IP address operations
ds['is_ipv4'] = ds['ip_addr'].ip.is_ipv4_string()
```

## Best Practices[​](#best-practices "Direct link to Best Practices")