title_wip

### 4. Compare Engines[​](#compare-engines "Direct link to 4. Compare Engines")

```
from chdb.datastore.config import config

# Profile with chdb
config.use_chdb()
profiler.clear()
result_chdb = query.to_df()
time_chdb = profiler.total_duration_ms

# Profile with pandas
config.use_pandas()
profiler.clear()
result_pandas = query.to_df()
time_pandas = profiler.total_duration_ms

print(f"chdb: {time_chdb:.2f}ms")
print(f"pandas: {time_pandas:.2f}ms")
```

---

## Best Practices[​](#best-practices "Direct link to Best Practices")

### 1. Profile Before Optimizing[​](#best-practice-1 "Direct link to 1. Profile Before Optimizing")

```
# Don't guess - measure!
config.enable_profiling()
result = your_query.to_df()
print(get_profiler().report())
```

### 2. Clear Between Tests[​](#best-practice-2 "Direct link to 2. Clear Between Tests")

```
profiler.clear()  # Clear previous data
# Run test
print(profiler.report())
```

### 3. Use min\_duration\_ms for Focus[​](#best-practice-3 "Direct link to 3. Use min_duration_ms for Focus")

```
# Only show operations >= 100ms
profiler.report(min_duration_ms=100)
```

### 4. Profile Representative Data[​](#best-practice-4 "Direct link to 4. Profile Representative Data")

```
# Profile with real-world data sizes
# Small test data may not show real bottlenecks
```

### 5. Disable in Production[​](#best-practice-5 "Direct link to 5. Disable in Production")

```
# Development
config.enable_profiling()

# Production
config.set_profiling_enabled(False)  # Avoid overhead
```

---

## Example: Full Profiling Session[​](#example "Direct link to Example: Full Profiling Session")

```
from chdb import datastore as pd
from chdb.datastore.config import config, get_profiler

# Setup
config.enable_profiling()
config.enable_debug()  # Also see what's happening
profiler = get_profiler()

# Load data
profiler.clear()
print("=== Loading Data ===")
ds = pd.read_csv("sales_2024.csv")  # 10M rows
print(profiler.report())

# Query 1: Simple filter
profiler.clear()
print("\n=== Query 1: Simple Filter ===")
result1 = ds.filter(ds['amount'] > 1000).to_df()
print(profiler.report())

# Query 2: Complex aggregation
profiler.clear()
print("\n=== Query 2: Complex Aggregation ===")
result2 = (ds
    .filter(ds['amount'] > 100)
    .groupby('region', 'category')
    .agg({
        'amount': ['sum', 'mean', 'count'],
        'quantity': 'sum'
    })
    .sort('sum', ascending=False)
    .head(20)
    .to_df()
)
print(profiler.report())

# Summary
print("\n=== Summary ===")
print(f"Query 1: {len(result1)} rows")
print(f"Query 2: {len(result2)} rows")
```

- [Quick Start](#quick-start)- [Enabling Profiling](#enabling)- [Profiler API](#api)
      - [Getting the Profiler](#get-profiler)- [report()](#report)- [step()](#step)- [clear()](#clear)- [summary()](#summary)- [Understanding the Report](#understanding)
        - [Step Names](#step-names)- [Duration](#duration)- [Identifying Bottlenecks](#bottlenecks)- [Profiling Patterns](#patterns)
          - [Profile a Single Query](#single-query)- [Profile Multiple Queries](#multiple-queries)- [Compare Approaches](#compare)- [Optimization Tips](#optimization)
            - [1. Check SQL Execution Time](#check-sql)- [2. Check I/O Time](#check-io)- [3. Check Data Transfer](#check-transfer)- [4. Compare Engines](#compare-engines)- [Best Practices](#best-practices)
              - [1. Profile Before Optimizing](#best-practice-1)- [2. Clear Between Tests](#best-practice-2)- [3. Use min\_duration\_ms for Focus](#best-practice-3)- [4. Profile Representative Data](#best-practice-4)- [5. Disable in Production](#best-practice-5)- [Example: Full Profiling Session](#example)

Was this page helpful?