iflow-mcp_disler-quick-data-mcp 0.1.0
Add your description here
Requires Python: >=3.11

# Generic Data Analytics MCP Server

A MCP (Model Context Protocol) server that transforms any structured dataset (JSON/CSV) into intelligent, AI-guided analytics workflows. This server demonstrates advanced modular architecture with **dataset-agnostic design** - it automatically adapts to ANY data without hardcoded schemas.

## 🚀 Quick Setup

1. **Configure for your MCP client**:
   ```bash
   cp .mcp.json.sample .mcp.json
   # Edit .mcp.json and update paths to your system
   ```

2. **Find your UV path and update configuration**:
   ```bash
   which uv
   # Example output: /Users/yourusername/.local/bin/uv
   
   pwd  
   # Example output: /Users/yourusername/path/to/quick-data-mcp
   ```

3. **Test the server**:
   ```bash
   uv run python main.py
   ```

## 🚀 Getting Started in Claude Code

Once your MCP server is configured and running, **start with this slash command in Claude Code to get oriented**:

```
/quick-data:list_mcp_assets_prompt
```

This will show you all available tools, resources, and prompts with descriptions - your complete toolkit for data analytics!

## 🚀 What Makes This Special

### Universal Data Analytics
- **Works with ANY JSON/CSV dataset** - no schema definition required
- **Automatic column type detection** - numerical, categorical, temporal, identifier
- **AI-powered analysis suggestions** - recommends analyses based on your data characteristics
- **Adaptive conversation prompts** - guides users through analytics workflows using actual column names

### Tested Architecture
- **32 Analytics Tools** (20 analytics + 12 resource mirrors) for comprehensive data analysis
- **12 Dynamic Resources** providing real-time data context  
- **7 Adaptive Prompts** for AI-guided exploration
- **100% Test Coverage** (103 tests passing)
- **Universal MCP Client Compatibility** (supports tool-only clients)
- **Memory optimization** with usage monitoring

## 📊 Complete Capabilities

### 🔧 Analytics Tools (32 total)

#### **Data Loading & Management**
- `load_dataset(file_path, dataset_name, sample_size?)` - Load any JSON/CSV with automatic schema discovery
- `list_loaded_datasets()` - Show all datasets currently in memory with statistics
- `clear_dataset(dataset_name)` - Remove specific dataset from memory
- `clear_all_datasets()` - Clear all datasets from memory
- `get_dataset_info(dataset_name)` - Get comprehensive dataset information

#### **Core Analytics**
- `segment_by_column(dataset_name, column_name, method?, top_n?)` - Generic segmentation on any categorical column
- `find_correlations(dataset_name, columns?, threshold?)` - Correlation analysis with configurable thresholds
- `analyze_distributions(dataset_name, column_name)` - Statistical distribution analysis for any column
- `detect_outliers(dataset_name, columns?, method)` - Outlier detection (IQR, Z-score methods)
- `time_series_analysis(dataset_name, date_column, value_column, frequency?)` - Temporal analysis with trend detection

#### **Advanced Analytics**
- `validate_data_quality(dataset_name)` - Comprehensive data quality assessment (0-100 scoring)
- `compare_datasets(dataset_a, dataset_b, common_columns?)` - Multi-dataset comparison analysis
- `merge_datasets(dataset_configs, join_strategy?)` - Join datasets with flexible strategies
- `calculate_feature_importance(dataset_name, target_column, feature_columns?)` - ML feature importance
- `memory_optimization_report(dataset_name)` - Performance analysis and optimization suggestions

#### **Visualization & Export**
- `create_chart(dataset_name, chart_type, x_column, y_column?, groupby_column?, title?, save_path?)` - Generate charts (bar, scatter, histogram, line, box)
- `generate_dashboard(dataset_name, chart_configs)` - Multi-chart interactive dashboards
- `export_insights(dataset_name, format?, include_charts?)` - Export in JSON, CSV, HTML formats

#### **AI-Powered Assistance**
- `suggest_analysis(dataset_name)` - AI recommendations based on data characteristics
- `execute_custom_analytics_code(dataset_name, python_code)` - Execute custom Python code against datasets with full pandas/numpy/plotly support

#### **🔄 Resource Mirror Tools** (Tool-Only Client Support)
*For MCP clients that don't support resources, all resource functionality is available through mirror tools:*

**Dataset Context Tools (4)**
- `resource_datasets_loaded()` - List all loaded datasets (mirrors `datasets://loaded`)
- `resource_datasets_schema(dataset_name)` - Get dataset schema (mirrors `datasets://{name}/schema`)
- `resource_datasets_summary(dataset_name)` - Statistical summary (mirrors `datasets://{name}/summary`)
- `resource_datasets_sample(dataset_name)` - Sample data rows (mirrors `datasets://{name}/sample`)

**Analytics Intelligence Tools (5)**
- `resource_analytics_current_dataset()` - Currently active dataset (mirrors `analytics://current_dataset`)
- `resource_analytics_available_analyses()` - Applicable analysis types (mirrors `analytics://available_analyses`)
- `resource_analytics_column_types()` - Column classifications (mirrors `analytics://column_types`)
- `resource_analytics_suggested_insights()` - AI recommendations (mirrors `analytics://suggested_insights`)
- `resource_analytics_memory_usage()` - Memory monitoring (mirrors `analytics://memory_usage`)

**System Tools (3)**
- `resource_config_server()` - Server configuration (mirrors `config://server`)
- `resource_users_profile(user_id)` - User profile access (mirrors `users://{user_id}/profile`)
- `resource_system_status()` - System health info (mirrors `system://status`)

### 📚 Dynamic Resources (12 total)

#### **Dataset Context Resources**
- `datasets://loaded` - Real-time inventory of all loaded datasets
- `datasets://{dataset_name}/schema` - Dynamic schema with column classification
- `datasets://{dataset_name}/summary` - Statistical summary (pandas.describe() equivalent)
- `datasets://{dataset_name}/sample` - Sample rows for data preview

#### **Analytics Intelligence Resources**
- `analytics://current_dataset` - Currently active dataset context
- `analytics://available_analyses` - Applicable analysis types for current data
- `analytics://column_types` - Column role classification (numerical, categorical, temporal, identifier)
- `analytics://suggested_insights` - AI-generated analysis recommendations
- `analytics://memory_usage` - Real-time memory monitoring

#### **System Resources** (Legacy Compatibility)
- `config://server` - Server configuration information
- `users://{user_id}/profile` - User profile access by ID
- `system://status` - System health and status information

### 💬 Adaptive Prompts (7 total)

#### **Data Exploration Prompts**
- `dataset_first_look(dataset_name)` - Personalized initial exploration guide based on actual data structure
- `segmentation_workshop(dataset_name)` - Interactive segmentation strategy using real column names
- `data_quality_assessment(dataset_name)` - Systematic quality review with specific recommendations

#### **Analysis Workflow Prompts**
- `correlation_investigation(dataset_name)` - Guided correlation analysis workflow
- `pattern_discovery_session(dataset_name)` - Open-ended pattern mining conversation

#### **Business Intelligence Prompts**
- `insight_generation_workshop(dataset_name, business_context?)` - Business insight generation with domain context
- `dashboard_design_consultation(dataset_name, audience?)` - Audience-specific dashboard planning

## 🏗️ Project Structure

```
quick-data-mcp/
├── .mcp.json                      # Ready-to-use MCP client configuration
├── data/                          # Sample datasets
│   ├── ecommerce_orders.json      # E-commerce transaction data
│   ├── employee_survey.csv        # HR analytics dataset
│   ├── product_performance.csv    # Product metrics dataset
│   └── README.md                  # Data documentation
├── src/mcp_server/               # Core server implementation
│   ├── server.py                 # Main server with 31 tools, 12 resources, 7 prompts
│   ├── tools/                    # Tool implementations
│   │   ├── pandas_tools.py       # Pandas-based tools grouped module
│   │   ├── __init__.py           # All tools (32 total)
│   │   └── [individual_tool_files.py]  # Individual tool implementations
│   ├── resources/                # Resource handlers
│   │   └── data_resources.py     # Dynamic data access (12 resources)
│   ├── prompts/                  # Conversation starters
│   │   ├── __init__.py           # All prompts (9 total)
│   │   └── [individual_prompt_files.py]  # Individual prompt implementations
│   ├── models/                   # Data models and schemas
│   │   └── schemas.py            # DatasetManager, ColumnInfo, DatasetSchema
│   └── config/                   # Configuration
│       └── settings.py           # Server settings
├── tests/                            # Comprehensive test suite (130 tests)
│   ├── test_pandas_tools.py              # Pandas tools tests
│   ├── test_analytics_tools.py           # Advanced tools tests
│   ├── test_analytics_prompts.py         # Prompts functionality tests
│   ├── test_data_resources.py            # Resource access tests
│   ├── test_resource_mirror_tools.py     # Resource mirror tool tests
│   └── test_custom_analytics_code.py     # Custom code execution tests
├── outputs/                      # Generated files (excluded from git)
│   ├── charts/                   # Generated HTML charts and dashboards
│   └── reports/                  # Exported insights and reports
└── main.py                       # Entry point
```

## 📦 Dependencies

### Core Analytics Stack
- `mcp[cli]>=1.9.2` - Official MCP Python SDK
- `pandas>=2.2.3` - Data manipulation and analysis
- `plotly>=6.1.2` - Interactive visualizations

### Testing & Development
- `pytest>=8.3.5` - Testing framework
- `pytest-asyncio>=1.0.0` - Async testing support

## 🚀 Usage

### MCP Client Integration

Once configured, your MCP client can access all **32 tools**, **12 resources**, and **9 prompts** for comprehensive data analytics.

### Example Analytics Workflow

```python
# 1. Load any dataset
await load_dataset("data/ecommerce_orders.json", "sales")

# 2. Get AI-powered first look guidance
await dataset_first_look("sales")
# → Returns personalized exploration guide with actual column names

# 3. Automatic analysis suggestions
await suggest_analysis("sales")
# → AI recommends: correlation_analysis, segmentation_analysis based on detected columns

# 4. Perform suggested analyses
await find_correlations("sales")
# → Finds relationships between numerical columns

await segment_by_column("sales", "customer_segment")
# → Groups data and calculates statistics automatically

# 5. Create adaptive visualizations
await create_chart("sales", "bar", "region", "order_value")
# → Generates interactive plotly charts

# 6. Comprehensive data quality assessment
await validate_data_quality("sales")
# → Returns 0-100 quality score with detailed recommendations
```

### Advanced Multi-Dataset Analysis

```python
# Load multiple datasets
await load_dataset("data/employee_survey.csv", "hr")
await load_dataset("data/product_performance.csv", "products")

# Compare datasets
await compare_datasets("sales", "products", ["category"])

# Generate business insights
await insight_generation_workshop("sales", "e-commerce")

# Create executive dashboard
await dashboard_design_consultation("hr", "executive")
```

### 🔥 Custom Analytics Code Execution

Execute any Python code against your datasets with full pandas/numpy/plotly support:

```python
# Custom analysis that goes beyond predefined tools
output = await execute_custom_analytics_code("sales", """
print("=== Custom Customer Segmentation ===")

# Advanced customer scoring algorithm
customer_scores = df.groupby('customer_id').agg({
    'order_value': ['sum', 'mean', 'count'],
    'date': ['min', 'max']
}).round(2)

# Flatten column names
customer_scores.columns = ['total_spent', 'avg_order', 'order_count', 'first_order', 'last_order']

# Calculate customer lifetime (days)
customer_scores['lifetime_days'] = (
    pd.to_datetime(customer_scores['last_order']) - 
    pd.to_datetime(customer_scores['first_order'])
).dt.days

# Custom scoring formula
customer_scores['loyalty_score'] = (
    customer_scores['total_spent'] * 0.4 + 
    customer_scores['order_count'] * 50 + 
    customer_scores['lifetime_days'] * 0.1
).round(1)

# Segment customers
def segment_customer(score):
    if score >= 1000: return 'VIP'
    elif score >= 500: return 'Gold'
    elif score >= 200: return 'Silver'
    else: return 'Bronze'

customer_scores['segment'] = customer_scores['loyalty_score'].apply(segment_customer)

print("Customer Segments:")
print(customer_scores['segment'].value_counts())

print("\\nTop 5 Customers:")
top_customers = customer_scores.sort_values('loyalty_score', ascending=False).head()
for idx, (customer_id, data) in enumerate(top_customers.iterrows(), 1):
    print(f"{idx}. {customer_id}: {data['segment']} (Score: {data['loyalty_score']})")
""")

# Agents can iterate on code based on output
if "ERROR:" in output:
    # Fix the code and try again
    pass
else:
    print("Analysis completed successfully!")
```

### 🔄 Resource Mirror Tools Usage (Tool-Only Clients)

For MCP clients that don't support resources, use the resource mirror tools for identical functionality:

```python
# Instead of accessing resource: datasets://loaded
datasets = await resource_datasets_loaded()
# → Returns: {"datasets": [...], "total_datasets": 2, "status": "loaded"}

# Instead of accessing resource: datasets://sales/schema  
schema = await resource_datasets_schema("sales")
# → Returns: {"dataset_name": "sales", "columns_by_type": {...}}

# Instead of accessing resource: analytics://memory_usage
memory = await resource_analytics_memory_usage()
# → Returns: {"datasets": [...], "total_memory_mb": 15.2}

# Instead of accessing resource: config://server
config = await resource_config_server()
# → Returns: {"name": "Generic Data Analytics MCP", "features": [...]}

# All 12 resource mirror tools provide identical data to their resource counterparts
# Perfect for tool-only MCP clients or when resource support is unavailable
```

## 🧪 Testing

```bash
# Run all 130 tests
uv run python -m pytest tests/ -v

# Test specific functionality
uv run python -m pytest tests/test_pandas_tools.py -v              # Pandas tools
uv run python -m pytest tests/test_analytics_tools.py -v           # Advanced tools
uv run python -m pytest tests/test_analytics_prompts.py -v         # Prompts functionality
uv run python -m pytest tests/test_resource_mirror_tools.py -v     # Resource mirror tools
uv run python -m pytest tests/test_custom_analytics_code.py -v     # Custom code execution

# Quick test run
uv run python -m pytest tests/ -q
# Expected: 130 passed
```

## 🔧 MCP Client Configuration

### Quick Setup (Recommended)

This project includes a sample configuration that you can customize:

1. **Copy the sample configuration**:
   ```bash
   cp .mcp.json.sample .mcp.json
   ```

2. **Update paths in `.mcp.json`** to match your system:
   ```json
   {
     "mcpServers": {
       "quick-data": {
         "command": "/path/to/uv",
         "args": [
           "--directory",
           "/path/to/your/quick-data-mcp",
           "run",
           "python",
           "main.py"
         ],
         "env": {
           "LOG_LEVEL": "INFO"
         }
       }
     }
   }
   ```

3. **Find your UV path**:
   ```bash
   which uv
   # Example output: /Users/yourusername/.local/bin/uv
   ```

4. **Get absolute path to this directory**:
   ```bash
   pwd
   # Example output: /Users/yourusername/path/to/quick-data-mcp
   ```

5. **Update `.mcp.json`** with your actual paths:
   - Replace `/path/to/uv` with your UV path
   - Replace `/path/to/your/quick-data-mcp` with your absolute directory path

6. **Copy to your MCP client** or reference directly if supported

### Option 2: Manual Configuration

If you prefer to configure manually, add to your MCP client configuration:

```json
{
  "mcpServers": {
    "quick-data": {
      "command": "/path/to/uv",
      "args": [
        "--directory", 
        "/absolute/path/to/quick-data-mcp",
        "run", 
        "python", 
        "main.py"
      ],
      "env": {
        "LOG_LEVEL": "INFO"
      }
    }
  }
}
```

**Important**: Replace the placeholder paths with your actual system paths.

### Configuration Notes

- **Use absolute paths** for reliability across different working directories
- **`--directory` flag** ensures UV operates in the correct project directory  
- **`.mcp.json` is gitignored** - each user needs their own copy with local paths
- **Use `.mcp.json.sample`** as a template to avoid path conflicts
- **Environment variables** can be customized per deployment

### Environment Variables

- `LOG_LEVEL` - Logging level (default: INFO)
- `SERVER_NAME` - Server name (default: "Generic Data Analytics MCP")

## 🚀 Getting Started in Claude Code

Once your MCP server is configured and running, **start with this slash command in Claude Code to get oriented**:

```
/quick-data:list_mcp_assets_prompt
```

This will show you all available tools, resources, and prompts with descriptions - your complete toolkit for data analytics!

## 💡 Sample Datasets Included

### E-commerce Orders (`data/ecommerce_orders.json`)
- **15 orders** with customer segments, regions, product categories
- **Use cases**: Revenue analysis, customer segmentation, regional performance

### Employee Survey (`data/employee_survey.csv`) 
- **25 employees** with satisfaction scores, departments, tenure
- **Use cases**: HR analytics, satisfaction analysis, department comparisons

### Product Performance (`data/product_performance.csv`)
- **20 products** with sales, suppliers, ratings, launch dates
- **Use cases**: Product analysis, supplier performance, market trends

## 🎯 Architecture Benefits

### Dataset Agnosticism
- **Works with ANY structured data** - no hardcoded schemas required
- **Intelligent column detection** - automatically classifies data types
- **Zero configuration** - drop in data files and start analyzing immediately

### Modular Excellence  
- **Clean separation** - tools, resources, prompts, and models organized logically
- **Independent testing** - each component tested in isolation
- **Easy extension** - add new analytics without affecting existing functionality

### Production Ready
- **Comprehensive error handling** - graceful failures with actionable messages
- **Memory optimization** - efficient pandas operations with usage monitoring
- **Performance monitoring** - built-in analytics for large datasets

### AI Integration
- **Smart recommendations** - analysis suggestions based on data characteristics
- **Context-aware prompts** - conversations that reference real column names
- **Adaptive workflows** - tools that adjust behavior based on data types

## 🔮 Extension Examples

### Adding Custom Analytics
```python
# Add to tools/__init__.py or individual tool file
@staticmethod
async def custom_analysis(dataset_name: str, parameters: dict) -> dict:
    """Your custom analysis function."""
    df = DatasetManager.get_dataset(dataset_name)
    # Your analysis logic here
    return {"analysis": "results"}

# Register in server.py
@mcp.tool()
async def custom_analysis(dataset_name: str, parameters: dict) -> dict:
    return await tools.custom_analysis(dataset_name, parameters)
```

### Adding Domain-Specific Prompts
```python
# Add to prompts/__init__.py
@staticmethod
async def financial_analysis_workshop(dataset_name: str) -> str:
    """Guide financial analysis workflows."""
    # Custom financial analysis guidance
    return prompt_text

# Register in server.py  
@mcp.prompt()
async def financial_analysis_workshop(dataset_name: str) -> str:
    return await prompts.financial_analysis_workshop(dataset_name)
```

## 🏆 Success Metrics

- ✅ **Comprehensive Test Coverage** - 130 tests passing
- ✅ **Universal Data Compatibility** - Works with any JSON/CSV structure  
- ✅ **Universal MCP Client Compatibility** - Supports both resource-enabled and tool-only clients
- ✅ **Custom Code Execution** - Full Python analytics capabilities with pandas/numpy/plotly
- ✅ **AI Integration** - Smart recommendations and adaptive conversations
- ✅ **Performance Optimized** - Memory-efficient operations with monitoring

This MCP server transforms the concept of data analytics from rigid, schema-dependent tools into a flexible, AI-guided platform that adapts to any dataset while providing expert-level guidance through conversational interfaces.