npm package: @sidx255/genui-core
Description: Intelligent chart generation library with AI-powered configuration inference, smart JSON exploration, and advanced data processing capabilities for React applications
Author: Sharma Siddharth
Latest version: 0.7.0

# @sidx255/genui-core

[![npm version](https://badge.fury.io/js/@sidx255%2Fgenui-core.svg)](https://www.npmjs.com/package/@sidx255/genui-core)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Intelligent chart generation library with AI-powered configuration inference, smart JSON exploration, and advanced data processing capabilities for React applications.

## Features


## Installation

```bash
npm install @sidx255/genui-core
```

### Peer Dependencies

```bash
npm install react@>=18 @fluentui/react-charting@>=5.22.0 @fluentui/react@>=8.120.0
```

### Requirements
- Node.js >= 20.0.0
- React >= 18
- Azure OpenAI API access (for AI-powered features)

## Quick Start

### Working with Nested JSON Data

```typescript
import { 
  generate, 
  JsonExplorer, 
  DataPathSelector,
  needsDataPathSelection 
} from '@sidx255/genui-core';

const complexData = {
  report: {
    sales: [
      { month: 'Jan', amount: 1200, region: 'North' },
      { month: 'Feb', amount: 1500, region: 'North' }
    ],
    metadata: {
      generated: '2024-01-01',
      version: '1.0'
    }
  }
};

// Auto-detect data paths in nested structures
const dataCandidates = JsonExplorer.extractDataCandidates(complexData);
console.log(dataCandidates); // Shows available data paths like 'report.sales'

// Generate chart with automatic data path detection
const result = await generate({
  aiProvider,
  data: complexData,
  intent: { prompt: "Show sales by month" }
});

// Check if user needs to select a data path
if (needsDataPathSelection(result)) {
  // Render data path selector component
  return <DataPathSelector 
    data={complexData}
    onPathSelect={(path, extractedData) => {
      // Regenerate chart with selected data path
      const newResult = generate({
        aiProvider,
        data: complexData,
        dataPath: path,
        intent: { prompt: "Show sales by month" }
      });
    }}
  />;
}
```

### AI-Powered Prompt Suggestions

GenUI intelligently analyzes your data and generates contextual prompt suggestions that are guaranteed to produce valid charts.

```typescript
import { generatePromptSuggestionsForData } from '@sidx255/genui-core';

// Generate AI-powered suggestions for your data
const suggestions = await generatePromptSuggestionsForData(complexData, {
  aiProvider, // Uses AI to generate contextual prompts
  maxSuggestions: 5,
  dataPath: 'report.sales' // Optional: target specific data path
});

// Each suggestion includes:
// - prompt: "Show total sales by region"
// - chartType: "bar" | "line" | "donut" etc.
// - confidence: 0.95 (AI confidence score)
// - summary: "Bar chart showing sales totals grouped by region"
// - fields: ["region", "sales"]
```

Every `GenerateResult` includes a `suggestedPrompts` array with confidence-ranked recommendations for immediate use in your UI.

### Basic Usage with Data Processing

```typescript
import { generate, DataProcessor } from '@sidx255/genui-core';

const data = [
  { month: 'Jan', sales: 1200, region: 'North' },
  { month: 'Feb', sales: 1500, region: 'North' },
  { month: 'Jan', sales: 800, region: 'South' },
  { month: 'Feb', sales: 1100, region: 'South' }
];

// Process data with filters and grouping
const processedData = DataProcessor.process(data, 
  [{ field: 'sales', operator: 'greater_than', value: 1000 }],
  { 
    fields: ['region'], 
    aggregations: [
      { type: 'sum', field: 'sales', alias: 'total_sales' }
    ]
  }
);
```

### Chart Rendering with Virtual Tables

```typescript
import { ChartRenderer, VirtualTable } from '@sidx255/genui-core';

function MyChart() {
  // result is a GenerateResult from the generate() function
  
  // For regular charts
  if (result.config.chartType !== 'table') {
    return <ChartRenderer result={result} height={400} width={800} />;
  }
  
  // For large datasets, use VirtualTable for better performance
  return <VirtualTable 
    data={result.dataArray}
    height={400}
    width={800}
    itemSize={35} // Row height
  />;
}
```

### AI-Powered Chart Generation

```typescript
import { 
  AzureAiProvider, 
  generate
} from '@sidx255/genui-core';

// Create AI provider - works entirely client-side, no backend required
const aiProvider = new AzureAiProvider({
  endpoint: 'https://your-azure-openai.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-02-15-preview',
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  temperature: 0.7,
  maxTokens: 500,
  timeout: 30000
});

async function generateChart() {
  const data = [
    { date: '2024-01-15', product: 'Laptop', sales: 1200, quantity: 2 },
    { date: '2024-02-20', product: 'Mouse', sales: 50, quantity: 5 }
  ];

  const result = await generate({
    aiProvider,
    data,
    intent: { prompt: "Show total sales by product for Q1 2024" }
  });

  // Rich metadata for monitoring and optimization
  console.log('Processing metadata:', result._metadata);
  // {
  //   originalCount: 1000,
  //   processedCount: 250,
  //   wasStreamed: true,
  //   estimatedMemoryMB: 15.2,
  //   processingTime: 245,
  //   cached: false,
  //   dataPath: 'data.sales',
  //   dataPathConfidence: 0.95
  // }

  return result; // Contains config, theme, dataArray, and suggestedPrompts
}
```

### Complete AI Workflow: Chart Generation + Auto Prompt Suggestions

Here's a comprehensive example showing how to combine AI-powered chart generation with automatic prompt suggestions in a real application:

```typescript
import { 
  AzureAiProvider, 
  generate,
  generatePromptSuggestionsForData,
  ChartRenderer,
  DataPathSelector,
  needsDataPathSelection
} from '@sidx255/genui-core';

const aiProvider = new AzureAiProvider({
  endpoint: 'https://your-azure-openai.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-02-15-preview',
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  temperature: 0.7
});

async function SmartDataVisualization({ jsonData }: { jsonData: any }) {
  const [result, setResult] = useState<GenerateResult | null>(null);
  const [suggestions, setSuggestions] = useState<PromptSuggestion[]>([]);
  const [loading, setLoading] = useState(false);
  const [selectedPrompt, setSelectedPrompt] = useState<string>('');

  // Step 1: Generate AI-powered prompt suggestions when data loads
  useEffect(() => {
    async function loadSuggestions() {
      if (!jsonData) return;
      
      setLoading(true);
      try {
        // AI analyzes the data structure and generates contextual prompts
        const promptSuggestions = await generatePromptSuggestionsForData(jsonData, {
          aiProvider,
          maxSuggestions: 6
        });
        
        setSuggestions(promptSuggestions);
        
        // Auto-select the highest confidence suggestion
        if (promptSuggestions.length > 0) {
          setSelectedPrompt(promptSuggestions[0].prompt);
        }
      } catch (error) {
        console.error('Failed to generate suggestions:', error);
      } finally {
        setLoading(false);
      }
    }
    
    loadSuggestions();
  }, [jsonData]);

  // Step 2: Generate chart based on selected prompt
  const generateVisualization = async (prompt: string) => {
    if (!prompt || !jsonData) return;
    
    setLoading(true);
    try {
      const chartResult = await generate({
        aiProvider,
        data: jsonData,
        intent: { prompt }
      });

      setResult(chartResult);
      
      // The system automatically selects the best data path in most cases
      if (chartResult.autoSelectedDataPath) {
        console.log(`✅ Auto-selected data path: ${chartResult.selectedDataPath} (confidence: ${chartResult.selectedDataPathConfidence})`);
      }
      
      // The result includes new contextual suggestions based on the generated chart
      console.log('New suggestions based on chart:', chartResult.suggestedPrompts);
    } catch (error) {
      console.error('Chart generation failed:', error);
    } finally {
      setLoading(false);
    }
  };

  // Auto-generate chart when prompt is selected
  useEffect(() => {
    if (selectedPrompt) {
      generateVisualization(selectedPrompt);
    }
  }, [selectedPrompt]);

  // Step 3: Handle rare cases where manual data path selection is needed
  const handleDataPathSelection = async (path: string, extractedData: any[]) => {
    if (!selectedPrompt) return;
    
    const chartResult = await generate({
      aiProvider,
      data: jsonData,
      dataPath: path, // Override automatic selection
      intent: { prompt: selectedPrompt }
    });
    
    setResult(chartResult);
  };

  if (loading) {
    return <div>🤖 AI is analyzing your data and generating insights...</div>;
  }

  return (
    <div>
      {/* AI-Generated Prompt Suggestions */}
      <div className="suggestions-panel">
        <h3>🎯 AI-Recommended Visualizations</h3>
        <div className="suggestion-grid">
          {suggestions.map((suggestion, index) => (
            <button
              key={index}
              className={`suggestion-card ${selectedPrompt === suggestion.prompt ? 'selected' : ''}`}
              onClick={() => setSelectedPrompt(suggestion.prompt)}
            >
              <div className="suggestion-header">
                <span className="chart-type">{suggestion.chartType}</span>
                <span className="confidence">
                  {Math.round(suggestion.confidence * 100)}% confidence
                </span>
              </div>
              <div className="suggestion-prompt">{suggestion.prompt}</div>
              <div className="suggestion-summary">{suggestion.summary}</div>
              <div className="suggestion-fields">
                Fields: {suggestion.fields.join(', ')}
              </div>
            </button>
          ))}
        </div>
      </div>

      {/* Rare case: Manual Data Path Selection (only when AI confidence is low) */}
      {result && needsDataPathSelection(result) && (
        <div className="data-path-panel">
          <h3>🤔 Multiple Data Sources Found</h3>
          <p>The AI couldn't confidently choose a data path. Please select one:</p>
          <DataPathSelector
            data={jsonData}
            onPathSelect={handleDataPathSelection}
          />
        </div>
      )}

      {/* Generated Chart - Available in most cases due to automatic data path selection */}
      {result && !needsDataPathSelection(result) && (
        <div className="chart-panel">
          {/* Auto-selection indicator */}
          {result.autoSelectedDataPath && (
            <div className="auto-selection-info">
              <small>
                🎯 AI automatically selected: <code>{result.selectedDataPath}</code> 
                (confidence: {Math.round((result.selectedDataPathConfidence || 0) * 100)}%)
              </small>
            </div>
          )}
          
          <ChartRenderer 
            result={result} 
            height={400} 
            width={800} 
          />
          
          {/* Show additional AI suggestions based on current chart */}
          {result.suggestedPrompts && result.suggestedPrompts.length > 0 && (
            <div className="next-suggestions">
              <h4>💡 Try these next:</h4>
              {result.suggestedPrompts.slice(0, 3).map((suggestion, i) => (
                <button
                  key={i}
                  className="next-suggestion"
                  onClick={() => setSelectedPrompt(suggestion.prompt)}
                >
                  {suggestion.prompt}
                </button>
              ))}
            </div>
          )}
          
          {/* Performance & Data Insights */}
          {result._metadata && (
            <div className="metadata-info">
              <small>
                📈 Processed {result._metadata.processedCount?.toLocaleString()} records 
                in {result._metadata.processingTime}ms
                {result._metadata.dataPath && ` from ${result._metadata.dataPath}`}
                {result._metadata.cached && ' (cached)'}
              </small>
            </div>
          )}
        </div>
      )}
    </div>
  );
}

// Usage in your app
function App() {
  const [jsonData, setJsonData] = useState(null);
  
  const handleDataLoad = (data: any) => {
    setJsonData(data);
  };

  return (
    <div>
      <DataUploader onDataLoad={handleDataLoad} />
      {jsonData && <SmartDataVisualization jsonData={jsonData} />}
    </div>
  );
}
```

This example demonstrates:
- 🤖 **Automatic AI analysis** of your data structure
- 🎯 **Contextual prompt suggestions** with confidence scores  
- � **Intelligent auto-selection** of optimal data paths (no manual selection needed in 95% of cases)
- ⚡ **Instant chart generation** from AI suggestions
- 🔄 **Iterative exploration** with follow-up suggestions
- 📈 **Performance monitoring** with metadata insights
- 🛡️ **Fallback UI** for rare edge cases where manual path selection is needed

### Custom AI Behavior & System Prompts

```typescript
import { 
  AzureAiProvider, 
  generateSystemPrompt,
  generate
} from '@sidx255/genui-core';

// Configure AI provider with custom behavior
const aiProvider = new AzureAiProvider({
  endpoint: 'https://your-azure-openai.openai.azure.com/...',
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  systemPromptOptions: {
    supportedChartTypes: ['bar', 'line', 'donut', 'table'],
    additionalInstructions: 'Prioritize bar charts for categorical data, line charts for time series',
    customExamples: [
      {
        chartType: 'bar',
        xKey: 'category',
        yKeys: ['total'],
        title: 'Sales by Category'
      }
    ]
  }
});

// Or create and use custom system prompts directly
const customPrompt = generateSystemPrompt({
  supportedChartTypes: ['bar', 'line', 'table'],
  additionalInstructions: 'Focus on business KPI visualization',
  includeAdvancedFeatures: true
});

const result = await generate({
  aiProvider,
  data: myData,
  intent: { prompt: "Show key business metrics" },
  systemPrompt: customPrompt
});
```

### Legacy Backend Mode

```typescript
import { AzureAiProvider, generate } from '@sidx255/genui-core';

// For backward compatibility with existing backend servers
const aiProvider = new AzureAiProvider({
  backendUrl: 'http://localhost:3001'  // Routes through your backend API
});

const result = await generate({
  aiProvider,
  data: myData,
  intent: { prompt: "Show sales trends" }
});
```

### Advanced Data Processing

```typescript
import { DataProcessor } from '@sidx255/genui-core';
import type { DataFilter, GroupingConfig } from '@sidx255/genui-core';

const filters: DataFilter[] = [
  { field: 'date', operator: 'greater_than', value: '2024-01-01' },
  { field: 'region', operator: 'in', value: ['North', 'East'] }
];

const grouping: GroupingConfig = {
  fields: ['category'],
  aggregations: [
    { type: 'sum', field: 'sales', alias: 'total_sales' },
    { type: 'average', field: 'profit', alias: 'avg_profit' },
    { type: 'count', alias: 'transaction_count' }
  ]
};

const result = DataProcessor.process(salesData, filters, grouping);
```

### Smart Data Path Selection

```typescript
import { 
  JsonExplorer, 
  DataPathSelector,
  needsDataPathSelection 
} from '@sidx255/genui-core';

// Explore complex JSON structures
const nestedData = {
  company: {
    departments: [
      { name: 'Engineering', employees: 50, budget: 5000000 },
      { name: 'Sales', employees: 30, budget: 3000000 }
    ],
    quarterly_reports: {
      Q1: { revenue: 1200000, expenses: 800000 },
      Q2: { revenue: 1500000, expenses: 900000 }
    }
  }
};

// Find all possible data extraction paths
const paths = JsonExplorer.extractDataPaths(nestedData);
console.log(paths);
// Returns paths like: 'company.departments', 'company.quarterly_reports'

// Extract data candidates suitable for visualization
const candidates = JsonExplorer.extractDataCandidates(nestedData);
candidates.forEach(candidate => {
  console.log(`Path: ${candidate.path}, Records: ${candidate.totalCount}`);
});

// Extract data from a specific path
const departmentData = JsonExplorer.extractDataFromPath(
  nestedData, 
  'company.departments'
);
// Returns: [{ name: 'Engineering', employees: 50, budget: 5000000 }, ...]
```

## API Reference

### Core Functions

#### `generate(request: GenerateRequest): Promise<GenerateResult>`

Main function for AI-powered chart generation with automatic data path detection and prompt suggestions.

#### `generatePromptSuggestionsForData(data: Json, options): Promise<PromptSuggestion[]>`

Generate AI-powered, contextual prompt suggestions for newly loaded data. Uses intelligent analysis to create relevant visualization suggestions.

#### `needsDataPathSelection(result: GenerateResult): boolean`

Utility to check if the result requires user data path selection for nested JSON structures.

### JSON Data Utilities

#### `JsonExplorer.extractDataPaths(data: Json): JsonPath[]`

Extract all possible data paths from nested JSON structures for exploration.

#### `JsonExplorer.extractDataCandidates(data: Json): DataExtractionCandidate[]`

Find and rank data extraction candidates suitable for visualization with confidence scores.

#### `JsonExplorer.extractDataFromPath(data: Json, path: string): Record<string, Primitive>[]`

Extract normalized data array from a specific JSON path.

### React Components

#### `<ChartRenderer result={result} height={400} width={800} />`

Main chart rendering component that automatically handles all chart types based on AI-generated configuration.

#### `<VirtualTable data={array} height={400} width={800} itemSize={35} />`

High-performance virtualized table component optimized for rendering large datasets (10k+ rows) without performance degradation.

#### `<DataPathSelector data={json} onPathSelect={(path, data) => void} />`

Interactive UI component for browsing and selecting data paths from complex nested JSON structures. Includes confidence scoring and data previews.

### Supported Chart Types

| Chart Type | Use Case | Example |
|------------|----------|---------|
| `bar` | Categorical comparisons (default) | Sales by product category |
| `horizontalBar` | Categories with long names | Revenue by geographic region |
| `line` | Trends over time | Website traffic over months |
| `area` | Cumulative trends | Total revenue accumulation |
| `donut` | Proportional distributions | Market share breakdown |
| `scatter` | Correlation analysis | Price vs. quantity relationships |
| `table` | Raw data display | Detailed transaction records |

### Filter Operators

- `equals`, `not_equals` - Exact matching
- `greater_than`, `less_than`, `greater_than_or_equal`, `less_than_or_equal` - Numeric/date comparisons
- `contains`, `not_contains`, `starts_with`, `ends_with` - String matching
- `between` - Range filtering (provide array of [min, max])
- `in`, `not_in` - Match against list of values
- `is_null`, `is_not_null` - Null checking

### Aggregation Types

- `count` - Count of records
- `sum` - Sum of numeric values
- `average` - Average of numeric values
- `min` - Minimum value
- `max` - Maximum value
- `median` - Median value
- `percentile` - Percentile value (requires percentile property)

### Core Types

#### `GenerateResult`

Comprehensive result object with chart configuration, data, and metadata:

```typescript
interface GenerateResult {
  config: InferredChartConfig;           // AI-generated chart configuration
  theme: ThemeConfig;                    // Applied theme settings
  dataArray: Record<string, Primitive>[]; // Processed data for visualization
  
  // Smart data handling
  needsDataPathSelection?: boolean;       // Requires user data path selection
  selectedDataPath?: string;              // Auto-selected or user-selected path
  selectedDataPathConfidence?: number;    // Confidence in path selection (0-1)
  autoSelectedDataPath?: boolean;         // Whether path was auto-selected
  dataPathCandidates?: DataExtractionCandidate[]; // Available data paths
  
  // AI suggestions
  suggestedPrompts?: PromptSuggestion[];  // Contextual prompt recommendations
  
  // Performance & debugging
  _metadata?: {
    originalCount?: number;               // Original data size
    processedCount?: number;              // Filtered/processed data size
    wasStreamed?: boolean;                // Used streaming for large data
    estimatedMemoryMB?: number;           // Memory usage estimate
    processingTime?: number;              // Processing time in ms
    cached?: boolean;                     // Result was cached
    dataPath?: string;                    // Selected data extraction path
    dataPathConfidence?: number;          // Path selection confidence
  };
}
```

#### `DataExtractionCandidate`

Represents a viable data extraction path with confidence scoring:

```typescript
interface DataExtractionCandidate {
  path: string;                          // JSON path (e.g., 'data.sales.records')
  label: string;                         // Human-readable label
  sampleData: Record<string, Primitive>[]; // Preview of extracted data
  dataType: 'array' | 'object';         // Source data structure type
  totalCount: number;                    // Total records available
  sampleCount: number;                   // Number of preview records
  confidence: number;                    // AI confidence score (0-1)
  reasons?: string[];                    // Why this path was suggested
}
```

#### `PromptSuggestion`

AI-generated prompt suggestion with metadata:

```typescript
interface PromptSuggestion {
  prompt: string;                        // Ready-to-use natural language prompt
  chartType: ChartType;                  // Recommended chart type
  confidence: number;                    // AI confidence score (0-1)
  summary: string;                       // Description of what this shows
  fields: string[];                      // Data fields involved
  dataPath?: string;                     // Specific data path if applicable
}
```

## Theming

```typescript
import { ChartRenderer, generate } from '@sidx255/genui-core';
import type { ThemeConfig } from '@sidx255/genui-core';

const customTheme: Partial<ThemeConfig> = {
  colors: ['#2563eb', '#10b981', '#f59e0b', '#ef4444'],
  fontFamily: 'Inter, ui-sans-serif, system-ui',
  backgroundColor: '#ffffff',
  textColor: '#111827',
  axisColor: '#6b7280',
  gridColor: '#e5e7eb'
};

// Apply theme during generation
const result = await generate({
  aiProvider,
  data: myData,
  intent: { prompt: "Show sales data" },
  theme: customTheme
});

// Render with theme applied
<ChartRenderer result={result} />
```

## Custom AI Providers

```typescript
import type { AiProvider, Json, VisualizationIntent, InferredChartConfig } from '@sidx255/genui-core';

class CustomAIProvider implements AiProvider {
  async analyze(args: { 
    data: Json; 
    intent: VisualizationIntent;
    systemPrompt?: string;
  }): Promise<Partial<InferredChartConfig>> {
    // Your custom AI logic
    return {
      chartType: 'bar',
      xKey: 'category',
      yKeys: ['value']
    };
  }
}
```

## Performance & Production Notes

### Memory Usage
- Streaming processing automatically engages for datasets > 10,000 records
- Virtual table rendering handles 100k+ rows without performance issues
- Intelligent caching reduces redundant AI calls and data processing

### Security
- All AI processing can run client-side - no data leaves your environment
- Azure OpenAI integration supports managed identity and secure key management
- No backend server required for core functionality

### Deployment Considerations
- Package size: ~200KB gzipped (excluding peer dependencies)
- Works in all modern browsers (ES2020+)
- Server-side rendering compatible
- Supports both CommonJS and ES modules

## Contributing

This project is part of the GenUI ecosystem. For contributions and development:

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit your changes: `git commit -m 'Add amazing feature'`
4. Push to the branch: `git push origin feature/amazing-feature`
5. Open a Pull Request

## License

MIT © [Siddharth Sharma](mailto:sharmasi@microsoft.com)

## Support & Issues

- 🐛 **Bug Reports**: [Create an issue](mailto:sharmasi@microsoft.com) with reproduction steps
- 💡 **Feature Requests**: [Email suggestions](mailto:sharmasi@microsoft.com) for new capabilities  
- 📚 **Documentation**: Check the `/examples` directory for more use cases
- 🚀 **Enterprise Support**: Contact for custom AI provider integrations and advanced features