istari-core 0.1.2
Infer user intent states from behavioral event streams
License: Apache-2.0
Requires Python: >=3.8
Keywords: intent, analytics, behavior, cro, personalization, ai

# Istari

Istari is a Python library that infers user intent states from raw behavioral event streams and exposes them as a structured, machine-readable layer for CRO systems, personalization engines, and agentic AI workflows.

Instead of treating user behavior as isolated events or funnels, Istari models sessions as continuous intent trajectories over time.

## What Problem Istari Solves

Modern products collect events like:
- page views
- scroll depth
- clicks
- add to cart
- remove from cart
- time gaps
- navigation loops
- rage clicks (Clarity)
- dead clicks (Clarity)

But downstream systems still reason in crude terms like funnels and conversion rates.

Istari sits between raw analytics and decision-making systems and answers:
- What is the user trying to do right now?
- Are they exploring, comparing, hesitating, or ready to act?
- What friction or uncertainty is blocking progress?
- How intent is evolving over the session?

## Core Abstraction

Istari introduces a first-class abstraction called an **Intent State**.

An intent state is:
- **time-aware**: associated with timestamps
- **probabilistic**: has confidence scores
- **explainable**: includes attribution data
- **composable**: can be combined across domains

Example intent states:
- `browsing`
- `evaluating_options`
- `price_sensitive`
- `trust_seeking`
- `purchase_ready`
- `abandonment_risk`

## High-Level Flow

1. Raw events are normalized into a canonical schema
2. Temporal patterns and transitions are extracted
3. Intent hypotheses are scored
4. A state machine produces the most likely intent trajectory
5. Outputs are exposed to CRO rules or agentic systems

## Design Goals

- **Python-first, backend friendly**
- **Deterministic core** with optional ML layers
- **Explainability by default**
- **Domain-adaptable** without retraining
- **Open-core friendly architecture**

## Installation

```bash
pip install istari-core
```

For development:

```bash
git clone https://github.com/istari/istari.git
cd istari
pip install -e .
```

## Quick Start

```python
from datetime import datetime, timedelta, timezone
from istari.core.events import Event
from istari.core.session import Session
from istari.inference.rules import RuleBasedInference
from istari.inference.heuristics import BrowsingRule, PurchaseReadyRule

# Create a session
session = Session(
    session_id="session_123",
    user_id="user_456",
    started_at=datetime.now(timezone.utc),
)

# Add events
session.add_event(Event(
    event_type="page_view",
    timestamp=datetime.now(timezone.utc),
    user_id="user_456",
    session_id="session_123",
    properties={"page": "/products"},
))

# Infer intent state
inference = RuleBasedInference()
inference.add_rule(BrowsingRule())
inference.add_rule(PurchaseReadyRule())

intent_state = inference.infer(session)
print(f"Intent: {intent_state.state_type}")
print(f"Confidence: {intent_state.confidence:.2%}")
```

## Microsoft Clarity Integration

Istari includes first-class support for Microsoft Clarity behavioral signals. You can import data either from Clarity's API or from exported data files.

### API Integration

Fetch data directly from Clarity's Data Export API:

```python
from istari.sources.clarity import ClaritySource

# Initialize Clarity source
clarity_source = ClaritySource()

# Get your API key from Clarity project settings:
# Settings > Data Export > Generate new API token
api_key = "your-api-key"
project_id = "your-project-id"

# Fetch from API and process
events, signals = clarity_source.import_from_api(
    api_key=api_key,
    project_id=project_id,
    start_date="2024-01-01",  # Optional: filter by date range
    end_date="2024-01-31",    # Optional: filter by date range
)

print(f"Imported {len(events)} events")
print(f"Signals: {signals}")
```

### Manual Data Import

You can also import from exported data:

```python
from istari.sources.clarity import ClaritySource

clarity_source = ClaritySource()

# Parse Clarity events/exports
clarity_events = [
    {
        "event": "rage_click",
        "timestamp": 1234567890000,
        "userId": "user_123",
        "sessionId": "session_456",
        "clickCount": 5,
    },
    # ... more events
]

# Parse and extract signals
events, signals = clarity_source.process(clarity_events)
```

### Getting Your API Key

1. Go to your Clarity project dashboard
2. Navigate to **Settings** > **Data Export**
3. Click **Generate new API token**
4. Copy the token and use it as your `api_key`

**API Endpoint**: `https://www.clarity.ms/export-data/api/v1/project-live-insights`

**Authentication**: Bearer token in `Authorization` header

See the [Clarity Data Export API documentation](https://learn.microsoft.com/en-us/clarity/setup-and-installation/clarity-data-export-api) for more details.

### Clarity Signal Mappings

| Clarity Signal | Istari Signal | Intent Impact |
|----------------|---------------|---------------|
| Rage clicks | `friction.high` | Contributes to `abandonment_risk` |
| Dead clicks | `intent.confusion` | Contributes to `hesitating` |
| Scroll depth | `content_engagement` | Contributes to `browsing` |
| Quick back nav | `dissatisfaction` | Contributes to `abandonment_risk` |
| Excessive hover | `hesitation` | Contributes to `hesitating` |

## Architecture

### Core Modules

- **`core/`**: Event normalization, session modeling, intent states, transitions, scoring
- **`schemas/`**: Event schema normalization (base, ecommerce, web)
- **`inference/`**: Rule-based inference, heuristics, state machine, confidence calculation
- **`signals/`**: Signal extraction (dwell, navigation, comparison, friction, price)
- **`explainability/`**: Attribution calculation, narrative generation, summaries
- **`plugins/`**: Plugin system for extending inference capabilities
- **`sources/`**: Event source integrations (Clarity, etc.)
- **`integrations/`**: Analytics platform integrations (Mixpanel, Amplitude, Segment)

### Open-Source vs Proprietary

**Open-source core:**
- Event normalization
- Session modeling
- Intent state definitions
- Rule-based inference
- Explainability primitives
- Plugin interfaces
- Clarity integration

**Proprietary extensions (planned):**
- Learned intent models
- Vertical-specific heuristics
- Advanced uncertainty modeling
- Real-time streaming inference
- Deep analytics integrations

## Examples

See the `examples/` directory for:
- `basic_inference.py`: Basic intent inference
- `ecommerce_session.py`: Full e-commerce session analysis
- `explain_intent.py`: Intent explanation and narratives
- `clarity_integration.py`: Microsoft Clarity integration example

## Documentation

- [Concepts](docs/concepts.md)
- [Intent States](docs/intent_states.md)
- [Schemas](docs/schemas.md)
- [Explainability](docs/explainability.md)
- [Plugins](docs/plugins.md)

## License

Apache License 2.0 - see [LICENSE](LICENSE) file for details.

## Contributing

Contributions welcome! Please see our contributing guidelines (coming soon).