npm package: @startsimpli/funnels
Description: Brutally generic filtering pipeline package for any Simpli product
Author: Simpli
Latest version: 0.4.6

# @simpli/funnels

> Brutally generic filtering pipeline package for any Simpli product

[![Tests](https://img.shields.io/badge/tests-487%20passing-green)]()
[![TypeScript](https://img.shields.io/badge/typescript-5.9%2B-blue)]()
[![React](https://img.shields.io/badge/react-18%2B%20%7C%2019%2B-blue)]()
[![Zustand](https://img.shields.io/badge/zustand-4%2B%20%7C%205%2B-blue)]()

## What is this?

A reusable package for building multi-stage filtering funnels. Works with **ANY entity type** - investors, recipes, leads, tasks, GitHub issues, or whatever you dream up.

### The Philosophy

**BRUTALLY GENERIC** - No domain-specific types. No investor-specific fields. No recipe-specific logic.

It's just:
1. Start with entities (any type)
2. Apply sequential filter stages
3. Each stage: keep/exclude/tag based on rules
4. End with filtered subset + accumulated tags/context

### Why Use This?

- **Zero domain coupling** - Works for investors, recipes, leads, products, tasks, anything
- **Type-safe** - Full TypeScript support with generics
- **Modular** - Import only what you need (tree-shakeable)
- **Server-compatible** - Core engine has NO React dependencies
- **Battle-tested** - 487 tests passing
- **Production-ready** - Used across multiple Simpli products

## Installation

```bash
npm install @simpli/funnels
```

## Quick Start

### Example 1: Investor Funnel

Filter investors for a Series A fundraise:

```typescript
import { Funnel, FunnelEngine } from '@simpli/funnels';

interface Investor {
  name: string;
  firm: {
    stage: string;
    check_size_min: number;
    check_size_max: number;
  };
}

const funnel: Funnel<Investor> = {
  id: 'series-a-funnel',
  name: 'Series A Investor Qualification',
  status: 'active',
  input_type: 'contacts',
  stages: [
    {
      id: 'stage-1',
      order: 0,
      name: 'Stage Filter',
      filter_logic: 'OR',
      rules: [
        { field_path: 'firm.stage', operator: 'eq', value: 'Series A' },
        { field_path: 'firm.stage', operator: 'eq', value: 'Multi-Stage' }
      ],
      match_action: 'tag_continue',
      no_match_action: 'exclude',
      match_tags: ['qualified_stage']
    },
    {
      id: 'stage-2',
      order: 1,
      name: 'Check Size',
      filter_logic: 'AND',
      rules: [
        { field_path: 'firm.check_size_min', operator: 'lte', value: 5000000 },
        { field_path: 'firm.check_size_max', operator: 'gte', value: 3000000 }
      ],
      match_action: 'output',
      no_match_action: 'exclude',
      match_tags: ['qualified']
    }
  ],
  created_at: new Date().toISOString(),
  updated_at: new Date().toISOString()
};

// Execute funnel
const engine = new FunnelEngine();
const investors = [/* your data */];
const results = engine.executeSync(funnel, investors);

console.log(`Matched: ${results.matched.length}`);
console.log(`Excluded: ${results.excluded.length}`);
```

### Example 2: Recipe Funnel

Find quick, easy, vegetarian recipes:

```typescript
import { Funnel } from '@simpli/funnels';

interface Recipe {
  name: string;
  prep_time_minutes: number;
  difficulty: 'easy' | 'medium' | 'hard';
  dietary_restrictions: string[];
}

const funnel: Funnel<Recipe> = {
  id: 'quick-dinner',
  name: 'Quick Weeknight Dinner',
  status: 'active',
  input_type: 'any',
  stages: [
    {
      id: 'dietary',
      order: 0,
      name: 'Dietary Restrictions',
      filter_logic: 'AND',
      rules: [
        { field_path: 'dietary_restrictions', operator: 'has_all', value: ['vegetarian'] }
      ],
      match_action: 'tag_continue',
      no_match_action: 'exclude',
      match_tags: ['vegetarian']
    },
    {
      id: 'time',
      order: 1,
      name: 'Quick Prep',
      filter_logic: 'AND',
      rules: [
        { field_path: 'prep_time_minutes', operator: 'lte', value: 30 }
      ],
      match_action: 'tag_continue',
      no_match_action: 'exclude',
      match_tags: ['quick']
    },
    {
      id: 'difficulty',
      order: 2,
      name: 'Easy to Make',
      filter_logic: 'OR',
      rules: [
        { field_path: 'difficulty', operator: 'eq', value: 'easy' },
        { field_path: 'difficulty', operator: 'eq', value: 'medium' }
      ],
      match_action: 'output',
      no_match_action: 'exclude',
      match_tags: ['beginner_friendly']
    }
  ],
  created_at: new Date().toISOString(),
  updated_at: new Date().toISOString()
};
```

### Example 3: Lead Scoring

Score sales leads based on company size and engagement:

```typescript
import { Funnel } from '@simpli/funnels';

interface Lead {
  company: { size: number };
  engagement: {
    email_opens: number;
    demo_requested: boolean;
  };
  tags: string[];
}

const funnel: Funnel<Lead> = {
  id: 'lead-scoring',
  name: 'Enterprise Lead Scoring',
  status: 'active',
  input_type: 'any',
  stages: [
    {
      id: 'company-size',
      order: 0,
      name: 'Enterprise Size',
      filter_logic: 'AND',
      rules: [
        { field_path: 'company.size', operator: 'gte', value: 100 }
      ],
      match_action: 'tag_continue',
      no_match_action: 'tag_continue',
      match_tags: ['enterprise'],
      no_match_tags: ['smb']
    },
    {
      id: 'engagement',
      order: 1,
      name: 'High Engagement',
      filter_logic: 'OR',
      rules: [
        { field_path: 'engagement.email_opens', operator: 'gte', value: 5 },
        { field_path: 'engagement.demo_requested', operator: 'is_true', value: null }
      ],
      match_action: 'output',
      no_match_action: 'output',
      match_tags: ['hot_lead'],
      match_context: { tier: 'A', score: 100 }
    }
  ],
  created_at: new Date().toISOString(),
  updated_at: new Date().toISOString()
};
```

## Core Concepts

### Funnel

A sequential pipeline with multiple filtering stages. Each funnel has:
- **Stages**: Ordered sequence of filter conditions
- **Status**: draft | active | paused | archived
- **Metadata**: Tags, context, ownership info

```typescript
interface Funnel<TEntity = any> {
  id: string;
  name: string;
  status: 'draft' | 'active' | 'paused' | 'archived';
  stages: FunnelStage<TEntity>[];
  // ... more fields
}
```

### Stage

A single filtering step with rules, actions, and tags:

```typescript
interface FunnelStage<TEntity = any> {
  id: string;
  order: number;
  name: string;
  filter_logic: 'AND' | 'OR';
  rules: FilterRule[];
  match_action: 'continue' | 'tag' | 'tag_continue' | 'output';
  no_match_action: 'continue' | 'exclude' | 'tag_exclude';
  match_tags?: string[];
  no_match_tags?: string[];
  custom_evaluator?: (entity: TEntity) => boolean;
}
```

### Filter Rule

A single condition with field path, operator, and value:

```typescript
interface FilterRule {
  field_path: string;    // 'firm.stage', 'recipe.cuisine', 'tags'
  operator: Operator;    // 'eq', 'gt', 'contains', 'has_any', etc.
  value: any;           // Value to compare against
  negate?: boolean;     // Optional negation
}
```

**Supported operators:**
- **Equality**: `eq`, `ne`
- **Comparison**: `gt`, `lt`, `gte`, `lte`
- **String**: `contains`, `not_contains`, `startswith`, `endswith`, `matches`
- **Array**: `in`, `not_in`, `has_any`, `has_all`
- **Null**: `isnull`, `isnotnull`
- **Tags**: `has_tag`, `not_has_tag`
- **Boolean**: `is_true`, `is_false`

### Field Registry

Defines what fields are available for filtering in your domain:

```typescript
const investorRegistry: FieldRegistry = {
  entity_type: 'investor',
  fields: [
    {
      name: 'firm.stage',
      label: 'Investment Stage',
      type: 'string',
      operators: ['eq', 'ne', 'in', 'not_in'],
      category: 'Firm Details',
      constraints: {
        choices: ['Seed', 'Series A', 'Series B', 'Growth']
      }
    },
    {
      name: 'firm.check_size_min',
      label: 'Min Check Size',
      type: 'number',
      operators: ['eq', 'ne', 'gt', 'lt', 'gte', 'lte'],
      category: 'Firm Details'
    }
  ]
};
```

## Modular Exports

Import only what you need for optimal bundle size:

```typescript
// Full package (everything)
import { FunnelEngine, FunnelPreview, createFunnelStore } from '@simpli/funnels';

// Core only (NO React dependencies - perfect for workers, CLI, Node.js)
import { FunnelEngine, evaluateRule, applyOperator } from '@simpli/funnels/core';

// Components only (React UI)
import { FunnelCard, FunnelPreview, FunnelStageBuilder } from '@simpli/funnels/components';

// Hooks only (React hooks)
import { useDebouncedValue } from '@simpli/funnels/hooks';

// State only (Zustand store)
import { createFunnelStore } from '@simpli/funnels/store';
```

**Use cases:**
- **Server-side worker**: `@simpli/funnels/core` (no React, no DOM)
- **Next.js app**: `@simpli/funnels` (full package)
- **Component library**: `@simpli/funnels/components` (UI only)
- **State management**: `@simpli/funnels/store` (Zustand store)

## API Reference

See [API_REFERENCE.md](./API_REFERENCE.md) for complete API documentation.

### Core Engine

```typescript
import { FunnelEngine } from '@simpli/funnels/core';

const engine = new FunnelEngine();

// Synchronous execution
const results = engine.executeSync(funnel, entities);

// Get matched entities
const matched = results.matched.map(r => r.entity);

// Get excluded entities
const excluded = results.excluded.map(r => r.entity);
```

### React Components

```typescript
import { FunnelCard, FunnelPreview, FunnelStageBuilder } from '@simpli/funnels/components';

// Display funnel card
<FunnelCard funnel={myFunnel} onEdit={handleEdit} onDelete={handleDelete} />

// Preview funnel results
<FunnelPreview funnel={myFunnel} entities={myData} />

// Build funnel stages
<FunnelStageBuilder
  funnel={myFunnel}
  fieldRegistry={registry}
  onChange={handleChange}
/>
```

### State Management

```typescript
import { createFunnelStore } from '@simpli/funnels/store';

const useFunnelStore = createFunnelStore();

function MyComponent() {
  const { funnel, updateStage, addStage } = useFunnelStore();

  // Use store methods
  addStage(newStage);
  updateStage(stage.id, { name: 'New Name' });
}
```

## Architecture

### Why Brutally Generic?

Traditional filtering systems hardcode domain models:

```typescript
// ❌ Domain-specific (not reusable)
interface InvestorFilter {
  firm_stage: string[];
  check_size_min: number;
  geography: string[];
}

interface RecipeFilter {
  cuisine: string[];
  prep_time_max: number;
  dietary: string[];
}
```

With @simpli/funnels, one system handles everything:

```typescript
// ✅ Brutally generic (reusable everywhere)
interface FilterRule {
  field_path: string;    // Works for ANY field
  operator: Operator;    // Works for ANY comparison
  value: any;           // Works for ANY value
}
```

**Benefits:**
- Write filtering logic once, use everywhere
- Add new domains without code changes
- Type-safe with TypeScript generics
- Test once, trust everywhere

### Sequential Stage Processing

Stages execute in order (0, 1, 2, ...). Each stage can:

1. **Continue** - Pass entity to next stage
2. **Exclude** - Remove entity from output (stop processing)
3. **Tag** - Add tags and stop processing
4. **Tag + Continue** - Add tags and pass to next stage
5. **Output** - Mark as matched (final stage)

```typescript
Stage 0: Filter by investment stage
  ├─ Match → tag 'qualified_stage', continue
  └─ No match → exclude

Stage 1: Filter by check size
  ├─ Match → tag 'qualified_check_size', continue
  └─ No match → tag 'excluded_check_size', exclude

Stage 2: Filter by geography
  ├─ Match → output (final)
  └─ No match → exclude
```

### Accumulated State

Tags and context accumulate across stages:

```typescript
{
  entity: { /* investor data */ },
  matched: true,
  accumulated_tags: ['qualified_stage', 'qualified_check_size', 'qualified_geography'],
  context: {
    stage: 'qualified',
    tier: 'A',
    score: 100
  }
}
```

## Examples

See [EXAMPLES.md](./EXAMPLES.md) for 6+ real-world examples:

- Investor qualification funnel
- Recipe recommendation funnel
- Lead scoring funnel
- GitHub issue triage funnel
- E-commerce product filtering
- Task prioritization funnel

## Integration

See [INTEGRATION_GUIDE.md](./INTEGRATION_GUIDE.md) for step-by-step integration instructions.

Quick summary:

1. Install package: `npm install @simpli/funnels`
2. Create field registry for your domain
3. Use components in your UI
4. Connect to your API/backend
5. Configure Tailwind CSS (if using components)

## Storybook

Explore 50+ interactive examples:

```bash
cd packages/funnels
npm run storybook
```

Or view the built Storybook in `./storybook-static/index.html`

See [STORYBOOK.md](./STORYBOOK.md) for details.

## Testing

The package includes comprehensive test coverage:

```bash
npm run test              # Run all tests
npm run test:watch        # Watch mode
npm run test:coverage     # Coverage report
```

**Test stats:**
- **487 tests** passing
- **Core engine**: 262 tests
- **Components**: 185 tests
- **Store**: 29 tests
- **API client**: 24 tests

## Contributing

See [CONTRIBUTING.md](./CONTRIBUTING.md) for contribution guidelines.

Quick summary:

1. Fork the repo
2. Create a feature branch
3. Make your changes
4. Add tests
5. Run `npm test` and `npm run type-check`
6. Submit a PR

## Changelog

See [CHANGELOG.md](./CHANGELOG.md) for version history.

## License

MIT - See [LICENSE](./LICENSE) for details.

---

## Why "Brutally Generic"?

Because it works for **literally anything**:

- ✅ Investors (VC fundraising)
- ✅ Recipes (cooking app)
- ✅ Leads (sales CRM)
- ✅ GitHub issues (project management)
- ✅ Products (e-commerce)
- ✅ Tasks (task manager)
- ✅ Your domain (whatever it is)

**Zero domain-specific types. Just generic entity processing with rich filtering capabilities.**

---

Built with ❤️ by the Simpli team