npm package: @kabon/email-ingest
Description: Universal email ingestion and outbound reply service with IMAP and SMTP support
Author: KabonShare
Latest version: 1.1.0

# @kabon/email-ingest

Universal email ingestion and outbound reply service. Connects to any IMAP-compatible email provider (Zoho, Gmail, Outlook, Yahoo, etc.) and sends threaded replies via SMTP.

Two capabilities:
- **IMAP** — Real-time inbound email detection via IDLE (push) or polling (pull). Works with any provider.
- **SMTP Outbound** — Send threaded replies that show up in the same email conversation.

## Install

```bash
npm install @kabon/email-ingest
```

## IMAP Inbound (IDLE Mode — Default)

```js
import { EmailIngest } from '@kabon/email-ingest';

const ingest = new EmailIngest({
  adapter: 'imap',
  imap: {
    host: 'imap.zoho.com',       // or imap.gmail.com, outlook.office365.com
    port: 993,
    auth: {
      user: 'support@yourapp.com',
      pass: 'your-app-password',
    },
    folder: 'INBOX',
    markSeen: true,
    mode: 'idle',  // 'idle' (default) or 'poll'
  },
});

ingest.on('email', (email) => {
  console.log(`New email from ${email.from.address}: ${email.subject}`);
});

ingest.on('ready', () => console.log('Listening for emails...'));
ingest.on('error', (err) => console.error(err.message));

await ingest.start();
```

### IDLE vs Poll Mode

| Mode | How it works | Best for |
|------|-------------|----------|
| `idle` | Server pushes notifications via IMAP IDLE | Real-time detection, most providers |
| `poll` | Periodic NOOP + fetch on a timer | Providers that don't support IDLE well |

```js
// Poll mode
{
  mode: 'poll',
  pollInterval: 30_000, // Check every 30 seconds
}
```

## UID Persistence (Survive Restarts)

By default, `lastSeenUid` is lost on process restart, causing previously-seen emails to be re-emitted. Use `onUidUpdate` and `lastSeenUid` to persist across restarts:

```js
import { EmailIngest } from '@kabon/email-ingest';
import fs from 'fs';

// Restore from persistence
const savedUid = fs.existsSync('.last-uid') ? parseInt(fs.readFileSync('.last-uid', 'utf8')) : 0;

const ingest = new EmailIngest({
  adapter: 'imap',
  imap: {
    host: 'imap.zoho.com',
    auth: { user: 'support@yourapp.com', pass: 'app-password' },
    lastSeenUid: savedUid,         // Restore from last run
    onUidUpdate: (uid) => {        // Persist on every new email
      fs.writeFileSync('.last-uid', String(uid));
    },
  },
});
```

## Email Filtering

Skip emails before they're emitted using a `filter` callback. Useful for ignoring specific senders, subjects, or automated emails:

```js
const ingest = new EmailIngest({
  adapter: 'imap',
  imap: {
    host: 'imap.zoho.com',
    auth: { user: 'support@yourapp.com', pass: 'app-password' },
    filter: (email) => {
      // Ignore noreply and automated emails
      if (email.from.address.includes('noreply')) return false;
      if (email.subject.startsWith('[AUTOMATED]')) return false;
      return true;
    },
  },
});
```

## Health Check

```js
// Check if the connection is alive
console.log(ingest.isConnected); // true/false

// Get the current UID tracking position
console.log(ingest.adapter.lastSeenUid); // number
```

## SMTP Outbound (Replying to Emails)

Send threaded replies that appear in the same conversation in Gmail, Outlook, etc.

```js
import { SmtpOutbound } from '@kabon/email-ingest';

const smtp = new SmtpOutbound({
  host: 'smtp.zoho.com',       // or smtp.gmail.com, smtp.office365.com
  port: 587,
  auth: {
    user: 'support@yourapp.com',
    pass: 'your-app-password',
  },
  from: '"YourApp Support" <support@yourapp.com>',
});

const { messageId } = await smtp.sendReply({
  to: 'customer@example.com',
  subject: 'Help with login',
  tag: 'TKT-00042',
  body: 'Hi! We have resolved your issue...',
  inReplyTo: '<original-message-id@mail.zoho.com>',
  references: ['<msg1@mail.zoho.com>', '<msg2@mail.zoho.com>'],
});

console.log('Sent with Message-ID:', messageId);

await smtp.verify(); // Check connection
smtp.close();        // Clean up
```

### Sending Attachments

Pass a nodemailer-compatible `attachments` array to include files in your reply:

```js
await smtp.sendReply({
  to: 'customer@example.com',
  subject: 'Your invoice',
  tag: 'TKT-00042',
  body: 'Please find your invoice attached.',
  attachments: [
    // From a URL
    { filename: 'invoice.pdf', href: 'https://storage.example.com/invoices/123.pdf' },
    // From a Buffer
    { filename: 'report.csv', content: Buffer.from('id,name\n1,Alice'), contentType: 'text/csv' },
    // From a file path
    { filename: 'logo.png', path: '/tmp/logo.png' },
  ],
});
```

### How Threading Works

When replying, `SmtpOutbound` sets:
- **`In-Reply-To`** — Points to the last email in the thread
- **`References`** — Lists all message IDs in the thread
- **Subject** — Automatically prefixed with `Re: [TAG]` (e.g. `Re: [TKT-00042] Help with login`)

This makes Gmail, Outlook, Apple Mail, etc. group all messages into one conversation.

## Extracting Tags from Subjects

Use `extractTag()` to pull reference IDs from inbound email subjects for routing replies.

```js
import { extractTag } from '@kabon/email-ingest';

// Default: extracts content inside brackets [...]
extractTag('Re: [TKT-00042] Help with login');  // → 'TKT-00042'
extractTag('Re: [ORD-1234] Shipping delay');     // → 'ORD-1234'

// Custom pattern
extractTag('Re: [TKT-00042] Help', /TKT-\d{5}/i);    // → 'TKT-00042'
extractTag('Case #98765 - Billing', /Case #(\d+)/i);  // → '98765'

// No match returns null
extractTag('Hello world');  // → null
```

## Custom Adapters

```js
import { EmailIngest } from '@kabon/email-ingest';
import { BaseAdapter } from '@kabon/email-ingest/src/adapters/BaseAdapter.js';

class MyAdapter extends BaseAdapter {
  async start() { /* connect and emit 'email' events */ }
  async stop()  { /* disconnect */ }
}

EmailIngest.registerAdapter('custom', MyAdapter);

const ingest = new EmailIngest({ adapter: 'custom', custom: { /* config */ } });
```

## Email Object

```ts
{
  messageId: string,
  uid: number,
  from: { address: string, name: string },
  to: [{ address: string, name: string }],
  subject: string,
  text: string,           // Plain text body
  html: string,           // HTML body
  date: Date,
  headers: {
    inReplyTo: string,    // For thread detection
    references: string[], // For thread detection
  },
  attachments: [{
    filename: string,
    contentType: string,
    size: number,
    content: Buffer,
  }],
}
```

## Provider Setup

### Zoho Mail
- IMAP: `imap.zoho.com:993`, SMTP: `smtp.zoho.com:587`
- Auth: Use an [App Password](https://accounts.zoho.com/home#security/security_pwd) (not your login password)

### Gmail
- IMAP: `imap.gmail.com:993`, SMTP: `smtp.gmail.com:587`
- Auth: Use an [App Password](https://myaccount.google.com/apppasswords) (enable 2FA first)

### Outlook / Microsoft 365
- IMAP: `outlook.office365.com:993`, SMTP: `smtp.office365.com:587`
- Auth: Use OAuth2 access token: `{ user: 'email', accessToken: 'token' }`

## Config Reference (IMAP)

| Option | Default | Description |
|--------|---------|-------------|
| `host` | required | IMAP server hostname |
| `port` | `993` | IMAP port |
| `secure` | `true` | Use TLS |
| `auth.user` | required | Email address |
| `auth.pass` | required | Password or app password |
| `auth.accessToken` | — | OAuth2 token (instead of pass) |
| `folder` | `'INBOX'` | Mailbox folder to watch |
| `markSeen` | `true` | Mark processed emails as read |
| `mode` | `'idle'` | `'idle'` (push) or `'poll'` (pull) |
| `pollInterval` | `30000` | Poll interval in ms (poll mode only) |
| `fetchOnConnect` | `true` | Fetch unseen emails on startup |
| `reconnectDelay` | `5000` | Ms between reconnect attempts |
| `maxReconnects` | `Infinity` | Max reconnect attempts |
| `filter` | — | `(email) => boolean` — skip emails |
| `onUidUpdate` | — | `(uid) => void` — persist UID |
| `lastSeenUid` | `0` | Restore UID from persistence |

## License

MIT