Skill: kolemaravilla/tokenflow (version 1.0.0)

--- name: tokenflow version: 2.0.0 description: "Convert files and URLs to structured text using the TokenFlow API — agent-agnostic with per-filetype configuration" author: Iron Lion International license: MIT triggers: "tokenflow", "convert file", "flow this", "make it flow" env: TOKENFLOW_API_KEY: "Your TokenFlow API key (from dashboard)" --- # TokenFlow **Name:** `tokenflow` **Version:** 2.0.0 **Description:** Convert files and URLs to structured text using the TokenFlow API **Author:** Iron Lion International **License:** MIT --- ## Overview This skill enables agents to automatically convert supported file types and URLs to structured text using the TokenFlow API. Configuration happens agent-side — edit `config.json` to set per-filetype behavior. **Supported Platforms:** OpenClaw, Hermes --- ## Requirements - `TOKENFLOW_API_KEY` — Your TokenFlow API key (required) - Internet access to reach `https://tokenflow.fly.dev` --- ## Installation ### OpenClaw ```bash openclaw skills install tokenflow ``` Or manually copy this skill directory to `~/.openclaw/skills/`. ### Hermes Copy the `tokenflow/` skill directory to `~/.hermes/skills/`. --- ## Configuration Edit `config.json` in the skill directory: ```json { "apiKey": "your-api-key-here", "apiUrl": "https://tokenflow.fly.dev", "filetypeBehavior": { "docx": { "action": "convert", "askEachTime": false }, "xlsx": { "action": "convert", "askEachTime": true }, "pdf": { "action": "convert", "askEachTime": false }, "audio": { "action": "convert", "askEachTime": false } }, "outputFormat": "markdown", "maxRetries": 1, "fallbackBehavior": "use_file" } ``` ### Filetype Behavior | Filetype | Extensions | `action` | `askEachTime` | |----------|-----------|----------|---------------| | `docx` | `.docx`, `.pptx` | `"convert"` or `"skip"` | `true` = prompt user each time | | `xlsx` | `.xlsx`, `.xls`, `.csv` | `"convert"` or `"skip"` | `true` = prompt user each time | | `pdf` | `.pdf` | `"convert"` or `"skip"` | `true` = prompt user each time | | `audio` | `.wav`, `.mp3`, `.ogg`, `.flac`, `.m4a` | `"convert"` or `"skip"` | `true` = prompt user each time | ### Options - `outputFormat`: `"markdown"` (default) or `"html"`. Agents should use `"markdown"`. - `maxRetries`: `0`–`3`. Retry count on failure. - `fallbackBehavior`: `"use_file"` (use original file on failure) or `"fail"` (report error) --- ## Supported File Types - **Documents:** `.pdf`, `.docx`, `.pptx`, `.html`, `.htm` - **Spreadsheets:** `.xlsx`, `.xls`, `.csv` - **Data:** `.json`, `.xml`, `.txt`, `.md` - **Audio:** `.wav`, `.mp3`, `.ogg`, `.flac`, `.m4a` → transcription - **Archives:** `.zip` --- ## Agent Behavior ### On File Attachment ``` 1. Check if extension is supported └─ No → Skip, use file as-is 2. Look up filetypeBehavior in config ├─ action = "skip" → Skip ├─ askEachTime = true and not explicit → Return shouldAsk signal └─ action = "convert" → Proceed 3. For XLSX: call /convert/preview to get sheet names └─ If multiple sheets, pass sheets="all" 4. Call TokenFlow API: POST /convert Headers: Authorization: Bearer {apiKey} Body: multipart/form-data with file + output_format=markdown 5. If success (200): └─ Return structured text to agent 6. If failure or empty result: ├─ Retries < maxRetries → Retry (step 4) ├─ Fallback = use_file → Use original file, log warning └─ Fallback = fail → Report error to user ``` ### On URL Paste 1. Detect URL in user message 2. Call `POST /convert/url` with `{url: "..."}` 3. Return structured text from the webpage --- ## API Integration ### POST /convert ```bash curl -X POST "https://tokenflow.fly.dev/convert" \ -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \ -F "file=@document.pdf" \ -F "output_format=markdown" ``` **Response:** JSON with `markdown` or `html` field. ### POST /convert/preview (XLSX only) ```bash curl -X POST "https://tokenflow.fly.dev/convert/preview" \ -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \ -F "file=@data.xlsx" ``` **Response:** `{"sheets": ["Sheet1", "Sheet2"]}` ### GET /health ```bash curl "https://tokenflow.fly.dev/health" ``` ### GET /usage ```bash curl -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \ "https://tokenflow.fly.dev/usage" ``` ### POST /convert/url ```bash curl -X POST "https://tokenflow.fly.dev/convert/url" \ -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` --- ## Error Handling | Scenario | Behavior | |----------|----------| | API key invalid | Log error, use original file (if fallback=use_file) | | Quota exceeded (429) | Report to user: "TokenFlow quota exceeded" | | Rate limited (429) | Wait 1s, retry once if retries > 0 | | File too large (413) | Skip conversion, use original file | | Unsupported format (400) | Skip conversion, use original file | | API timeout (5s) | Fail fast, use original file | | Empty result | Treat as failure, apply fallback | --- ## Changelog **v2.0.0** (2026-05-13) - Agent-agnostic: supports OpenClaw and Hermes - Configuration moved agent-side (`config.json`) - Per-filetype behavior with `askEachTime` option - PDFs return markdown with base64-embedded images when `outputFormat=markdown` - Standalone image conversion removed (base64 always inflates tokens) **v1.2.0** (2026-05-05) - Added persistent config file - Added automatic first-run onboarding - Added AGENTS.md sync protocol **v1.0.0** (2026-04-27) - Initial release --- ## Support - TokenFlow: https://tokenflow.ironlion.cc - Issues: https://github.com/iron-lion-international/tokenflow/issues

path: tokenflow-v1.0.0/config.js
content: /**
 * TokenFlow Configuration Manager
 * Handles persistent user preferences across sessions
 */

import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const CONFIG_DIR = __dirname;
const CONFIG_PATH = join(CONFIG_DIR, 'config.json');

const DEFAULTS = {
  apiKey: '',
  apiUrl: 'https://tokenflow.fly.dev',
  filetypeBehavior: {
    docx:   { action: 'convert', askEachTime: false },
    xlsx:   { action: 'convert', askEachTime: true },
    pdf:    { action: 'convert', askEachTime: false },
    audio:  { action: 'convert', askEachTime: false },
  },
  outputFormat: 'markdown',
  maxRetries: 1,
  fallbackBehavior: 'use_file',
  firstRunComplete: false,
};

/**
 * Load config from disk, merging with defaults
 * @returns {Object}
 */
export function loadConfig() {
  try {
    if (existsSync(CONFIG_PATH)) {
      const raw = readFileSync(CONFIG_PATH, 'utf-8');
      const saved = JSON.parse(raw);
      return { ...DEFAULTS, ...saved };
    }
  } catch (err) {
    console.warn('TokenFlow config read failed, using defaults:', err.message);
  }
  return { ...DEFAULTS };
}

/**
 * Save config to disk atomically
 * @param {Object} prefs
 */
export function saveConfig(prefs) {
  try {
    if (!existsSync(CONFIG_DIR)) {
      mkdirSync(CONFIG_DIR, { recursive: true });
    }
    const merged = { ...loadConfig(), ...prefs };
    writeFileSync(CONFIG_PATH, JSON.stringify(merged, null, 2), { mode: 0o600 });
  } catch (err) {
    console.error('TokenFlow config save failed:', err.message);
    throw err;
  }
}

/**
 * Check if this is the first time the skill is being used
 * @returns {boolean}
 */
export function isFirstRun() {
  const config = loadConfig();
  return !config.firstRunComplete;
}

/**
 * Mark onboarding as complete
 */
export function markOnboardingComplete() {
  saveConfig({ firstRunComplete: true });
}

/**
 * Get filetype behavior for a given file
 * @param {string} filename
 * @returns {{action: string, askEachTime: boolean}}
 */
export function getFiletypeBehavior(filename) {
  const config = loadConfig();
  const ext = filename.split('.').pop()?.toLowerCase();
  const behaviors = config.filetypeBehavior || DEFAULTS.filetypeBehavior;

  let key = null;
  if (['docx', 'pptx'].includes(ext)) key = 'docx';
  else if (['xlsx', 'xls', 'csv'].includes(ext)) key = 'xlsx';
  else if (ext === 'pdf') key = 'pdf';
  // Note: standalone images are not supported; images are only extracted from PDFs
  else if (['wav', 'mp3', 'ogg', 'flac', 'm4a'].includes(ext)) key = 'audio';

  return key && behaviors[key] ? behaviors[key] : { action: 'convert', askEachTime: false };
}

/**
 * Get AGENTS.md formatted preference summary
 * @returns {string}
 */
export function getAgentsMdSummary() {
  const c = loadConfig();
  const b = c.filetypeBehavior || DEFAULTS.filetypeBehavior;
  return `## TokenFlow Preferences
- API: ${c.apiUrl}
- Output format: ${c.outputFormat}
- Filetype behaviors:
  - DOCX: ${b.docx?.action || 'convert'} (ask: ${b.docx?.askEachTime ? 'yes' : 'no'})
  - XLSX: ${b.xlsx?.action || 'convert'} (ask: ${b.xlsx?.askEachTime ? 'yes' : 'no'})
  - PDF: ${b.pdf?.action || 'convert'} (ask: ${b.pdf?.askEachTime ? 'yes' : 'no'})
  - Note: standalone images are not supported; images are only extracted from PDFs
  - Audio: ${b.audio?.action || 'convert'} (ask: ${b.audio?.askEachTime ? 'yes' : 'no'})
- Fallback: ${c.fallbackBehavior}
`;
}

export { DEFAULTS };
export default { loadConfig, saveConfig, isFirstRun, markOnboardingComplete, getFiletypeBehavior, getAgentsMdSummary };


path: tokenflow-v1.0.0/hermes/DESCRIPTION.md
content: TokenFlow — Convert files and URLs to structured text for AI consumption.

Supports: PDF, DOCX, XLSX, PPTX, HTML, CSV, JSON, XML, TXT, MD, audio (WAV, MP3, OGG, FLAC, M4A), ZIP. Images are extracted from PDFs only.

## Installation

Copy the `tokenflow/` skill directory to `~/.hermes/skills/`, then configure your API key.

## Authentication

If you already have an API key, set it in `~/.hermes/skills/tokenflow/config.json`:

```json
{ "apiKey": "your-api-key-here" }
```

If you don't have an API key, you can sign up directly from the CLI:

```bash
# Sign up (creates account + generates API key)
tokenflow signup --email [REDACTED_SECRET] --password yourpassword

# Or sign in with existing credentials
tokenflow signin --email [REDACTED_SECRET] --password yourpassword
```

Your API key will be saved automatically to the config file.

## Quota & Upgrades

TokenFlow operates on a tiered quota system (Free: 20 docs/mo, Pro: 500/mo, etc.). If you exceed your quota, the skill will return the original file unchanged and notify you. To upgrade your plan, visit: https://tokenflow.fly.dev


path: tokenflow-v1.0.0/skill.json
content: {
  "name": "tokenflow",
  "version": "2.0.0",
  "description": "Convert files and URLs to structured text using the TokenFlow API — agent-agnostic with per-filetype configuration.",
  "author": "Iron Lion International",
  "license": "MIT",
  "keywords": ["tokenflow", "convert", "file", "url", "document", "pdf", "docx", "xlsx", "audio"],
  "commands": [
    {
      "name": "flow",
      "description": "Convert attached file or URL to structured text",
      "usage": "/flow"
    },
    {
      "name": "flow-settings",
      "description": "Show TokenFlow preferences from config.json",
      "usage": "/flow settings"
    }
  ]
}


path: config.js
content: /**
 * TokenFlow Configuration Manager
 * Handles persistent user preferences across sessions
 */

import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const CONFIG_DIR = __dirname;
const CONFIG_PATH = join(CONFIG_DIR, 'config.json');

const DEFAULTS = {
  apiKey: '',
  apiUrl: 'https://tokenflow.fly.dev',
  filetypeBehavior: {
    docx:   { action: 'convert', askEachTime: false },
    xlsx:   { action: 'convert', askEachTime: true },
    pdf:    { action: 'convert', askEachTime: false },
    audio:  { action: 'convert', askEachTime: false },
  },
  outputFormat: 'markdown',
  maxRetries: 1,
  fallbackBehavior: 'use_file',
  firstRunComplete: false,
};

/**
 * Load config from disk, merging with defaults
 * @returns {Object}
 */
export function loadConfig() {
  try {
    if (existsSync(CONFIG_PATH)) {
      const raw = readFileSync(CONFIG_PATH, 'utf-8');
      const saved = JSON.parse(raw);
      return { ...DEFAULTS, ...saved };
    }
  } catch (err) {
    console.warn('TokenFlow config read failed, using defaults:', err.message);
  }
  return { ...DEFAULTS };
}

/**
 * Save config to disk atomically
 * @param {Object} prefs
 */
export function saveConfig(prefs) {
  try {
    if (!existsSync(CONFIG_DIR)) {
      mkdirSync(CONFIG_DIR, { recursive: true });
    }
    const merged = { ...loadConfig(), ...prefs };
    writeFileSync(CONFIG_PATH, JSON.stringify(merged, null, 2), { mode: 0o600 });
  } catch (err) {
    console.error('TokenFlow config save failed:', err.message);
    throw err;
  }
}

/**
 * Check if this is the first time the skill is being used
 * @returns {boolean}
 */
export function isFirstRun() {
  const config = loadConfig();
  return !config.firstRunComplete;
}

/**
 * Mark onboarding as complete
 */
export function markOnboardingComplete() {
  saveConfig({ firstRunComplete: true });
}

/**
 * Get filetype behavior for a given file
 * @param {string} filename
 * @returns {{action: string, askEachTime: boolean}}
 */
export function getFiletypeBehavior(filename) {
  const config = loadConfig();
  const ext = filename.split('.').pop()?.toLowerCase();
  const behaviors = config.filetypeBehavior || DEFAULTS.filetypeBehavior;

  let key = null;
  if (['docx', 'pptx'].includes(ext)) key = 'docx';
  else if (['xlsx', 'xls', 'csv'].includes(ext)) key = 'xlsx';
  else if (ext === 'pdf') key = 'pdf';
  // Note: standalone images are not supported; images are only extracted from PDFs
  else if (['wav', 'mp3', 'ogg', 'flac', 'm4a'].includes(ext)) key = 'audio';

  return key && behaviors[key] ? behaviors[key] : { action: 'convert', askEachTime: false };
}

/**
 * Get AGENTS.md formatted preference summary
 * @returns {string}
 */
export function getAgentsMdSummary() {
  const c = loadConfig();
  const b = c.filetypeBehavior || DEFAULTS.filetypeBehavior;
  return `## TokenFlow Preferences
- API: ${c.apiUrl}
- Output format: ${c.outputFormat}
- Filetype behaviors:
  - DOCX: ${b.docx?.action || 'convert'} (ask: ${b.docx?.askEachTime ? 'yes' : 'no'})
  - XLSX: ${b.xlsx?.action || 'convert'} (ask: ${b.xlsx?.askEachTime ? 'yes' : 'no'})
  - PDF: ${b.pdf?.action || 'convert'} (ask: ${b.pdf?.askEachTime ? 'yes' : 'no'})
  - Note: standalone images are not supported; images are only extracted from PDFs
  - Audio: ${b.audio?.action || 'convert'} (ask: ${b.audio?.askEachTime ? 'yes' : 'no'})
- Fallback: ${c.fallbackBehavior}
`;
}

export { DEFAULTS };
export default { loadConfig, saveConfig, isFirstRun, markOnboardingComplete, getFiletypeBehavior, getAgentsMdSummary };


path: skill.json
content: {
  "name": "tokenflow",
  "version": "2.0.0",
  "description": "Convert files and URLs to structured text using the TokenFlow API — agent-agnostic with per-filetype configuration.",
  "author": "Iron Lion International",
  "license": "MIT",
  "keywords": ["tokenflow", "convert", "file", "url", "document", "pdf", "docx", "xlsx", "audio"],
  "commands": [
    {
      "name": "flow",
      "description": "Convert attached file or URL to structured text",
      "usage": "/flow"
    },
    {
      "name": "flow-settings",
      "description": "Show TokenFlow preferences from config.json",
      "usage": "/flow settings"
    }
  ]
}


path: tokenflow-client.js
content: /**
 * TokenFlow Client
 * Reference implementation for agent skill integration
 *
 * Usage: import { TokenFlowClient } from './tokenflow-client.js'
 */

import { loadConfig, saveConfig, getFiletypeBehavior } from './config.js';

export class TokenFlowClient {
  constructor(apiKey, baseUrl = 'https://tokenflow.fly.dev') {
    this.[REDACTED_SECRET];
    this.baseUrl = baseUrl.replace(/\/$/, '');
  }

  /**
   * Convert a file to structured text
   * @param {File|Blob} file - The file to convert
   * @param {Object} options - Conversion options
   * @param {string} options.sheets - For XLSX: 'all' or comma-separated sheet names
   * @param {string} options.outputFormat - 'html' or 'markdown' (default: 'markdown')
   * @returns {Promise<string|Object>} Text content or multi-sheet results
   */
  async convert(file, options = {}) {
    const formData = new FormData();
    formData.append('file', file);

    if (options.sheets) {
      formData.append('sheets', options.sheets);
    }

    const outputFormat = options.outputFormat || 'markdown';
    formData.append('output_format', outputFormat);

    const response = await fetch(`${this.baseUrl}/convert`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
      body: formData,
    });

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`TokenFlow conversion failed: ${response.status} ${error}`);
    }

    const contentType = response.headers.get('content-type') || '';

    if (contentType.includes('application/json')) {
      return await response.json();
    }

    return await response.text();
  }

  /**
   * Preview XLSX sheet names without converting
   * @param {File|Blob} file - XLSX file
   * @returns {Promise<{filename: string, sheets: string[]}>}
   */
  async preview(file) {
    const formData = new FormData();
    formData.append('file', file);

    const response = await fetch(`${this.baseUrl}/convert/preview`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
      body: formData,
    });

    if (!response.ok) {
      throw new Error(`Preview failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Check API health and features
   * @returns {Promise<Object>}
   */
  async health() {
    const response = await fetch(`${this.baseUrl}/health`);
    return await response.json();
  }

  /**
   * Get current usage statistics
   * @returns {Promise<Object>}
   */
  async usage() {
    const response = await fetch(`${this.baseUrl}/usage`, {
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
    });

    if (!response.ok) {
      throw new Error(`Usage check failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Get billing summary (read-only, no session creation)
   * @returns {Promise<{tier: string, status: string, quota_used: number, quota_limit: number, remaining: number, current_period_end: string|null, reset_date: string}>}
   */
  async billingSummary() {
    const response = await fetch(`${this.baseUrl}/billing/summary`, {
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
    });

    if (!response.ok) {
      throw new Error(`Billing summary failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Convert a URL to structured text
   * @param {string} url - The URL to convert
   * @returns {Promise<{markdown: string, output_tokens: number}>}
   */
  async convertUrl(url) {
    const response = await fetch(`${this.baseUrl}/convert/url`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ url }),
    });

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`URL conversion failed: ${response.status} ${error}`);
    }

    return await response.json();
  }

  /**
   * Sign up with email/password (CLI auth)
   * @param {string} email
   * @param {string} password
   * @returns {Promise<{[REDACTED_SECRET], user_id: string, email: string}>}
   */
  async signup(email, password) {
    const response = await fetch(`${this.baseUrl}/auth/signup`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ email, password }),
    });
    const data = await response.json();
    if (!response.ok) {
      throw new Error(`Signup failed: ${data.detail || response.statusText}`);
    }
    this.[REDACTED_SECRET];
    saveConfig({ [REDACTED_SECRET] });
    return data;
  }

  /**
   * Sign in with email/password (CLI auth)
   * @param {string} email
   * @param {string} password
   * @returns {Promise<{[REDACTED_SECRET], user_id: string, email: string}>}
   */
  async signin(email, password) {
    const response = await fetch(`${this.baseUrl}/auth/signin`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ email, password }),
    });
    const data = await response.json();
    if (!response.ok) {
      throw new Error(`Signin failed: ${data.detail || response.statusText}`);
    }
    this.[REDACTED_SECRET];
    saveConfig({ [REDACTED_SECRET] });
    return data;
  }

  /**
   * Check current quota usage
   * @returns {Promise<{quota_used: number, quota_limit: number, tier: string}>}
   */
  async checkQuota() {
    return this.usage();
  }

  /**
   * Check if file type is supported
   * @param {string} filename - Filename with extension
   * @returns {boolean}
   */
  isSupported(filename) {
    const ext = filename.split('.').pop()?.toLowerCase();
    const supported = [
      'pdf', 'docx', 'xlsx', 'pptx', 'html', 'htm',
      'csv', 'json', 'xml', 'txt', 'md',
      'wav', 'mp3', 'ogg', 'flac', 'm4a',
      'zip'
    ];
    return supported.includes(ext);
  }
}

/**
 * Agent-side file processing with user preferences
 */
export class TokenFlowAgent {
  constructor(client, preferences = {}) {
    this.client = client;

    // Auto-load saved preferences if none explicitly provided
    const saved = Object.keys(preferences).length === 0 ? loadConfig() : {};

    this.prefs = {
      outputFormat: preferences.outputFormat || saved.outputFormat || 'markdown',
      maxRetries: parseInt(preferences.maxRetries) || saved.maxRetries || 1,
      fallbackBehavior: preferences.fallbackBehavior || saved.fallbackBehavior || 'use_file',
      ...preferences
    };
  }

  /**
   * Process a file attachment based on user preferences
   * @param {File} file - The attached file
   * @param {boolean} explicitRequest - Whether user explicitly asked to convert
   * @returns {Promise<{content: string, source: string, success: boolean, shouldAsk?: boolean}>}
   */
  async processFile(file, explicitRequest = false) {
    // Check if supported
    if (!this.client.isSupported(file.name)) {
      return { content: null, source: 'original', success: true };
    }

    // Check filetype behavior
    const behavior = getFiletypeBehavior(file.name);

    if (behavior.action === 'skip') {
      return { content: null, source: 'original', success: true };
    }

    if (behavior.askEachTime && !explicitRequest) {
      return { content: null, source: 'original', success: true, shouldAsk: true };
    }

    // XLSX sheet handling
    let sheets = null;
    const ext = file.name.split('.').pop()?.toLowerCase();
    if (ext === 'xlsx' || ext === 'xls') {
      try {
        const preview = await this.client.preview(file);
        if (preview.sheets && preview.sheets.length > 1) {
          sheets = 'all';
        }
      } catch (e) {
        // Preview failed, proceed without sheets
      }
    }

    // Attempt conversion with retries
    let lastError = null;
    for (let attempt = 0; attempt <= this.prefs.maxRetries; attempt++) {
      try {
        const result = await this.client.convert(file, {
          sheets,
          outputFormat: this.prefs.outputFormat,
        });

        // Extract content from response
        let content = '';
        if (typeof result === 'string') {
          content = result;
        } else if (result.markdown) {
          content = result.markdown;
        } else if (result.html) {
          content = result.html;
        } else if (Array.isArray(result.markdown)) {
          content = result.markdown.map(r => `## ${r.sheet_name}\n\n${r.markdown}`).join('\n\n---\n\n');
        }

        if (!content || content.trim().length === 0) {
          throw new Error('Empty conversion result');
        }

        return { content, source: 'tokenflow', success: true };
      } catch (error) {
        lastError = error;
        if (attempt < this.prefs.maxRetries) {
          await new Promise(r => setTimeout(r, 1000));
        }
      }
    }

    // All retries exhausted
    if (this.prefs.fallbackBehavior === 'fail') {
      throw new Error(`TokenFlow conversion failed after ${this.prefs.maxRetries + 1} attempts: ${lastError.message}`);
    }

    // Check for quota exceeded
    const isQuotaError = lastError.message && (
      lastError.message.includes('quota') ||
      lastError.message.includes('429') ||
      lastError.message.includes('rate limit')
    );

    // Fallback: use original file
    return {
      content: null,
      source: 'original',
      success: false,
      error: lastError.message,
      quotaExceeded: isQuotaError,
      upgradeUrl: isQuotaError ? 'https://tokenflow.fly.dev' : undefined,
    };
  }
}

export default TokenFlowClient;


path: hermes/DESCRIPTION.md
content: TokenFlow — Convert files and URLs to structured text for AI consumption.

Supports: PDF, DOCX, XLSX, PPTX, HTML, CSV, JSON, XML, TXT, MD, audio (WAV, MP3, OGG, FLAC, M4A), ZIP. Images are extracted from PDFs only.

## Installation

Copy the `tokenflow/` skill directory to `~/.hermes/skills/`, then configure your API key.

## Authentication

If you already have an API key, set it in `~/.hermes/skills/tokenflow/config.json`:

```json
{ "apiKey": "your-api-key-here" }
```

If you don't have an API key, you can sign up directly from the CLI:

```bash
# Sign up (creates account + generates API key)
tokenflow signup --email [REDACTED_SECRET] --password yourpassword

# Or sign in with existing credentials
tokenflow signin --email [REDACTED_SECRET] --password yourpassword
```

Your API key will be saved automatically to the config file.

## Quota & Upgrades

TokenFlow operates on a tiered quota system (Free: 20 docs/mo, Pro: 500/mo, etc.). If you exceed your quota, the skill will return the original file unchanged and notify you. To upgrade your plan, visit: https://tokenflow.fly.dev


path: tokenflow-v1.0.0/tokenflow-client.js
content: /**
 * TokenFlow Client
 * Reference implementation for agent skill integration
 *
 * Usage: import { TokenFlowClient } from './tokenflow-client.js'
 */

import { loadConfig, saveConfig, getFiletypeBehavior } from './config.js';

export class TokenFlowClient {
  constructor(apiKey, baseUrl = 'https://tokenflow.fly.dev') {
    this.[REDACTED_SECRET];
    this.baseUrl = baseUrl.replace(/\/$/, '');
  }

  /**
   * Convert a file to structured text
   * @param {File|Blob} file - The file to convert
   * @param {Object} options - Conversion options
   * @param {string} options.sheets - For XLSX: 'all' or comma-separated sheet names
   * @param {string} options.outputFormat - 'html' or 'markdown' (default: 'markdown')
   * @returns {Promise<string|Object>} Text content or multi-sheet results
   */
  async convert(file, options = {}) {
    const formData = new FormData();
    formData.append('file', file);

    if (options.sheets) {
      formData.append('sheets', options.sheets);
    }

    const outputFormat = options.outputFormat || 'markdown';
    formData.append('output_format', outputFormat);

    const response = await fetch(`${this.baseUrl}/convert`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
      body: formData,
    });

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`TokenFlow conversion failed: ${response.status} ${error}`);
    }

    const contentType = response.headers.get('content-type') || '';

    if (contentType.includes('application/json')) {
      return await response.json();
    }

    return await response.text();
  }

  /**
   * Preview XLSX sheet names without converting
   * @param {File|Blob} file - XLSX file
   * @returns {Promise<{filename: string, sheets: string[]}>}
   */
  async preview(file) {
    const formData = new FormData();
    formData.append('file', file);

    const response = await fetch(`${this.baseUrl}/convert/preview`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
      body: formData,
    });

    if (!response.ok) {
      throw new Error(`Preview failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Check API health and features
   * @returns {Promise<Object>}
   */
  async health() {
    const response = await fetch(`${this.baseUrl}/health`);
    return await response.json();
  }

  /**
   * Get current usage statistics
   * @returns {Promise<Object>}
   */
  async usage() {
    const response = await fetch(`${this.baseUrl}/usage`, {
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
    });

    if (!response.ok) {
      throw new Error(`Usage check failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Get billing summary (read-only, no session creation)
   * @returns {Promise<{tier: string, status: string, quota_used: number, quota_limit: number, remaining: number, current_period_end: string|null, reset_date: string}>}
   */
  async billingSummary() {
    const response = await fetch(`${this.baseUrl}/billing/summary`, {
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
      },
    });

    if (!response.ok) {
      throw new Error(`Billing summary failed: ${response.status}`);
    }

    return await response.json();
  }

  /**
   * Convert a URL to structured text
   * @param {string} url - The URL to convert
   * @returns {Promise<{markdown: string, output_tokens: number}>}
   */
  async convertUrl(url) {
    const response = await fetch(`${this.baseUrl}/convert/url`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${this.apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ url }),
    });

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`URL conversion failed: ${response.status} ${error}`);
    }

    return await response.json();
  }

  /**
   * Sign up with email/password (CLI auth)
   * @param {string} email
   * @param {string} password
   * @returns {Promise<{[REDACTED_SECRET], user_id: string, email: string}>}
   */
  async signup(email, password) {
    const response = await fetch(`${this.baseUrl}/auth/signup`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ email, password }),
    });
    const data = await response.json();
    if (!response.ok) {
      throw new Error(`Signup failed: ${data.detail || response.statusText}`);
    }
    this.[REDACTED_SECRET];
    saveConfig({ [REDACTED_SECRET] });
    return data;
  }

  /**
   * Sign in with email/password (CLI auth)
   * @param {string} email
   * @param {string} password
   * @returns {Promise<{[REDACTED_SECRET], user_id: string, email: string}>}
   */
  async signin(email, password) {
    const response = await fetch(`${this.baseUrl}/auth/signin`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ email, password }),
    });
    const data = await response.json();
    if (!response.ok) {
      throw new Error(`Signin failed: ${data.detail || response.statusText}`);
    }
    this.[REDACTED_SECRET];
    saveConfig({ [REDACTED_SECRET] });
    return data;
  }

  /**
   * Check current quota usage
   * @returns {Promise<{quota_used: number, quota_limit: number, tier: string}>}
   */
  async checkQuota() {
    return this.usage();
  }

  /**
   * Check if file type is supported
   * @param {string} filename - Filename with extension
   * @returns {boolean}
   */
  isSupported(filename) {
    const ext = filename.split('.').pop()?.toLowerCase();
    const supported = [
      'pdf', 'docx', 'xlsx', 'pptx', 'html', 'htm',
      'csv', 'json', 'xml', 'txt', 'md',
      'wav', 'mp3', 'ogg', 'flac', 'm4a',
      'zip'
    ];
    return supported.includes(ext);
  }
}

/**
 * Agent-side file processing with user preferences
 */
export class TokenFlowAgent {
  constructor(client, preferences = {}) {
    this.client = client;

    // Auto-load saved preferences if none explicitly provided
    const saved = Object.keys(preferences).length === 0 ? loadConfig() : {};

    this.prefs = {
      outputFormat: preferences.outputFormat || saved.outputFormat || 'markdown',
      maxRetries: parseInt(preferences.maxRetries) || saved.maxRetries || 1,
      fallbackBehavior: preferences.fallbackBehavior || saved.fallbackBehavior || 'use_file',
      ...preferences
    };
  }

  /**
   * Process a file attachment based on user preferences
   * @param {File} file - The attached file
   * @param {boolean} explicitRequest - Whether user explicitly asked to convert
   * @returns {Promise<{content: string, source: string, success: boolean, shouldAsk?: boolean}>}
   */
  async processFile(file, explicitRequest = false) {
    // Check if supported
    if (!this.client.isSupported(file.name)) {
      return { content: null, source: 'original', success: true };
    }

    // Check filetype behavior
    const behavior = getFiletypeBehavior(file.name);

    if (behavior.action === 'skip') {
      return { content: null, source: 'original', success: true };
    }

    if (behavior.askEachTime && !explicitRequest) {
      return { content: null, source: 'original', success: true, shouldAsk: true };
    }

    // XLSX sheet handling
    let sheets = null;
    const ext = file.name.split('.').pop()?.toLowerCase();
    if (ext === 'xlsx' || ext === 'xls') {
      try {
        const preview = await this.client.preview(file);
        if (preview.sheets && preview.sheets.length > 1) {
          sheets = 'all';
        }
      } catch (e) {
        // Preview failed, proceed without sheets
      }
    }

    // Attempt conversion with retries
    let lastError = null;
    for (let attempt = 0; attempt <= this.prefs.maxRetries; attempt++) {
      try {
        const result = await this.client.convert(file, {
          sheets,
          outputFormat: this.prefs.outputFormat,
        });

        // Extract content from response
        let content = '';
        if (typeof result === 'string') {
          content = result;
        } else if (result.markdown) {
          content = result.markdown;
        } else if (result.html) {
          content = result.html;
        } else if (Array.isArray(result.markdown)) {
          content = result.markdown.map(r => `## ${r.sheet_name}\n\n${r.markdown}`).join('\n\n---\n\n');
        }

        if (!content || content.trim().length === 0) {
          throw new Error('Empty conversion result');
        }

        return { content, source: 'tokenflow', success: true };
      } catch (error) {
        lastError = error;
        if (attempt < this.prefs.maxRetries) {
          await new Promise(r => setTimeout(r, 1000));
        }
      }
    }

    // All retries exhausted
    if (this.prefs.fallbackBehavior === 'fail') {
      throw new Error(`TokenFlow conversion failed after ${this.prefs.maxRetries + 1} attempts: ${lastError.message}`);
    }

    // Check for quota exceeded
    const isQuotaError = lastError.message && (
      lastError.message.includes('quota') ||
      lastError.message.includes('429') ||
      lastError.message.includes('rate limit')
    );

    // Fallback: use original file
    return {
      content: null,
      source: 'original',
      success: false,
      error: lastError.message,
      quotaExceeded: isQuotaError,
      upgradeUrl: isQuotaError ? 'https://tokenflow.fly.dev' : undefined,
    };
  }
}

export default TokenFlowClient;


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal:  | SkillSpector: suspicious (CRITICAL)