Skill: imgolye/agent-task-queue (version 0.1.0)

--- name: agent-task-queue description: Build and operate a multi-agent task queue in OpenClaw with priority queues, delayed/dead-letter queues, scheduling, retry/timeout control, dependency management, parallel execution, and execution tracking using the bundled TypeScript runtime. --- # Agent Task Queue Use this skill when a task needs queue-based orchestration for multiple agents or workers. The bundled runtime is in `src/` and covers: - Priority queue, delayed queue, and dead-letter queue behavior - Scheduler polling with concurrency limits, retries, and timeouts - Dependency validation, dependency gating, and result propagation - Task logs and aggregate metrics - Pluggable storage through in-memory, SQLite, or Redis backends ## Workflow 1. Import `TaskQueue` and `Scheduler` from `src/index.ts`. 2. Pick storage: - Default `InMemoryStorage` for local execution or tests - `SQLiteStorage` for single-node persistence - `RedisStorage` for distributed workers 3. Register handlers with `scheduler.register(taskType, handler)`. 4. Enqueue tasks with `priority`, `runAt`, `dependencies`, `retryPolicy`, and `timeoutMs` as needed. 5. Drive execution via `scheduler.tick()` or `scheduler.start()`. 6. Inspect `queue.logs()`, `queue.metrics()`, `queue.getSnapshot()`, and `queue.get(taskId)` for state and traceability. ## Key Files - `src/TaskQueue.ts`: queue lifecycle, ready/dead-letter snapshots, logs, metrics - `src/Scheduler.ts`: polling loop, concurrency control, retries, timeout handling - `src/DependencyManager.ts`: DAG validation and dependency result propagation - `src/storage/`: storage implementations - `tests/task-queue.test.ts`: behavior coverage - `examples/basic.ts`: end-to-end usage ## Implementation Notes - Dependency tasks must already exist when a dependent task is enqueued. - A task becomes runnable only after all dependencies are `completed`. - Completed dependency results are stored and exposed to downstream handlers through `context.dependencies`. - When retries are exhausted, the task is moved to `dead_letter`. - Timeout cancellation uses `AbortSignal`; long-running handlers should watch `context.signal`. ## Validation Run: ```bash npm run check ``` If Redis or SQLite packages are unavailable in the environment, install dependencies first with `npm install`.

path: README.md
content: # agent-task-queue

`agent-task-queue` is an OpenClaw skill project for multi-agent task orchestration. It provides a TypeScript queue runtime with priority ordering, delayed execution, dependency-aware scheduling, retries, timeout handling, logs, and metrics.

## Features

- Priority queue with delayed tasks and dead-letter handling
- Scheduler with polling, concurrency limits, retries, and task timeouts
- Dependency manager with DAG validation, dependency gating, and result propagation
- Storage abstraction with in-memory, SQLite, and Redis implementations
- Execution logs and queue metrics for observability

## Project Structure

```text
.
├── SKILL.md
├── README.md
├── examples/
├── src/
│   ├── DependencyManager.ts
│   ├── Scheduler.ts
│   ├── TaskQueue.ts
│   └── storage/
└── tests/
```

## Install

```bash
npm install
```

## Use

```ts
import { Scheduler, TaskQueue } from "./src/index.js";

const queue = new TaskQueue();
const scheduler = new Scheduler(queue, { concurrency: 2 });

scheduler.register("echo", async (payload) => ({ echoed: payload }));

await queue.enqueue({
  type: "echo",
  payload: { value: "hello" },
  priority: 5,
  retryPolicy: { maxAttempts: 3, backoffMs: 500 }
});

await scheduler.tick();
```

For a fuller example, see [examples/basic.ts](/Users/gaolei/.openclaw/workspace-zhongshu/skills/agent-task-queue/examples/basic.ts).

## Scripts

- `npm run build` compiles the project
- `npm test` runs the test suite
- `npm run check` runs build plus tests

## Storage

- `InMemoryStorage` is the default for tests and local runs
- `SQLiteStorage` persists tasks and logs to a local SQLite database
- `RedisStorage` supports shared queue state for distributed workers

## Notes

- Dependencies must refer to existing tasks and cannot form cycles
- Exhausted retries move tasks to the dead-letter queue
- Timeout handling uses `AbortController`; task handlers should respect `context.signal`


path: examples/basic.ts
content: import { Scheduler, TaskQueue } from "../src/index.js";

async function main(): Promise<void> {
  const queue = new TaskQueue();
  const scheduler = new Scheduler(queue, { concurrency: 2, pollIntervalMs: 50 });

  scheduler.register("fetch-context", async (payload, context) => {
    await context.log("info", "Fetching context", payload as Record<string, unknown>);
    return {
      summary: "Repository state loaded",
      branch: "main"
    };
  });

  scheduler.register("write-report", async (payload, context) => {
    const upstream = context.dependencies.get("task-context") as { summary: string; branch: string } | undefined;
    return {
      message: `Report for ${String((payload as { task: string }).task)}`,
      upstream
    };
  });

  await queue.enqueue({
    id: "task-context",
    type: "fetch-context",
    payload: { repo: "agent-task-queue" },
    priority: 10
  });

  await queue.enqueue({
    id: "task-report",
    type: "write-report",
    payload: { task: "daily-sync" },
    dependencies: ["task-context"]
  });

  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 25));
  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 25));

  console.log(await queue.metrics());
  console.log(await queue.get("task-report"));
}

void main();


path: package.json
content: {
  "name": "agent-task-queue",
  "version": "0.1.0",
  "description": "OpenClaw skill for multi-agent task queue management with priority, dependencies, retries, and execution tracking.",
  "type": "module",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "files": [
    "dist",
    "SKILL.md",
    "README.md",
    "examples"
  ],
  "scripts": {
    "build": "tsc -p tsconfig.json",
    "clean": "rm -rf dist",
    "test": "node --import tsx --test tests/**/*.test.ts",
    "check": "npm run build && npm run test"
  },
  "keywords": [
    "openclaw",
    "skill",
    "task-queue",
    "scheduler",
    "multi-agent"
  ],
  "license": "MIT",
  "engines": {
    "node": ">=20"
  },
  "dependencies": {
    "better-sqlite3": "^11.9.0",
    "redis": "^5.1.0"
  },
  "devDependencies": {
    "@types/better-sqlite3": "^7.6.13",
    "@types/node": "^22.13.10",
    "tsx": "^4.19.3",
    "typescript": "^5.8.2"
  }
}


path: src/DependencyManager.ts
content: import type { QueueStorage, TaskRecord } from "./types.js";

export class DependencyManager {
  constructor(private readonly storage: QueueStorage) {}

  async validateTask(task: TaskRecord): Promise<void> {
    if (task.dependencies.includes(task.id)) {
      throw new Error(`Task ${task.id} cannot depend on itself`);
    }

    for (const dependencyId of task.dependencies) {
      const dependency = await this.storage.getTask(dependencyId);
      if (!dependency) {
        throw new Error(`Task ${task.id} depends on missing task ${dependencyId}`);
      }
    }

    const allTasks = await this.storage.listTasks();
    const graph = new Map<string, string[]>();
    for (const item of allTasks) {
      graph.set(item.id, [...item.dependencies]);
    }
    graph.set(task.id, [...task.dependencies]);
    this.assertNoCycle(graph, task.id);
  }

  async areDependenciesSatisfied(task: TaskRecord): Promise<boolean> {
    for (const dependencyId of task.dependencies) {
      const dependency = await this.storage.getTask(dependencyId);
      if (!dependency || dependency.status !== "completed") {
        return false;
      }
    }
    return true;
  }

  async persistDependencyResults(task: TaskRecord): Promise<void> {
    for (const dependencyId of task.dependencies) {
      const dependency = await this.storage.getTask(dependencyId);
      if (dependency?.status === "completed") {
        await this.storage.saveDependencyResult(task.id, dependencyId, dependency.result);
      }
    }
  }

  async markBlockedDependents(completedTaskId: string): Promise<void> {
    const tasks = await this.storage.listTasks();
    for (const task of tasks) {
      if (!task.dependencies.includes(completedTaskId)) {
        continue;
      }
      if (task.status !== "waiting_dependencies") {
        continue;
      }
      if (!(await this.areDependenciesSatisfied(task))) {
        continue;
      }
      await this.persistDependencyResults(task);
      task.status = new Date(task.runAt).getTime() <= Date.now() ? "queued" : "retry_scheduled";
      task.updatedAt = new Date().toISOString();
      await this.storage.updateTask(task);
      await this.storage.appendLog({
        taskId: task.id,
        level: "info",
        message: "All dependencies satisfied",
        timestamp: task.updatedAt,
        context: { completedTaskId }
      });
    }
  }

  private assertNoCycle(graph: Map<string, string[]>, start: string): void {
    const visited = new Set<string>();
    const path = new Set<string>();
    const visit = (node: string): void => {
      if (path.has(node)) {
        throw new Error(`Dependency cycle detected at task ${node}`);
      }
      if (visited.has(node)) {
        return;
      }
      visited.add(node);
      path.add(node);
      for (const dependency of graph.get(node) ?? []) {
        visit(dependency);
      }
      path.delete(node);
    };
    visit(start);
  }
}


path: src/Scheduler.ts
content: import type { TaskHandler, TaskRecord } from "./types.js";
import { TaskQueue } from "./TaskQueue.js";

export interface SchedulerOptions {
  pollIntervalMs?: number;
  concurrency?: number;
}

export class Scheduler {
  private readonly handlers = new Map<string, TaskHandler>();
  private readonly pollIntervalMs: number;
  private readonly concurrency: number;
  private timer?: NodeJS.Timeout;
  private activeCount = 0;
  private draining = false;

  constructor(private readonly queue: TaskQueue, options: SchedulerOptions = {}) {
    this.pollIntervalMs = options.pollIntervalMs ?? 250;
    this.concurrency = options.concurrency ?? 4;
  }

  register<TPayload, TResult>(taskType: string, handler: TaskHandler<TPayload, TResult>): void {
    this.handlers.set(taskType, handler as TaskHandler);
  }

  start(): void {
    if (this.timer) {
      return;
    }
    this.timer = setInterval(() => {
      void this.tick();
    }, this.pollIntervalMs);
  }

  async stop(): Promise<void> {
    this.draining = true;
    if (this.timer) {
      clearInterval(this.timer);
      this.timer = undefined;
    }
    while (this.activeCount > 0) {
      await new Promise((resolve) => setTimeout(resolve, 10));
    }
  }

  async tick(): Promise<void> {
    if (this.draining) {
      return;
    }
    while (this.activeCount < this.concurrency) {
      const task = await this.queue.claimNextReady();
      if (!task) {
        return;
      }
      this.activeCount += 1;
      void this.execute(task).finally(() => {
        this.activeCount -= 1;
      });
    }
  }

  private async execute(task: TaskRecord): Promise<void> {
    const handler = this.handlers.get(task.type);
    if (!handler) {
      await this.queue.fail(task.id, `No handler registered for task type "${task.type}"`);
      return;
    }
    const controller = new AbortController();
    const timeout = task.timeoutMs
      ? setTimeout(() => controller.abort(new Error(`Task ${task.id} timed out after ${task.timeoutMs}ms`)), task.timeoutMs)
      : undefined;
    try {
      const dependencies = await this.queue.storage.getDependencyResults(task.id);
      const result = await handler(task.payload, {
        attempt: task.attempts,
        signal: controller.signal,
        dependencies,
        log: async (level, message, context) => this.queue.log(task.id, level, message, context)
      });
      await this.queue.complete(task.id, result);
    } catch (error) {
      const reason = error instanceof Error ? error : new Error(String(error));
      await this.queue.fail(task.id, reason);
    } finally {
      if (timeout) {
        clearTimeout(timeout);
      }
    }
  }
}


path: src/TaskQueue.ts
content: import { DependencyManager } from "./DependencyManager.js";
import { InMemoryStorage } from "./storage/InMemoryStorage.js";
import { compareReadyTasks, computeRetryAt, createTaskRecord } from "./utils.js";
import type {
  QueueSnapshot,
  QueueStorage,
  TaskLogEntry,
  TaskMetrics,
  TaskOptions,
  TaskRecord,
  TaskStatus
} from "./types.js";

const TERMINAL_STATUSES: TaskStatus[] = ["completed", "failed", "dead_letter", "cancelled"];

export class TaskQueue {
  readonly storage: QueueStorage;
  readonly dependencies: DependencyManager;

  constructor(storage: QueueStorage = new InMemoryStorage()) {
    this.storage = storage;
    this.dependencies = new DependencyManager(storage);
  }

  async enqueue<TPayload>(options: TaskOptions<TPayload>): Promise<TaskRecord<TPayload>> {
    const task = createTaskRecord(options);
    await this.dependencies.validateTask(task);
    if (task.status === "waiting_dependencies" && (await this.dependencies.areDependenciesSatisfied(task))) {
      await this.dependencies.persistDependencyResults(task);
      task.status = new Date(task.runAt).getTime() <= Date.now() ? "queued" : "retry_scheduled";
    }
    await this.storage.saveTask(task);
    await this.log(task.id, "info", "Task enqueued", {
      priority: task.priority,
      runAt: task.runAt,
      dependencies: task.dependencies
    });
    return task;
  }

  async claimNextReady(): Promise<TaskRecord | undefined> {
    const tasks = await this.storage.listTasks();
    const now = Date.now();
    const ready = tasks
      .filter((task) => ["queued", "retry_scheduled"].includes(task.status))
      .filter((task) => new Date(task.runAt).getTime() <= now)
      .sort(compareReadyTasks);
    const next = ready[0];
    if (!next) {
      return undefined;
    }
    next.status = "running";
    next.attempts += 1;
    next.startedAt = new Date().toISOString();
    next.updatedAt = next.startedAt;
    await this.storage.updateTask(next);
    await this.log(next.id, "info", "Task claimed for execution", { attempt: next.attempts });
    return next;
  }

  async complete(taskId: string, result: unknown): Promise<TaskRecord> {
    const task = await this.mustGet(taskId);
    task.status = "completed";
    task.result = result;
    task.completedAt = new Date().toISOString();
    task.updatedAt = task.completedAt;
    await this.storage.updateTask(task);
    await this.log(taskId, "info", "Task completed", { result });
    await this.dependencies.markBlockedDependents(taskId);
    return task;
  }

  async fail(taskId: string, error: Error | string): Promise<TaskRecord> {
    const task = await this.mustGet(taskId);
    const message = typeof error === "string" ? error : error.message;
    task.lastError = message;
    task.updatedAt = new Date().toISOString();
    if (task.attempts >= task.retryPolicy.maxAttempts) {
      task.status = "dead_letter";
      task.completedAt = task.updatedAt;
    } else {
      task.status = "retry_scheduled";
      task.runAt = computeRetryAt(task).toISOString();
    }
    await this.storage.updateTask(task);
    await this.log(taskId, "error", "Task failed", {
      error: message,
      nextRunAt: task.status === "retry_scheduled" ? task.runAt : undefined
    });
    return task;
  }

  async cancel(taskId: string, reason = "Cancelled"): Promise<TaskRecord> {
    const task = await this.mustGet(taskId);
    if (TERMINAL_STATUSES.includes(task.status)) {
      return task;
    }
    task.status = "cancelled";
    task.lastError = reason;
    task.updatedAt = new Date().toISOString();
    task.completedAt = task.updatedAt;
    await this.storage.updateTask(task);
    await this.log(taskId, "warn", "Task cancelled", { reason });
    return task;
  }

  async get(taskId: string): Promise<TaskRecord | undefined> {
    return this.storage.getTask(taskId);
  }

  async list(status?: TaskStatus): Promise<TaskRecord[]> {
    const tasks = await this.storage.listTasks();
    return tasks
      .filter((task) => !status || task.status === status)
      .sort(compareReadyTasks);
  }

  async getSnapshot(): Promise<QueueSnapshot> {
    const tasks = await this.storage.listTasks();
    const now = Date.now();
    return {
      ready: tasks
        .filter((task) => ["queued", "retry_scheduled"].includes(task.status) && new Date(task.runAt).getTime() <= now)
        .sort(compareReadyTasks),
      delayed: tasks
        .filter((task) => ["queued", "retry_scheduled"].includes(task.status) && new Date(task.runAt).getTime() > now)
        .sort(compareReadyTasks),
      deadLetter: tasks.filter((task) => task.status === "dead_letter").sort(compareReadyTasks)
    };
  }

  async metrics(): Promise<TaskMetrics> {
    const tasks = await this.storage.listTasks();
    const completed = tasks.filter((task) => task.status === "completed");
    const avgDurationMs =
      completed.reduce((sum, task) => {
        if (!task.startedAt || !task.completedAt) {
          return sum;
        }
        return sum + (new Date(task.completedAt).getTime() - new Date(task.startedAt).getTime());
      }, 0) / (completed.length || 1);
    return {
      queued: tasks.filter((task) => task.status === "queued").length,
      running: tasks.filter((task) => task.status === "running").length,
      waitingDependencies: tasks.filter((task) => task.status === "waiting_dependencies").length,
      retryScheduled: tasks.filter((task) => task.status === "retry_scheduled").length,
      completed: completed.length,
      failed: tasks.filter((task) => task.status === "failed").length,
      deadLetter: tasks.filter((task) => task.status === "dead_letter").length,
      avgDurationMs: Number.isFinite(avgDurationMs) ? avgDurationMs : 0
    };
  }

  async logs(taskId?: string): Promise<TaskLogEntry[]> {
    return this.storage.listLogs(taskId);
  }

  async log(taskId: string, level: TaskLogEntry["level"], message: string, context?: Record<string, unknown>): Promise<void> {
    await this.storage.appendLog({
      taskId,
      level,
      message,
      timestamp: new Date().toISOString(),
      context
    });
  }

  private async mustGet(taskId: string): Promise<TaskRecord> {
    const task = await this.storage.getTask(taskId);
    if (!task) {
      throw new Error(`Task ${taskId} not found`);
    }
    return task;
  }
}


path: src/index.ts
content: export * from "./DependencyManager.js";
export * from "./Scheduler.js";
export * from "./TaskQueue.js";
export * from "./types.js";
export * from "./storage/InMemoryStorage.js";
export * from "./storage/RedisStorage.js";
export * from "./storage/SQLiteStorage.js";


path: src/storage/InMemoryStorage.ts
content: import type { QueueStorage, TaskLogEntry, TaskRecord } from "../types.js";

export class InMemoryStorage implements QueueStorage {
  private readonly tasks = new Map<string, TaskRecord>();
  private readonly logs: TaskLogEntry[] = [];
  private readonly dependencyResults = new Map<string, Map<string, unknown>>();

  async saveTask(task: TaskRecord): Promise<void> {
    this.tasks.set(task.id, structuredClone(task));
  }

  async getTask(taskId: string): Promise<TaskRecord | undefined> {
    const task = this.tasks.get(taskId);
    return task ? structuredClone(task) : undefined;
  }

  async updateTask(task: TaskRecord): Promise<void> {
    this.tasks.set(task.id, structuredClone(task));
  }

  async listTasks(): Promise<TaskRecord[]> {
    return [...this.tasks.values()].map((task) => structuredClone(task));
  }

  async listLogs(taskId?: string): Promise<TaskLogEntry[]> {
    return this.logs
      .filter((entry) => !taskId || entry.taskId === taskId)
      .map((entry) => structuredClone(entry));
  }

  async appendLog(entry: TaskLogEntry): Promise<void> {
    this.logs.push(structuredClone(entry));
  }

  async saveDependencyResult(taskId: string, dependencyId: string, value: unknown): Promise<void> {
    const taskResults = this.dependencyResults.get(taskId) ?? new Map<string, unknown>();
    taskResults.set(dependencyId, structuredClone(value));
    this.dependencyResults.set(taskId, taskResults);
  }

  async getDependencyResults(taskId: string): Promise<Map<string, unknown>> {
    const taskResults = this.dependencyResults.get(taskId) ?? new Map<string, unknown>();
    return new Map<string, unknown>([...taskResults.entries()].map(([key, value]) => [key, structuredClone(value)]));
  }
}


path: src/storage/RedisStorage.ts
content: import { createClient, type RedisClientType } from "redis";
import type { QueueStorage, TaskLogEntry, TaskRecord } from "../types.js";

export interface RedisStorageOptions {
  url?: string;
  namespace?: string;
}

export class RedisStorage implements QueueStorage {
  private readonly client: RedisClientType;
  private readonly namespace: string;

  constructor(options: RedisStorageOptions = {}) {
    this.client = createClient({ url: options.url });
    this.namespace = options.namespace ?? "agent-task-queue";
  }

  async connect(): Promise<void> {
    if (!this.client.isOpen) {
      await this.client.connect();
    }
  }

  async disconnect(): Promise<void> {
    if (this.client.isOpen) {
      await this.client.disconnect();
    }
  }

  async saveTask(task: TaskRecord): Promise<void> {
    await this.connect();
    await this.client.hSet(this.key("tasks"), task.id, JSON.stringify(task));
  }

  async getTask(taskId: string): Promise<TaskRecord | undefined> {
    await this.connect();
    const raw = await this.client.hGet(this.key("tasks"), taskId);
    return raw ? (JSON.parse(raw) as TaskRecord) : undefined;
  }

  async updateTask(task: TaskRecord): Promise<void> {
    await this.saveTask(task);
  }

  async listTasks(): Promise<TaskRecord[]> {
    await this.connect();
    const entries = await this.client.hVals(this.key("tasks"));
    return entries.map((entry) => JSON.parse(entry) as TaskRecord);
  }

  async listLogs(taskId?: string): Promise<TaskLogEntry[]> {
    await this.connect();
    const entries = taskId
      ? await this.client.lRange(this.key(`logs:${taskId}`), 0, -1)
      : await this.client.lRange(this.key("logs"), 0, -1);
    return entries.map((entry) => JSON.parse(entry) as TaskLogEntry);
  }

  async appendLog(entry: TaskLogEntry): Promise<void> {
    await this.connect();
    const value = JSON.stringify(entry);
    await this.client.rPush(this.key("logs"), value);
    await this.client.rPush(this.key(`logs:${entry.taskId}`), value);
  }

  async saveDependencyResult(taskId: string, dependencyId: string, value: unknown): Promise<void> {
    await this.connect();
    await this.client.hSet(this.key(`deps:${taskId}`), dependencyId, JSON.stringify(value));
  }

  async getDependencyResults(taskId: string): Promise<Map<string, unknown>> {
    await this.connect();
    const entries = await this.client.hGetAll(this.key(`deps:${taskId}`));
    return new Map(Object.entries(entries).map(([key, value]) => [key, JSON.parse(value)]));
  }

  private key(value: string): string {
    return `${this.namespace}:${value}`;
  }
}


path: src/storage/SQLiteStorage.ts
content: import Database from "better-sqlite3";
import type { QueueStorage, TaskLogEntry, TaskRecord } from "../types.js";

function parseTask(row: Record<string, unknown>): TaskRecord {
  return {
    ...JSON.parse(String(row.data)),
    id: String(row.id)
  } as TaskRecord;
}

export class SQLiteStorage implements QueueStorage {
  private readonly db: Database.Database;

  constructor(filename = "task-queue.sqlite") {
    this.db = new Database(filename);
    this.db.pragma("journal_mode = WAL");
    this.db.exec(`
      CREATE TABLE IF NOT EXISTS tasks (
        id TEXT PRIMARY KEY,
        data TEXT NOT NULL
      );
      CREATE TABLE IF NOT EXISTS logs (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        task_id TEXT NOT NULL,
        data TEXT NOT NULL
      );
      CREATE TABLE IF NOT EXISTS dependency_results (
        task_id TEXT NOT NULL,
        dependency_id TEXT NOT NULL,
        data TEXT NOT NULL,
        PRIMARY KEY (task_id, dependency_id)
      );
    `);
  }

  async saveTask(task: TaskRecord): Promise<void> {
    this.db.prepare("INSERT OR REPLACE INTO tasks (id, data) VALUES (?, ?)").run(task.id, JSON.stringify(task));
  }

  async getTask(taskId: string): Promise<TaskRecord | undefined> {
    const row = this.db.prepare("SELECT id, data FROM tasks WHERE id = ?").get(taskId) as Record<string, unknown> | undefined;
    return row ? parseTask(row) : undefined;
  }

  async updateTask(task: TaskRecord): Promise<void> {
    await this.saveTask(task);
  }

  async listTasks(): Promise<TaskRecord[]> {
    const rows = this.db.prepare("SELECT id, data FROM tasks").all() as Record<string, unknown>[];
    return rows.map(parseTask);
  }

  async listLogs(taskId?: string): Promise<TaskLogEntry[]> {
    const rows = taskId
      ? this.db.prepare("SELECT data FROM logs WHERE task_id = ? ORDER BY id ASC").all(taskId)
      : this.db.prepare("SELECT data FROM logs ORDER BY id ASC").all();
    return (rows as Array<{ data: string }>).map((row) => JSON.parse(row.data) as TaskLogEntry);
  }

  async appendLog(entry: TaskLogEntry): Promise<void> {
    this.db.prepare("INSERT INTO logs (task_id, data) VALUES (?, ?)").run(entry.taskId, JSON.stringify(entry));
  }

  async saveDependencyResult(taskId: string, dependencyId: string, value: unknown): Promise<void> {
    this.db
      .prepare("INSERT OR REPLACE INTO dependency_results (task_id, dependency_id, data) VALUES (?, ?, ?)")
      .run(taskId, dependencyId, JSON.stringify(value));
  }

  async getDependencyResults(taskId: string): Promise<Map<string, unknown>> {
    const rows = this.db
      .prepare("SELECT dependency_id, data FROM dependency_results WHERE task_id = ?")
      .all(taskId) as Array<{ dependency_id: string; data: string }>;
    return new Map(rows.map((row) => [row.dependency_id, JSON.parse(row.data)]));
  }
}


path: src/types.ts
content: export type TaskStatus =
  | "pending"
  | "queued"
  | "waiting_dependencies"
  | "running"
  | "retry_scheduled"
  | "completed"
  | "failed"
  | "dead_letter"
  | "cancelled";

export interface RetryPolicy {
  maxAttempts: number;
  backoffMs?: number;
  backoffMultiplier?: number;
  maxBackoffMs?: number;
}

export interface TaskOptions<TPayload = unknown> {
  id?: string;
  type: string;
  payload: TPayload;
  priority?: number;
  runAt?: Date;
  timeoutMs?: number;
  dependencies?: string[];
  retryPolicy?: Partial<RetryPolicy>;
  metadata?: Record<string, unknown>;
}

export interface TaskRecord<TPayload = unknown, TResult = unknown> {
  id: string;
  type: string;
  payload: TPayload;
  priority: number;
  runAt: string;
  timeoutMs?: number;
  dependencies: string[];
  retryPolicy: RetryPolicy;
  metadata: Record<string, unknown>;
  status: TaskStatus;
  attempts: number;
  createdAt: string;
  updatedAt: string;
  startedAt?: string;
  completedAt?: string;
  lastError?: string;
  result?: TResult;
}

export interface TaskLogEntry {
  taskId: string;
  level: "info" | "warn" | "error";
  message: string;
  timestamp: string;
  context?: Record<string, unknown>;
}

export interface TaskMetrics {
  queued: number;
  running: number;
  waitingDependencies: number;
  retryScheduled: number;
  completed: number;
  failed: number;
  deadLetter: number;
  avgDurationMs: number;
}

export interface TaskExecutionContext<TResult = unknown> {
  attempt: number;
  signal: AbortSignal;
  dependencies: Map<string, TResult>;
  log: (level: TaskLogEntry["level"], message: string, context?: Record<string, unknown>) => Promise<void>;
}

export type TaskHandler<TPayload = unknown, TResult = unknown> = (
  payload: TPayload,
  context: TaskExecutionContext<TResult>
) => Promise<TResult>;

export interface QueueStorage {
  saveTask(task: TaskRecord): Promise<void>;
  getTask(taskId: string): Promise<TaskRecord | undefined>;
  updateTask(task: TaskRecord): Promise<void>;
  listTasks(): Promise<TaskRecord[]>;
  listLogs(taskId?: string): Promise<TaskLogEntry[]>;
  appendLog(entry: TaskLogEntry): Promise<void>;
  saveDependencyResult(taskId: string, dependencyId: string, value: unknown): Promise<void>;
  getDependencyResults(taskId: string): Promise<Map<string, unknown>>;
}

export interface QueueSnapshot {
  ready: TaskRecord[];
  delayed: TaskRecord[];
  deadLetter: TaskRecord[];
}


path: src/utils.ts
content: import { randomUUID } from "node:crypto";
import type { RetryPolicy, TaskOptions, TaskRecord } from "./types.js";

export const DEFAULT_RETRY_POLICY: RetryPolicy = {
  maxAttempts: 3,
  backoffMs: 1_000,
  backoffMultiplier: 2,
  maxBackoffMs: 60_000
};

export function createTaskRecord<TPayload>(options: TaskOptions<TPayload>): TaskRecord<TPayload> {
  const now = new Date().toISOString();
  const retryPolicy = { ...DEFAULT_RETRY_POLICY, ...options.retryPolicy };
  const status = (options.dependencies?.length ?? 0) > 0 ? "waiting_dependencies" : "queued";
  return {
    id: options.id ?? randomUUID(),
    type: options.type,
    payload: options.payload,
    priority: options.priority ?? 0,
    runAt: (options.runAt ?? new Date()).toISOString(),
    timeoutMs: options.timeoutMs,
    dependencies: [...(options.dependencies ?? [])],
    retryPolicy,
    metadata: options.metadata ?? {},
    status,
    attempts: 0,
    createdAt: now,
    updatedAt: now
  };
}

export function computeRetryAt(task: TaskRecord): Date {
  const base = task.retryPolicy.backoffMs ?? DEFAULT_RETRY_POLICY.backoffMs!;
  const multiplier = task.retryPolicy.backoffMultiplier ?? DEFAULT_RETRY_POLICY.backoffMultiplier!;
  const max = task.retryPolicy.maxBackoffMs ?? DEFAULT_RETRY_POLICY.maxBackoffMs!;
  const exponent = Math.max(task.attempts - 1, 0);
  const delay = Math.min(base * multiplier ** exponent, max);
  return new Date(Date.now() + delay);
}

export function compareReadyTasks(left: TaskRecord, right: TaskRecord): number {
  if (left.priority !== right.priority) {
    return right.priority - left.priority;
  }
  const runAtDelta = new Date(left.runAt).getTime() - new Date(right.runAt).getTime();
  if (runAtDelta !== 0) {
    return runAtDelta;
  }
  return new Date(left.createdAt).getTime() - new Date(right.createdAt).getTime();
}


path: tests/task-queue.test.ts
content: import test from "node:test";
import assert from "node:assert/strict";
import { Scheduler } from "../src/Scheduler.js";
import { TaskQueue } from "../src/TaskQueue.js";

test("claims higher priority tasks first", async () => {
  const queue = new TaskQueue();
  const low = await queue.enqueue({ type: "job", payload: { name: "low" }, priority: 1 });
  const high = await queue.enqueue({ type: "job", payload: { name: "high" }, priority: 10 });

  const first = await queue.claimNextReady();
  const second = await queue.claimNextReady();

  assert.equal(first?.id, high.id);
  assert.equal(second?.id, low.id);
});

test("keeps delayed tasks out of the ready queue until due", async () => {
  const queue = new TaskQueue();
  await queue.enqueue({
    type: "job",
    payload: {},
    runAt: new Date(Date.now() + 60_000)
  });

  const snapshot = await queue.getSnapshot();
  assert.equal(snapshot.ready.length, 0);
  assert.equal(snapshot.delayed.length, 1);
});

test("unblocks dependent tasks and passes dependency results", async () => {
  const queue = new TaskQueue();
  const scheduler = new Scheduler(queue, { pollIntervalMs: 5, concurrency: 2 });
  const outputs: string[] = [];

  scheduler.register("producer", async () => "artifact-1");
  scheduler.register("consumer", async (_payload, context) => {
    outputs.push(String(context.dependencies.get("producer-task")));
    return "done";
  });

  await queue.enqueue({ id: "producer-task", type: "producer", payload: {} });
  await queue.enqueue({
    id: "consumer-task",
    type: "consumer",
    payload: {},
    dependencies: ["producer-task"]
  });

  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 20));
  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 20));

  const consumer = await queue.get("consumer-task");
  assert.equal(consumer?.status, "completed");
  assert.deepEqual(outputs, ["artifact-1"]);
});

test("moves exhausted retries to dead letter", async () => {
  const queue = new TaskQueue();
  const scheduler = new Scheduler(queue, { pollIntervalMs: 5, concurrency: 1 });
  let attempts = 0;

  scheduler.register("unstable", async () => {
    attempts += 1;
    throw new Error(`boom-${attempts}`);
  });

  await queue.enqueue({
    id: "unstable-task",
    type: "unstable",
    payload: {},
    retryPolicy: { maxAttempts: 2, backoffMs: 0, backoffMultiplier: 1 }
  });

  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 10));
  const firstFailure = await queue.get("unstable-task");
  assert.equal(firstFailure?.status, "retry_scheduled");

  if (firstFailure) {
    firstFailure.runAt = new Date(Date.now() - 1).toISOString();
    await queue.storage.updateTask(firstFailure);
  }

  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 10));

  const finalTask = await queue.get("unstable-task");
  assert.equal(finalTask?.status, "dead_letter");
  assert.equal(attempts, 2);
});

test("records timeout failures for long-running tasks", async () => {
  const queue = new TaskQueue();
  const scheduler = new Scheduler(queue, { pollIntervalMs: 5, concurrency: 1 });

  scheduler.register("slow", async (_payload, context) => {
    await new Promise((resolve, reject) => {
      const timer = setTimeout(resolve, 100);
      context.signal.addEventListener("abort", () => {
        clearTimeout(timer);
        reject(context.signal.reason);
      });
    });
    return "unreachable";
  });

  await queue.enqueue({
    id: "slow-task",
    type: "slow",
    payload: {},
    timeoutMs: 20,
    retryPolicy: { maxAttempts: 1 }
  });

  await scheduler.tick();
  await new Promise((resolve) => setTimeout(resolve, 40));

  const task = await queue.get("slow-task");
  assert.equal(task?.status, "dead_letter");
  assert.match(task?.lastError ?? "", /timed out/i);
});


path: tsconfig.json
content: {
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "outDir": "dist",
    "rootDir": ".",
    "strict": true,
    "declaration": true,
    "sourceMap": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "resolveJsonModule": true,
    "types": [
      "node"
    ]
  },
  "include": [
    "src/**/*.ts",
    "tests/**/*.ts",
    "examples/**/*.ts"
  ],
  "exclude": [
    "dist",
    "node_modules"
  ]
}


Security scan verdicts — ClawScan: clean | static analysis:  | VirusTotal: clean | SkillSpector: suspicious (MEDIUM)