Skill: zqh2333/model-registry-manager (version 1.3.0)

--- name: model-registry-manager description: "Detect provider models, deduplicate them, remove unusable ones, register missing models into OpenClaw, and safely keep provider-native model ids/names during model catalog sync." version: 1.3.0 --- # Model Registry Manager Use this skill when managing OpenClaw model catalogs and provider model sync for any provider. ## What this skill covers - Fetch provider model lists from the upstream `/models` endpoint - Deduplicate returned models by provider-native model id - Probe models and remove unusable ones from the registered catalog - Register newly discovered usable models into OpenClaw config - Keep provider-native ids and names instead of inventing renamed keys - Validate the registry before enabling scheduled sync jobs ## Required workflow ### 1. Inspect config/schema first Before changing config, inspect relevant config paths and read current config. Usually inspect: - `agents.defaults.model` - `agents.defaults.models` - `models.providers` ### 2. Detect before writing Always fetch remote models first and compare with current config. Do not write first when the state is unknown. ### 3. Keep provider-native ids/names - Use the provider-returned model `id` as the canonical registry id suffix - Use the provider-returned model `name` when available - Do not normalize ids into a different slug format unless OpenClaw requires it ### 4. Validate before keeping A model is only eligible for registration if it passes a lightweight probe. If probing fails, treat it as unusable and exclude it from the registered catalog. ### 5. Do not auto-select primary/fallbacks This skill should not automatically choose or rebuild primary/fallback chains. During sync: - keep provider model discovery, deduplication, probing, and registration focused on the catalog itself - do not introduce heuristic primary selection here - do not introduce heuristic fallback selection here - if model routing policy is needed, manage it outside this skill ### 6. Do not bundle generic failover policy here This skill should not define generic failover classes, retry ladders, downgrade policy, or circuit-breaker defaults. Keep this skill focused on model registry sync and validation. ### 7. Schedule only after one clean run First complete: - one successful sync run - one successful validation run - one confirmation that primary/fallback settings are correct Then you may add scheduled refresh jobs. ### 8. Default closing step: maintain the learnings archive After finishing real work, check whether the run produced reusable operational knowledge. Default behavior: - if the user corrected you, add the correction to `.learnings/` - if you debugged a failure, add the cause/fix to `.learnings/` - if you found a better generic strategy, add it to `.learnings/` - if you needed multiple attempts and eventually found a reliable path, add that successful path to `.learnings/` - do not wait for the user to remind you This is part of the skill’s normal operating procedure, not an optional extra. ## Validation checklist - fetched remote model list successfully - deduplicated duplicate remote ids - excluded unusable models - registered usable models - gateway/service still healthy after config update ## How to make this skill truly general-purpose To keep this skill generic instead of overfitting to one provider/model family: ### Keep generic - provider-native ids and names - provider model discovery and probe validation - per-provider sync + global catalog merge ### Avoid hardcoding - exact vendor model ids as permanent rules - one provider’s naming quirks as universal truth - one-off emergency fixes in the main sync algorithm - model routing policy into registry sync ### Prefer this structure - `SKILL.md`: only workflow + decision rules - `scripts/`: deterministic sync/validate logic ## Scheduling policy for this workspace After installation is validated, schedule these triggers: - startup/heartbeat safety net - recurring sync every 6 hours Keep scheduled runs idempotent and safe to repeat. ## Setup In an OpenClaw agent session, prefer the built-in `cron` tool instead of shelling out to `openclaw cron ...`. After one clean validation run, ensure these recurring jobs exist (create missing ones, update same-name jobs if they already exist): - `model-registry-manager validation safety-net` - `model-registry-manager recurring sync` Recommended default schedules (Asia/Shanghai): - validation safety-net: `10 4 * * *` - recurring sync: `40 */6 * * *` Recommended job shape: - `sessionTarget: "isolated"` - `payload.kind: "agentTurn"` - `payload.toolsAllow: ["exec", "read"]` - `delivery.mode: "none"` - failure alert enabled after 1 failure Recommended task behavior: - validation job runs `node skills/model-registry-manager/scripts/validate-model-sync.js` - recurring sync runs `node skills/model-registry-manager/scripts/sync-models.js all 0 --restart` - each job should read the latest report under `reports/model-registry/` and produce a concise summary Do not enable recurring sync before a clean validation run.

path: _meta.json
content: ownerId: kn76kp5gkqx0r5mnhk183yy8v983y9c6
slug: model-registry-manager
version: 1.2.0
publishedAt: 1775728933555

path: references/api-failover-upgrade.md
content: # API Failover Upgrade Notes

This reference is retained only as a historical note.

Current policy:
- `model-registry-manager` no longer carries generic failover policy guidance.
- It no longer defines retry ladders, fallback-routing semantics, or circuit-breaker defaults.
- The skill is now focused on model discovery, deduplication, probe validation, unusable-model cleanup, and registry sync only.


path: references/model-sync-policy.json
content: {
  "allowlist": [],
  "denylist": [],
  "notes": "allowlist non-empty means only listed model ids are eligible; denylist always wins"
}


path: references/model-sync-policy.md
content: # Model Sync Policy

Use `references/model-sync-policy.json` for sync-time filtering policy.

## Fields
- `allowlist`: when non-empty, only these remote model ids are eligible
- `denylist`: these remote model ids are always ignored

## Rules
- keep provider-native ids
- duplicate remote ids are deduplicated by id
- unusable models are excluded after probe
- allowlist/denylist affects eligibility before registration
- this skill does not choose primary/fallbacks


path: scripts/sync-models.js
content: #!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const { execSync } = require('child_process');

const CONFIG = path.resolve(process.env.HOME || '/home/zqh2333', '.openclaw/openclaw.json');
const POLICY_PATH = path.join(__dirname, '..', 'references', 'model-sync-policy.json');
const REPORT_DIR = path.resolve(__dirname, '..', '..', '..', 'reports', 'model-registry');
const PROVIDER_ARG = process.argv[2] || 'all';
const VERIFY_LIMIT = Number(process.argv[3] || 0); // 0 => all
const RESTART = process.argv.includes('--restart');
const DRY_RUN = process.argv.includes('--dry-run');

function catalogKey(providerKey, modelId) {
  return `${providerKey}/${modelId}`;
}

function loadPolicy() {
  try {
    const raw = JSON.parse(fs.readFileSync(POLICY_PATH, 'utf8'));
    return {
      allowlist: Array.isArray(raw.allowlist) ? new Set(raw.allowlist.map(String)) : null,
      denylist: new Set(Array.isArray(raw.denylist) ? raw.denylist.map(String) : []),
      notes: raw.notes || '',
    };
  } catch {
    return { allowlist: null, denylist: new Set(), notes: '' };
  }
}

function isAllowed(policy, modelId) {
  if (policy.denylist.has(modelId)) return false;
  if (policy.allowlist && policy.allowlist.size > 0) return policy.allowlist.has(modelId);
  return true;
}

async function fetchModels(provider) {
  const url = provider.baseUrl.replace(/\/$/, '') + '/models';
  const res = await fetch(url, { headers: { Authorization: `Bearer ${provider.apiKey}` } });
  if (!res.ok) throw new Error(`GET /models failed: ${res.status}`);
  const json = await res.json();
  return Array.isArray(json.data) ? json.data : [];
}

async function probeModel(provider, modelId) {
  const url = provider.baseUrl.replace(/\/$/, '') + '/chat/completions';
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'content-type': 'application/json', Authorization: `Bearer ${provider.apiKey}` },
    body: JSON.stringify({ model: modelId, messages: [{ role: 'user', content: 'ping' }], max_tokens: 1, temperature: 0 }),
  });
  return res.ok;
}

function normalizeModel(remote, previous = {}) {
  return {
    id: remote.id,
    name: remote.name || previous.name || remote.id,
    reasoning: typeof remote.reasoning === 'boolean' ? remote.reasoning : (typeof previous.reasoning === 'boolean' ? previous.reasoning : false),
    input: Array.isArray(remote.input) ? remote.input : (Array.isArray(previous.input) ? previous.input : ['text']),
    cost: remote.cost || previous.cost || { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
    contextWindow: remote.contextWindow || previous.contextWindow || 200000,
    maxTokens: remote.maxTokens || previous.maxTokens || 8192,
  };
}

function diffKeys(beforeKeys, afterKeys) {
  const before = new Set(beforeKeys);
  const after = new Set(afterKeys);
  return {
    added: afterKeys.filter(k => !before.has(k)),
    removed: beforeKeys.filter(k => !after.has(k)),
    kept: afterKeys.filter(k => before.has(k)),
  };
}

function writeReportFiles(report) {
  fs.mkdirSync(REPORT_DIR, { recursive: true });
  fs.writeFileSync(path.join(REPORT_DIR, 'sync-latest.json'), JSON.stringify(report, null, 2) + '\n', 'utf8');

  const lines = [];
  lines.push('# Model Registry Sync Report');
  lines.push('');
  lines.push(`- dryRun: ${report.dryRun}`);
  lines.push(`- changed: ${report.changed}`);
  lines.push(`- registeredModels: ${report.registeredModels}`);
  lines.push(`- providers: ${report.providers.length}`);
  lines.push(`- policy.allowlist: ${report.policy.allowlistSize}`);
  lines.push(`- policy.denylist: ${report.policy.denylistSize}`);
  if (report.policy.notes) lines.push(`- policy.notes: ${report.policy.notes}`);
  lines.push('');
  lines.push('## Diff');
  lines.push(`- added: ${report.diff.added.length}`);
  lines.push(`- removed: ${report.diff.removed.length}`);
  lines.push(`- kept: ${report.diff.kept.length}`);
  lines.push('');
  lines.push('## Provider stats');
  lines.push('| provider | fetched | unique | verified | denied | duplicates | unusable |');
  lines.push('| --- | ---: | ---: | ---: | ---: | ---: | ---: |');
  for (const p of report.providers) {
    lines.push(`| ${p.provider} | ${p.fetched} | ${p.unique} | ${p.verified} | ${p.denied} | ${p.duplicateRemoteIds.length} | ${p.unusable.length} |`);
  }
  lines.push('');
  lines.push('## Denied / unusable');
  for (const p of report.providers) {
    if (p.denied.length) lines.push(`- ${p.provider} denied: ${p.denied.join(', ')}`);
    if (p.unusable.length) lines.push(`- ${p.provider} unusable: ${p.unusable.join(', ')}`);
  }
  fs.writeFileSync(path.join(REPORT_DIR, 'sync-latest.md'), lines.join('\n') + '\n', 'utf8');
}

(async () => {
  const before = fs.readFileSync(CONFIG, 'utf8');
  const cfg = JSON.parse(before);
  const policy = loadPolicy();
  const providers = cfg.models?.providers || {};
  const providerKeys = PROVIDER_ARG === 'all' ? Object.keys(providers) : [PROVIDER_ARG];
  if (providerKeys.length === 0) throw new Error('No providers found');

  const previousCatalogKeys = Object.keys(cfg.agents?.defaults?.models || {});
  const allVerified = [];
  const providerReports = [];

  for (const providerKey of providerKeys) {
    const provider = providers[providerKey];
    if (!provider) throw new Error(`Provider not found: ${providerKey}`);
    const previousModels = new Map((provider.models || []).map(m => [m.id, m]));
    const remote = await fetchModels(provider);

    const dedup = new Map();
    const duplicateRemoteIds = [];
    const denied = [];
    for (const m of remote) {
      if (!m?.id) continue;
      if (!isAllowed(policy, m.id)) {
        denied.push(m.id);
        continue;
      }
      if (dedup.has(m.id)) {
        duplicateRemoteIds.push(m.id);
        continue;
      }
      dedup.set(m.id, m);
    }

    const verified = [];
    const unusable = [];
    const ordered = [...dedup.values()];
    const limit = VERIFY_LIMIT > 0 ? Math.min(VERIFY_LIMIT, ordered.length) : ordered.length;

    for (const model of ordered.slice(0, limit)) {
      const ok = await probeModel(provider, model.id).catch(() => false);
      if (!ok) {
        unusable.push(model.id);
        continue;
      }
      const normalized = normalizeModel(model, previousModels.get(model.id));
      verified.push(normalized);
      allVerified.push({ providerKey, ...normalized });
    }

    provider.models = verified;
    providerReports.push({
      provider: providerKey,
      fetched: remote.length,
      unique: dedup.size,
      verified: verified.length,
      denied,
      duplicateRemoteIds,
      unusable,
    });
  }

  cfg.agents = cfg.agents || {};
  cfg.agents.defaults = cfg.agents.defaults || {};
  cfg.agents.defaults.models = {};
  for (const m of allVerified) cfg.agents.defaults.models[catalogKey(m.providerKey, m.id)] = {};

  const afterCatalogKeys = Object.keys(cfg.agents.defaults.models);
  const diff = diffKeys(previousCatalogKeys, afterCatalogKeys);
  const next = JSON.stringify(cfg, null, 2) + '\n';
  const changed = next !== before;

  if (!DRY_RUN) {
    fs.writeFileSync(CONFIG, next, 'utf8');
    if (changed && RESTART) execSync('openclaw gateway restart', { stdio: 'inherit' });
  }

  const report = {
    dryRun: DRY_RUN,
    changed,
    configWritten: !DRY_RUN,
    providers: providerReports,
    policy: {
      allowlistSize: policy.allowlist ? policy.allowlist.size : 0,
      denylistSize: policy.denylist.size,
      notes: policy.notes,
    },
    diff,
    registeredModels: allVerified.length,
  };

  writeReportFiles(report);
  console.log(JSON.stringify(report, null, 2));
})().catch(err => {
  console.error(err.stack || err.message || String(err));
  process.exit(1);
});


path: scripts/validate-model-sync.js
content: #!/usr/bin/env node
const fs = require('fs');
const path = require('path');

const CONFIG = path.resolve(process.env.HOME || '/home/zqh2333', '.openclaw/openclaw.json');
const POLICY_PATH = path.join(__dirname, '..', 'references', 'model-sync-policy.json');
const REPORT_DIR = path.resolve(__dirname, '..', '..', '..', 'reports', 'model-registry');
const PROVIDER_ARG = process.argv[2] || 'all';

function loadPolicy() {
  try {
    const raw = JSON.parse(fs.readFileSync(POLICY_PATH, 'utf8'));
    return {
      allowlist: Array.isArray(raw.allowlist) ? new Set(raw.allowlist.map(String)) : null,
      denylist: new Set(Array.isArray(raw.denylist) ? raw.denylist.map(String) : []),
      notes: raw.notes || '',
    };
  } catch {
    return { allowlist: null, denylist: new Set(), notes: '' };
  }
}

function isAllowed(policy, modelId) {
  if (policy.denylist.has(modelId)) return false;
  if (policy.allowlist && policy.allowlist.size > 0) return policy.allowlist.has(modelId);
  return true;
}

async function fetchModels(provider) {
  const url = provider.baseUrl.replace(/\/$/, '') + '/models';
  const res = await fetch(url, { headers: { Authorization: `Bearer ${provider.apiKey}` } });
  if (!res.ok) throw new Error(`GET /models failed: ${res.status}`);
  const json = await res.json();
  return Array.isArray(json.data) ? json.data : [];
}

async function probeModel(provider, modelId) {
  const url = provider.baseUrl.replace(/\/$/, '') + '/chat/completions';
  const res = await fetch(url, {
    method: 'POST',
    headers: { 'content-type': 'application/json', Authorization: `Bearer ${provider.apiKey}` },
    body: JSON.stringify({ model: modelId, messages: [{ role: 'user', content: 'ping' }], max_tokens: 1, temperature: 0 }),
  });
  return res.ok;
}

function writeReportFiles(report) {
  fs.mkdirSync(REPORT_DIR, { recursive: true });
  fs.writeFileSync(path.join(REPORT_DIR, 'validate-latest.json'), JSON.stringify(report, null, 2) + '\n', 'utf8');

  const lines = [];
  lines.push('# Model Registry Validation Report');
  lines.push('');
  lines.push(`- fail: ${report.fail}`);
  lines.push(`- providers: ${report.providers.length}`);
  lines.push(`- registeredModels: ${report.registeredModels}`);
  lines.push(`- policy.allowlist: ${report.policy.allowlistSize}`);
  lines.push(`- policy.denylist: ${report.policy.denylistSize}`);
  if (report.policy.notes) lines.push(`- policy.notes: ${report.policy.notes}`);
  lines.push('');
  lines.push('## Provider stats');
  lines.push('| provider | fetched | unique | registered | usable | denied | duplicates |');
  lines.push('| --- | ---: | ---: | ---: | ---: | ---: | ---: |');
  for (const p of report.providers) {
    lines.push(`| ${p.provider} | ${p.fetched} | ${p.unique} | ${p.registered} | ${p.usable} | ${p.denied} | ${p.duplicates.length} |`);
  }
  lines.push('');
  lines.push('## Checks');
  for (const item of report.checks) lines.push(`- ${item.ok ? 'PASS' : 'FAIL'} ${item.message}`);
  fs.writeFileSync(path.join(REPORT_DIR, 'validate-latest.md'), lines.join('\n') + '\n', 'utf8');
}

(async () => {
  const cfg = JSON.parse(fs.readFileSync(CONFIG, 'utf8'));
  const policy = loadPolicy();
  const providers = cfg.models?.providers || {};
  const providerKeys = PROVIDER_ARG === 'all' ? Object.keys(providers) : [PROVIDER_ARG];
  if (providerKeys.length === 0) throw new Error('No providers found');

  const checks = [];
  const providerReports = [];
  const registered = [];
  let fail = 0;

  function check(ok, message) {
    checks.push({ ok, message });
    console.log(`${ok ? 'PASS' : 'FAIL'}\t${message}`);
    if (!ok) fail++;
  }

  for (const providerKey of providerKeys) {
    const provider = providers[providerKey];
    if (!provider) throw new Error(`Provider not found: ${providerKey}`);

    const remote = await fetchModels(provider);
    const remoteById = new Map();
    const duplicateRemoteIds = [];
    let allowedRemote = 0;
    for (const m of remote) {
      if (!m?.id) continue;
      if (!isAllowed(policy, m.id)) continue;
      allowedRemote++;
      if (remoteById.has(m.id)) {
        duplicateRemoteIds.push(m.id);
        continue;
      }
      remoteById.set(m.id, m);
    }

    const models = provider.models || [];
    const seen = new Set();
    const duplicateIds = [];
    let usable = 0;
    let denied = 0;

    for (const m of models) {
      registered.push({ providerKey, ...m });
      if (seen.has(m.id)) duplicateIds.push(m.id);
      seen.add(m.id);
      if (policy.denylist.has(m.id)) denied++;
      const remoteName = remoteById.get(m.id)?.name;
      const ok = await probeModel(provider, m.id).catch(() => false);
      if (ok) usable++;
      check(remoteById.has(m.id), `registered model exists remotely: ${providerKey}/${m.id}`);
      check(ok, `registered model is probe-usable: ${providerKey}/${m.id}`);
      check(Object.keys(cfg.agents?.defaults?.models || {}).includes(`${providerKey}/${m.id}`), `catalog key exists: ${providerKey}/${m.id}`);
      check(!remoteName || remoteName === m.name, `registered model name matches remote: ${providerKey}/${m.id}`);
    }

    check(duplicateIds.length === 0, `provider.models has no duplicate ids: ${providerKey}`);
    providerReports.push({
      provider: providerKey,
      fetched: remote.length,
      unique: remoteById.size,
      registered: models.length,
      usable,
      denied,
      duplicates: duplicateIds,
      allowedRemote,
      duplicateRemoteIds,
    });
  }

  check(Object.keys(cfg.agents?.defaults?.models || {}).length === registered.length, 'agents.defaults.models count matches registered models count across providers');

  const report = {
    fail,
    registeredModels: registered.length,
    providers: providerReports,
    policy: {
      allowlistSize: policy.allowlist ? policy.allowlist.size : 0,
      denylistSize: policy.denylist.size,
      notes: policy.notes,
    },
    checks,
  };

  writeReportFiles(report);
  console.log(`\nSUMMARY\tfail=${fail}\tregistered=${registered.length}\tproviders=${providerKeys.join(',')}`);
  process.exitCode = fail ? 1 : 0;
})().catch(err => {
  console.error(err.stack || err.message || String(err));
  process.exit(1);
});


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal: clean | SkillSpector: suspicious (HIGH)