Skill: xueyetianya/agent-toolkit (version 2.0.2)

--- version: "2.0.1" name: agent-toolkit description: "Configure and benchmark agent tools and integration patterns. Use when setting up agent workflows, comparing tools, or evaluating agents." author: BytesAgain homepage: https://bytesagain.com source: https://github.com/bytesagain/ai-skills --- # Agent Toolkit A comprehensive AI toolkit for configuring, benchmarking, comparing, and optimizing agent tools and integration patterns. Agent Toolkit provides persistent, file-based logging for each command category with timestamped entries, summary statistics, multi-format export, and full-text search across all records. ## Commands | Command | Description | |---------|-------------| | `configure` | Configure agent tools — log configuration entries or view recent ones | | `benchmark` | Benchmark tool performance — log benchmark results or view history | | `compare` | Compare tool outputs — log comparison data or view recent comparisons | | `prompt` | Prompt management — log prompt variations or view recent prompts | | `evaluate` | Evaluate tool results — log evaluation data or view history | | `fine-tune` | Fine-tune parameters — log fine-tuning sessions or view recent ones | | `analyze` | Analyze tool behavior — log analysis entries or view recent analyses | | `cost` | Cost tracking — log cost data or view recent cost entries | | `usage` | Usage monitoring — log usage metrics or view recent usage data | | `optimize` | Optimize configurations — log optimization runs or view history | | `test` | Test tool behavior — log test results or view recent tests | | `report` | Report generation — log report entries or view recent reports | | `stats` | Show summary statistics across all log categories (entry counts, data size, first entry date) | | `export <fmt>` | Export all data in json, csv, or txt format to the data directory | | `search <term>` | Full-text search across all log files (case-insensitive) | | `recent` | Show the 20 most recent entries from the activity history log | | `status` | Health check — show version, data directory, total entries, disk usage, and last activity | | `help` | Show the full help message with all available commands | | `version` | Print the current version string | Each data command (configure, benchmark, compare, etc.) works in two modes: - **Without arguments**: displays the 20 most recent entries from that category - **With arguments**: saves the input as a new timestamped entry and reports the total count ## Data Storage All data is stored in plain text files under the data directory: - **Category logs**: `$DATA_DIR/<command>.log` — one file per command (e.g., `configure.log`, `benchmark.log`, `prompt.log`), each entry is `timestamp|value` - **History log**: `$DATA_DIR/history.log` — audit trail of every command executed with timestamps - **Export files**: `$DATA_DIR/export.<fmt>` — generated by the `export` command in json, csv, or txt format Default data directory: `~/.local/share/agent-toolkit/` ## Requirements - Bash (with `set -euo pipefail` support) - Standard Unix utilities: `grep`, `cat`, `date`, `echo`, `wc`, `du`, `head`, `tail`, `basename` - No external dependencies or API keys required ## When to Use 1. **Setting up agent workflows** — When you need to configure and log settings for agent tool integrations, API connections, or pipeline configurations 2. **Benchmarking and comparing tools** — When you're evaluating different AI tools or agent frameworks and want to log performance metrics for comparison 3. **Cost and usage optimization** — When you need to track API costs, token usage, and resource consumption across different tools to optimize spending 4. **Fine-tuning and testing** — When running fine-tuning experiments or test suites and you want to log parameters, results, and observations 5. **Cross-tool analysis and reporting** — When you need to search across all logged data, generate reports, or export results for stakeholder review ## Examples ```bash # Check toolkit status agent-toolkit status # Configure a new tool integration agent-toolkit configure "OpenAI API key rotated, new model endpoint: gpt-4o-2024-08" # Benchmark a tool agent-toolkit benchmark "LangChain ReAct agent: 94% task completion, 3.4s avg response time" # Compare two tools agent-toolkit compare "LangChain vs CrewAI: LangChain 20% faster setup, CrewAI better multi-agent coordination" # Log a prompt template agent-toolkit prompt "Tool-use system prompt v3: Added structured output format and error handling instructions" # Track costs agent-toolkit cost "Weekly API spend: OpenAI $12.30, Anthropic $8.50, total $20.80" # View recent benchmarks agent-toolkit benchmark # Search across all logs agent-toolkit search "LangChain" # Export all data as CSV agent-toolkit export csv # View summary statistics agent-toolkit stats # Show recent activity agent-toolkit recent ``` ## Output All commands return output to stdout. Export files are written to the data directory: ```bash agent-toolkit export json # → ~/.local/share/agent-toolkit/export.json agent-toolkit export csv # → ~/.local/share/agent-toolkit/export.csv agent-toolkit export txt # → ~/.local/share/agent-toolkit/export.txt ``` Every command execution is logged to `$DATA_DIR/history.log` for auditing purposes. --- Powered by BytesAgain | bytesagain.com | [REDACTED_SECRET]


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)