Skill: noblepayne/tabstack-extractor (version 0.1.0)

--- name: tabstack-extractor description: Extract structured data from websites using Tabstack API. Use when you need to scrape job listings, news articles, product pages, or any structured web content. Provides JSON schema-based extraction and clean markdown conversion. Requires TABSTACK_API_KEY environment variable. --- # Tabstack Extractor ## Overview This skill enables structured data extraction from websites using the Tabstack API. It's ideal for web scraping tasks where you need consistent, schema-based data extraction from job boards, news sites, product pages, or any structured content. ## Quick Start ### 1. Install Babashka (if needed) ```bash # Option A: From GitHub (recommended for sharing) curl -s https://raw.githubusercontent.com/babashka/babashka/master/install | bash # Option B: From Nix nix-shell -p babashka # Option C: From Homebrew brew install borkdude/brew/babashka ``` ### 2. Set up API Key **Option A: Environment variable (recommended)** ```bash export TABSTACK_API_KEY="your_api_key_here" ``` **Option B: Configuration file** ```bash mkdir -p ~/.config/tabstack echo '{:api-key "your_api_key_here"}' > ~/.config/tabstack/config.edn ``` **Get an API key:** Sign up at [Tabstack Console](https://console.tabstack.ai/signup) ### 3. Test Connection ```bash bb scripts/tabstack.clj test ``` ### 4. Extract Markdown (Simple) ```bash bb scripts/tabstack.clj markdown "https://example.com" ``` ### 5. Extract JSON (Start Simple) ```bash # Start with simple schema (fast, reliable) bb scripts/tabstack.clj json "https://example.com" references/simple_article.json # Try more complex schemas (may be slower) bb scripts/tabstack.clj json "https://news.site" references/news_schema.json ``` ### 6. Advanced Features ```bash # Extract with retry logic (3 retries, 1s delay) bb scripts/tabstack.clj json-retry "https://example.com" references/simple_article.json # Extract with caching (24-hour cache) bb scripts/tabstack.clj json-cache "https://example.com" references/simple_article.json # Batch extract from URLs file echo "https://example.com" > urls.txt echo "https://example.org" >> urls.txt bb scripts/tabstack.clj batch urls.txt references/simple_article.json ``` ## Core Capabilities ### 1. Markdown Extraction Extract clean, readable markdown from any webpage. Useful for content analysis, summarization, or archiving. **When to use:** When you need the textual content of a page without the HTML clutter. **Example use cases:** - Extract article content for summarization - Archive webpage content - Analyze blog post content ### 2. JSON Schema Extraction Extract structured data using JSON schemas. Define exactly what data you want and get it in a consistent format. **When to use:** When scraping job listings, product pages, news articles, or any structured data. **Example use cases:** - Scrape job listings from BuiltIn/LinkedIn - Extract product details from e-commerce sites - Gather news articles with consistent metadata ### 3. Schema Templates Pre-built schemas for common scraping tasks. See `references/` directory for templates. **Available schemas:** - Job listing schema (see `references/job_schema.json`) - News article schema - Product page schema - Contact information schema ## Workflow: Job Scraping Example Follow this workflow to scrape job listings: 1. **Identify target sites** - BuiltIn, LinkedIn, company career pages 2. **Choose or create schema** - Use `references/job_schema.json` or customize 3. **Test extraction** - Run a single page to verify schema works 4. **Scale up** - Process multiple URLs 5. **Store results** - Save to database or file **Example job schema:** ```json { "type": "object", "properties": { "title": {"type": "string"}, "company": {"type": "string"}, "location": {"type": "string"}, "description": {"type": "string"}, "salary": {"type": "string"}, "apply_url": {"type": "string"}, "posted_date": {"type": "string"}, "requirements": {"type": "array", "items": {"type": "string"}} } } ``` ## Integration with Other Skills ### Combine with Web Search 1. Use `web_search` to find relevant URLs 2. Use Tabstack to extract structured data from those URLs 3. Store results in Datalevin (future skill) ### Combine with Browser Automation 1. Use `browser` tool to navigate complex sites 2. Extract page URLs 3. Use Tabstack for structured extraction ## Error Handling Common issues and solutions: 1. **Authentication failed** - Check `TABSTACK_API_KEY` environment variable 2. **Invalid URL** - Ensure URL is accessible and correct 3. **Schema mismatch** - Adjust schema to match page structure 4. **Rate limiting** - Add delays between requests ## Resources ### scripts/ - `tabstack.clj` - **Main API wrapper in Babashka** (recommended, has retry logic, caching, batch processing) - `tabstack_curl.sh` - Bash/curl fallback (simple, no dependencies) - `tabstack_api.py` - Python API wrapper (requires requests module) ### references/ - `job_schema.json` - Template schema for job listings - `api_reference.md` - Tabstack API documentation ## Best Practices 1. **Start small** - Test with single pages before scaling 2. **Respect robots.txt** - Check site scraping policies 3. **Add delays** - Avoid overwhelming target sites 4. **Validate schemas** - Test schemas on sample pages 5. **Handle errors gracefully** - Implement retry logic for failed requests ## Teaching Focus: How to Create Schemas This skill is designed to teach agents how to use Tabstack API effectively. The key is learning to create appropriate JSON schemas for different websites. ### Learning Path 1. **Start Simple** - Use `references/simple_article.json` (4 basic fields) 2. **Test Extensively** - Try schemas on multiple page types 3. **Iterate** - Add fields based on what the page actually contains 4. **Optimize** - Remove unnecessary fields for speed See [Schema Creation Guide](references/schema_guide.md) for detailed instructions and examples. ### Common Mistakes to Avoid - **Over-complex schemas** - Start with 2-3 fields, not 20 - **Missing fields** - Don't require fields that don't exist on the page - **No testing** - Always test with example.com first, then target sites - **Ignoring timeouts** - Complex schemas take longer (45s timeout) ## Babashka Advantages Using Babashka for this skill provides: 1. **Single binary** - Easy to share/install (GitHub releases, brew, nix) 2. **Fast startup** - No JVM warmup, ~50ms startup time 3. **Built-in HTTP client** - No external dependencies 4. **Clojure syntax** - Familiar to you (Wes), expressive 5. **Retry logic & caching** - Built into the skill 6. **Batch processing** - Parallel extraction for multiple URLs ## Example User Requests **For this skill to trigger:** - "Scrape job listings from Docker careers page" - "Extract the main content from this article" - "Get structured product data from this e-commerce page" - "Pull all the news articles from this site" - "Extract contact information from this company page" - "Batch extract job listings from these 20 URLs" - "Get cached results for this page (avoid API calls)"

path: references/api_reference.md
content: # Tabstack API Reference

## Base URL
```
https://api.tabstack.ai/v1
```

## Authentication
Bearer token authentication required:
```
Authorization: Bearer <your_api_key>
```

## Endpoints

### 1. Extract JSON
```
POST /extract/json
```
Extract structured data according to a provided JSON schema.

**Request body:**
```json
{
  "url": "https://example.com",
  "schema": {
    "type": "object",
    "properties": {
      "title": {"type": "string"},
      "content": {"type": "string"}
    }
  }
}
```

**Response:**
```json
{
  "title": "Example Page",
  "content": "Page content here..."
}
```

### 2. Extract Markdown
```
POST /extract/markdown
```
Fetches a URL and converts its HTML content to clean Markdown format.

**Request body:**
```json
{
  "url": "https://example.com"
}
```

**Response:**
Markdown text of the page content.

### 3. Generate JSON
```
POST /generate/json
```
Generate JSON from queries (research capabilities).

### 4. Automate
```
POST /automate
```
Browser automation capabilities.

### 5. Research
```
POST /research
```
Research capabilities.

## Response Codes

- **200** - OK
- **400** - Bad request - invalid input
- **401** - Unauthorized - Invalid or missing Bearer token
- **422** - Invalid URL or inaccessible resource
- **500** - Server error

## Rate Limits
Check Tabstack documentation for current rate limits. Default is typically 100 requests per minute.

## Best Practices

1. **Schema Design** - Create specific schemas for each site type
2. **Error Handling** - Implement retry logic for 422/500 errors
3. **Caching** - Cache results to avoid repeated requests
4. **Validation** - Validate extracted data matches schema expectations

Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: suspicious | SkillSpector: suspicious (HIGH)