Skill: liuyu020923/pangolin-amazon-scraper (version 1.0.2)

--- name: pangolinfo-amazon-scraper description: > Scrape Amazon product data using Pangolin APIs. Use this skill when the user wants to: look up Amazon products by ASIN, search Amazon by keyword, check bestsellers/new releases, get product reviews or pricing, compare products across Amazon regions, browse categories, or research sellers. Requires PANGOLIN_EMAIL + PANGOLIN_PASSWORD env vars (or PANGOLIN_API_KEY). --- # Pangolinfo Amazon Scraper Scrape Amazon product data via Pangolin's structured APIs. Returns parsed JSON with product details, search results, rankings, reviews, and more across 13 Amazon regions. ## When to Use This Skill Activate this skill when the user's intent matches any of these patterns: | Intent (English) | Intent (Chinese) | Action | |---|---|---| | Look up an Amazon product / ASIN | 查一下亚马逊上的这个商品 / ASIN | Product detail lookup | | Search Amazon for a keyword | 在亚马逊搜索关键词 | Keyword search | | Show bestsellers in a category | 看看某个类目的畅销榜 | Bestsellers | | Show new releases | 看看最新上架的商品 | New releases | | Browse a category on Amazon | 浏览亚马逊某个分类 | Category products | | Find products from a seller | 查看某个卖家的商品 | Seller products | | Get reviews for a product | 看看这个产品的评论 | Product reviews | | Compare prices across regions | 对比不同国家亚马逊的价格 | Multi-region lookup | | Check seller/variant options | 查看其他卖家选项 | Follow seller | **Keywords that trigger this skill:** Amazon, ASIN, B0-prefixed codes, product reviews, bestseller, new release, Amazon category, Amazon seller, amz_us/amz_uk/etc., product detail, Amazon search, price comparison across Amazon sites. ## Prerequisites ### Required - **Python 3.8+** (uses only the standard library -- zero external dependencies) - **Pangolin account** at [pangolinfo.com](https://www.pangolinfo.com) - **Environment variables** (one of the following): - `PANGOLIN_API_KEY` -- API key (skips login), OR - `PANGOLIN_EMAIL` + `PANGOLIN_PASSWORD` -- auto-login with caching ### macOS SSL Certificate Fix On macOS, Python may fail with `CERTIFICATE_VERIFY_FAILED` because it ships without root certificates by default. **Symptoms:** The script exits with error code `SSL_CERT`. **Solutions (pick one):** 1. Run the certificate installer bundled with Python: ```bash /Applications/Python\ 3.x/Install\ Certificates.command ``` (Replace `3.x` with your Python version, e.g., `3.11`) 2. Set `SSL_CERT_FILE` from the `certifi` package: ```bash pip3 install certifi export SSL_CERT_FILE=$(python3 -c "import certifi; print(certifi.where())") ``` 3. Add the export to your shell profile (`~/.zshrc` or `~/.bashrc`) so it persists. ## First-Time Setup Guide When a user tries to use this skill and authentication fails (error code `MISSING_ENV`), **do not just repeat the error hint**. Instead, walk the user through the full setup process interactively: ### Step 1: Explain what's needed Tell the user (in their language): > To use this skill, you need a Pangolin API account. Pangolin provides Amazon product data scraping through its APIs. > > 使用本技能需要 Pangolin API 账号。Pangolin 提供亚马逊商品数据抓取的 API 服务。 ### Step 2: Guide registration > 1. Go to [pangolinfo.com](https://www.pangolinfo.com) and create an account > 2. After login, find your API Key in the dashboard > > 1. 访问 [pangolinfo.com](https://www.pangolinfo.com) 注册账号 > 2. 登录后在控制台找到你的 API Key ### Step 3: Collect credentials and authenticate automatically When the user provides their credentials, **you (the AI agent) should configure them securely**. The script will automatically cache the API key at `~/.pangolin_api_key` for all future calls. **If user provides an API key (recommended):** Write it directly to the cache file — avoids shell history entirely: ```bash echo "<api_key>" > ~/.pangolin_api_key chmod 600 ~/.pangolin_api_key 2>/dev/null python3 scripts/pangolin.py --auth-only ``` **If user provides email + password:** Set env vars in the session and clean up after auth: ```bash export PANGOLIN_EMAIL="[REDACTED_SECRET]" export PANGOLIN_PASSWORD="their-password" python3 scripts/pangolin.py --auth-only unset PANGOLIN_EMAIL PANGOLIN_PASSWORD ``` This avoids passwords appearing in shell history (unlike inline `VAR=x command` syntax) and cleans up credentials after the API key is cached. Both methods cache the API key automatically. After this one-time setup, **no environment variables are needed** — all future calls will use the cached API key at `~/.pangolin_api_key`. ### Step 4: Confirm and proceed After auth returns `"success": true`: 1. Tell the user: "认证成功！API Key 已自动缓存，后续使用无需再次输入。" / "Authentication successful! API Key cached — no need to enter credentials again." 2. Immediately retry their original request. ### Important - **The user only needs to provide credentials ONCE** — the script caches the API key permanently at `~/.pangolin_api_key` - Do not ask the user to manually edit `.bashrc` or `.zshrc` — the script handles persistence automatically - If the user doesn't have an account yet, explain Pangolin's credit system (1 credit per Amazon json request, 5 credits per review page) and direct them to [pangolinfo.com](https://www.pangolinfo.com) - If auth succeeds but credits are exhausted (error code `2001`), tell the user to top up at pangolinfo.com - The API key is permanent and does not expire unless the account is deactivated ## Script Execution The script is located at `scripts/pangolin.py` relative to this skill's root directory. **Path resolution:** When invoking the script, resolve the absolute path from the skill directory. For example, if this skill is installed at [REDACTED_SECRET]: ```bash python3 /path/to/pangolinfo-amazon-scraper/scripts/pangolin.py --content B0DYTF8L2W --mode amazon --site amz_us ``` ## Intent-to-Command Mapping ### 1. Product Detail by ASIN User says: "Look up B0DYTF8L2W on Amazon" / "查一下 B0DYTF8L2W 这个产品" ```bash python3 scripts/pangolin.py --content B0DYTF8L2W --mode amazon --site amz_us # Or using the --asin shortcut: python3 scripts/pangolin.py --asin B0DYTF8L2W --site amz_us ``` The script auto-detects 10-character alphanumeric codes as ASINs and selects `amzProductDetail`. The `--asin` flag is a convenience shortcut that sets `--content` and defaults to `amzProductDetail` parser automatically. ### 2. Keyword Search User says: "Search Amazon for wireless mouse" / "在亚马逊搜索无线鼠标" ```bash python3 scripts/pangolin.py --q "wireless mouse" --mode amazon --site amz_us ``` Non-ASIN text auto-selects `amzKeyword` parser. ### 3. Bestsellers User says: "Show me electronics bestsellers" / "看看电子产品畅销榜" ```bash python3 scripts/pangolin.py --content "electronics" --mode amazon --parser amzBestSellers --site amz_us ``` ### 4. New Releases User says: "What's new in toys on Amazon?" / "亚马逊玩具类有什么新品？" ```bash python3 scripts/pangolin.py --content "toys" --mode amazon --parser amzNewReleases --site amz_us ``` ### 5. Category Products User says: "Browse category 172282 on Amazon" / "浏览亚马逊分类 172282" ```bash python3 scripts/pangolin.py --content "172282" --mode amazon --parser amzProductOfCategory --site amz_us ``` ### 6. Seller Products User says: "Show products from seller A1B2C3D4E5" / "查看卖家 A1B2C3D4E5 的商品" ```bash python3 scripts/pangolin.py --content "A1B2C3D4E5" --mode amazon --parser amzProductOfSeller --site amz_us ``` ### 7. Follow Seller (Variant / Seller Options) User says: "Show other seller options for B0G4QPYK4Z" / "查看 B0G4QPYK4Z 的其他卖家选项" ```bash python3 scripts/pangolin.py --asin B0G4QPYK4Z --parser amzFollowSeller --site amz_us ``` Uses a dedicated API endpoint (`/api/v1/scrape/follow-seller`). Returns variant options including price, delivery, fulfillment, and seller info for the given ASIN. ### 8. Product Reviews User says: "Get critical reviews for B00163U4LK" / "看看 B00163U4LK 的差评" ```bash python3 scripts/pangolin.py --content B00163U4LK --mode review --site amz_us --filter-star critical ``` **Star filter mapping:** | User says (EN) | User says (CN) | `--filter-star` value | |---|---|---| | all reviews | 所有评论 | `all_stars` | | 5-star / excellent | 五星 / 好评 | `five_star` | | 4-star | 四星 | `four_star` | | 3-star | 三星 | `three_star` | | 2-star | 两星 | `two_star` | | 1-star / terrible | 一星 / 最差评 | `one_star` | | positive / good | 好评 / 正面评价 | `positive` | | critical / negative / bad | 差评 / 负面评价 | `critical` | **Sort mapping:** | User says (EN) | User says (CN) | `--sort-by` value | |---|---|---| | newest / most recent | 最新的 | `recent` | | most helpful / top | 最有帮助的 | `helpful` | **Multiple pages:** Add `--pages N` (each page costs 5 credits). ### 9. Product by URL (Legacy) User says: "Scrape this Amazon page: https://www.amazon.com/dp/B0DYTF8L2W" ```bash python3 scripts/pangolin.py --url "https://www.amazon.com/dp/B0DYTF8L2W" --mode amazon ``` The site code is auto-inferred from the URL domain. ### 10. Different Region User says: "Check price on Amazon Japan" / "看看日本亚马逊的价格" **Region mapping:** | User says (EN) | User says (CN) | `--site` code | |---|---|---| | US / America | 美国 | `amz_us` | | UK / Britain | 英国 | `amz_uk` | | Canada | 加拿大 | `amz_ca` | | Germany | 德国 | `amz_de` | | France | 法国 | `amz_fr` | | Japan | 日本 | `amz_jp` | | Italy | 意大利 | `amz_it` | | Spain | 西班牙 | `amz_es` | | Australia | 澳大利亚 | `amz_au` | | Mexico | 墨西哥 | `amz_mx` | | Saudi Arabia | 沙特 | `amz_sa` | | UAE | 阿联酋 | `amz_ae` | | Brazil | 巴西 | `amz_br` | ## Smart Defaults The script automatically: 1. **Detects ASINs** -- 10-character alphanumeric strings like `B0DYTF8L2W` auto-select `amzProductDetail` 2. **Detects keywords** -- non-ASIN text like `"wireless mouse"` auto-selects `amzKeyword` 3. **Defaults to `amz_us`** if no `--site` is provided 4. **Switches to review mode** if `--filter-star` is used or `--parser amzReviewV2` is set 5. **Infers site from URL** when using `--url` with an Amazon domain You generally only need `--parser` for bestsellers, new releases, category, and seller lookups. ## Output Format ### Amazon Mode (product/keyword/ranking/category/seller) ```json { "success": true, "task_id": [REDACTED_SECRET], "url": "https://www.amazon.com/dp/B0DYTF8L2W", "metadata": { "executionTime": 1791, "parserType": "amzProductDetail", "parsedAt": "2026-01-13T06:42:01.861Z" }, "results": [ ... ], "results_count": 1 } ``` ### Review Mode Same envelope structure. Each result object contains review fields: `date`, `country`, `star`, `author`, `title`, `content`, `purchased`, `vineVoice`, `helpful`, `reviewId`. ### Error Output All errors use a unified envelope written to stderr: ```json { "success": false, "error": { "code": "ERROR_CODE", "message": "Human-readable description", "hint": "Actionable suggestion" } } ``` See [references/output-schema.md](references/output-schema.md) for complete field documentation. ## Response Presentation When presenting results to the user, follow these guidelines per parser type. **Always match the user's language** (respond in Chinese if the user wrote in Chinese, etc.). ### Product Detail (`amzProductDetail`) Present as a structured product card: - **Title** and brand - **Price** (include strikethrough/coupon if present) - **Rating**: star rating + number of reviews - **Stock status** and delivery estimate - **Key features** (bullet list, up to 5) - **Bestseller rank** and category - Include the product image if available - Note Amazon's Choice badge if `acBadge` is present ### Keyword Search (`amzKeyword`) Present as a **numbered list** of products: 1. **Title** -- price -- star rating (N reviews) -- delivery info 2. ... Highlight sponsored results. Show the total results count. If the user asked about a specific need, recommend the best match. ### Bestsellers / New Releases (`amzBestSellers` / `amzNewReleases`) Present as a **ranked list**: 1. #1: **Title** -- price -- star rating 2. #2: ... Emphasize rank position. Note trends if multiple queries are compared. ### Category / Seller Products (`amzProductOfCategory` / `amzProductOfSeller`) Present as a numbered product list similar to keyword search. Group by price range or rating if the user is comparing. ### Follow Seller (`amzFollowSeller`) Present variant options clearly: - Option 1: price, delivery, ships from, sold by - Option 2: ... Help the user compare variants (color, size, seller). ### Reviews (`amzReviewV2`) Present as: 1. **Summary**: average sentiment, common themes, verified purchase ratio 2. **Individual reviews** (top 3-5): star + title + excerpt + date + verified badge 3. **Pattern analysis**: recurring praise/complaints ### Empty Results If `results_count` is 0: - Suggest checking the ASIN/keyword spelling - Suggest trying a different Amazon region - Suggest broadening the search terms ### Error Results If `success` is false: - Present the error message in friendly, non-technical language - Provide the hint from the error object - Offer to retry or suggest an alternative approach ## All CLI Options | Flag | Description | Default | |---|---|---| | `--q` | Search query or keyword | -- | | `--url` | Target Amazon URL (legacy mode) | -- | | `--content` | Content ID: ASIN, keyword, category Node ID, seller ID | -- | | `--asin ASIN` | Amazon ASIN (shortcut for `--content <ASIN> --parser amzProductDetail`) | -- | | `--site` / `--region` | Amazon site/region code (e.g., `amz_us`) | `amz_us` | | `--mode` | `amazon` or `review` | `amazon` | | `--parser` | Parser name (auto-inferred when possible) | `amzProductDetail` | | `--zipcode` | US zipcode for localized pricing | `10041` | | `--format` | Response format: `json`, `rawHtml`, `markdown` | `json` | | `--filter-star` | Star filter for reviews | `all_stars` | | `--sort-by` | Sort order for reviews: `recent` or `helpful` | `recent` | | `--pages` | Number of review pages (5 credits/page) | `1` | | `--auth-only` | Test authentication only | -- | | `--raw` | Output raw API response (no extraction) | -- | | `--timeout` | API request timeout in seconds | `120` | ## Output Schema See [references/output-schema.md](references/output-schema.md) for the complete JSON output schema, including per-parser field documentation and guaranteed vs optional fields. ## Amazon Parsers Reference | Parser | Use Case | Content Value | |---|---|---| | `amzProductDetail` | Single product page | ASIN (e.g., `B0DYTF8L2W`) | | `amzKeyword` | Keyword search results | Keyword (e.g., `wireless mouse`) | | `amzProductOfCategory` | Category listing | Category Node ID (e.g., `172282`) | | `amzProductOfSeller` | Seller's products | Seller ID (e.g., `A1B2C3D4E5`) | | `amzBestSellers` | Best sellers ranking | Category keyword (e.g., `electronics`) | | `amzNewReleases` | New releases ranking | Category keyword (e.g., `toys`) | | `amzFollowSeller` | Variants / other sellers | ASIN (e.g., `B0DYTF8L2W`) | | `amzReviewV2` | Product reviews | ASIN (via `--mode review`) | ## Amazon Sites Reference | Site Code | Region | Domain | |---|---|---| | `amz_us` | United States | amazon.com | | `amz_uk` | United Kingdom | amazon.co.uk | | `amz_ca` | Canada | amazon.ca | | `amz_de` | Germany | amazon.de | | `amz_fr` | France | amazon.fr | | `amz_jp` | Japan | amazon.co.jp | | `amz_it` | Italy | amazon.it | | `amz_es` | Spain | amazon.es | | `amz_au` | Australia | amazon.com.au | | `amz_mx` | Mexico | amazon.com.mx | | `amz_sa` | Saudi Arabia | amazon.sa | | `amz_ae` | UAE | amazon.ae | | `amz_br` | Brazil | amazon.com.br | ## Cost | Operation | Credits | |---|---| | Amazon scrape (`json` format) | 1 per request | | Amazon scrape (`rawHtml` / `markdown`) | 0.75 per request | | Follow Seller | 1 per request | | Review page | 5 per page | Credits are only consumed on successful requests (API code 0). Average response time is approximately 5 seconds. ## Exit Codes | Code | Meaning | |---|---| | 0 | Success | | 1 | API error (non-zero response code) | | 2 | Usage error (bad arguments) | | 3 | Network error (connection, SSL, rate limit) | | 4 | Authentication error | ## Troubleshooting | Error Code | Meaning | User-Friendly Message | Resolution | |---|---|---|---| | `MISSING_ENV` | No credentials set | "I need your Pangolin credentials to access Amazon data." | Set `PANGOLIN_EMAIL` + `PANGOLIN_PASSWORD` or `PANGOLIN_API_KEY` | | `AUTH_FAILED` | Bad email/password | "Authentication failed. Please check your Pangolin credentials." | Verify email and password | | `RATE_LIMIT` | Too many requests | "The API is rate-limiting us. Let me wait and retry." | Wait and retry | | `NETWORK` | Connection failed | "I couldn't reach the API. Please check your connection." | Check internet, retry | | `SSL_CERT` | SSL verification failed | "SSL error -- likely a macOS Python issue." | See macOS SSL section above | | `API_ERROR` | API returned error | "The API returned an error." | Check error details, retry | | `USAGE_ERROR` | Bad CLI arguments | "I need more information to run this query." | Provide required arguments | **API error codes from the server:** | API Code | Meaning | Resolution | |---|---|---| | 1004 | Expired/invalid API key | Auto-refreshed by the script | | 2001 | Insufficient credits | Top up at pangolinfo.com | | 2007 | Account expired | Renew subscription | | 10000/10001 | Task execution failed | Retry the request | | 404 | Bad URL format | Verify the target URL | ## Important Notes for AI Agents 1. **Never dump raw JSON** to the user. Always parse and present results in a structured, readable format following the Response Presentation guidelines above. 2. **Match the user's language.** If the user writes in Chinese, respond in Chinese. If English, respond in English. This applies to product descriptions, error messages, and suggestions. 3. **Be proactive.** If a keyword search returns results, highlight the best match. If reviews show a pattern, summarize it. If a product is out of stock, say so immediately. 4. **Handle multi-step requests.** If the user says "compare this product on US and Japan Amazon," run two queries with different `--site` values and present a comparison table. 5. **Credit awareness.** Each request costs credits. Avoid unnecessary duplicate requests. If the user asks for reviews across multiple pages, warn them about the cost (5 credits per page). 6. **Error recovery.** If a request fails, check the error code and suggest a fix before retrying. Do not retry blindly. 7. **ASIN detection.** If the user provides a 10-character code starting with B0, it is almost certainly an ASIN. Auto-detect and use `--content` with no extra prompting. 8. **Default to `amz_us`** unless the user specifies a region or the context implies a different marketplace. 9. **Security.** Never expose API keys, passwords, or full API responses containing authentication data in the conversation. 10. **Combine results.** When the user asks a compound question (e.g., "find a good wireless mouse under $30 with good reviews"), run a keyword search first, then fetch reviews for the top candidates. Present a consolidated recommendation.

path: references/amazon-api.md
content: # Amazon Scrape API Reference

Scrape Amazon product data: details, search results, category listings, best sellers, and more.

## Endpoint

```
POST https://scrapeapi.pangolinfo.com/api/v1/scrape
```

Note: Amazon uses **v1** endpoint, not v2 (which is used by Google SERP APIs).

**Headers:**
- `Content-Type: application/json`
- `Authorization: Bearer <token>`

## Request Body

| Field | Required | Type | Description |
|-------|----------|------|-------------|
| url | Conditional | string | Amazon page URL. Not required if `site` + `content` are provided. |
| site | Conditional | string | Amazon site code (e.g. `amz_us`). Not required if `url` is provided. |
| content | Conditional | string | Content identifier (see content mapping below). Not required if `url` is provided. |
| parserName | Yes | string | Parser type (see below) |
| format | Yes | string | `"json"`, `"rawHtml"`, or `"markdown"` |
| bizContext | Yes | object | Business context |
| bizContext.zipcode | Yes | string | Zipcode for localized pricing |

### Input Modes

**Mode 1: URL-based (legacy)**
Provide `url` directly. `site` and `content` are optional.

```json
{
  "url": "https://www.amazon.com/dp/B0DYTF8L2W",
  "parserName": "amzProductDetail",
  "format": "json",
  "bizContext": {"zipcode": "10041"}
}
```

**Mode 2: Content-based (recommended)**
Provide `site` + `content`. No URL needed.

```json
{
  "site": "amz_us",
  "content": "B0DYTF8L2W",
  "parserName": "amzProductDetail",
  "format": "json",
  "bizContext": {"zipcode": "10041"}
}
```

### Content Mapping by Parser

| Parser | content value | Example |
|--------|-------------|---------|
| `amzProductDetail` | ASIN | `B0DYTF8L2W` |
| `amzKeyword` | Keyword | `wireless mouse` |
| `amzProductOfCategory` | Category Node ID | `172282` |
| `amzProductOfSeller` | Seller ID | `A1B2C3D4E5` |
| `amzBestSellers` | Category keyword | `electronics` |
| `amzNewReleases` | Category keyword | `toys` |
| `amzFollowSeller` | ASIN | `B0DYTF8L2W` |

### Amazon Site Codes

| Site Code | Region | Domain |
|-----------|--------|--------|
| `amz_us` | United States | amazon.com |
| `amz_uk` | United Kingdom | amazon.co.uk |
| `amz_ca` | Canada | amazon.ca |
| `amz_de` | Germany | amazon.de |
| `amz_fr` | France | amazon.fr |
| `amz_jp` | Japan | amazon.co.jp |
| `amz_it` | Italy | amazon.it |
| `amz_es` | Spain | amazon.es |
| `amz_au` | Australia | amazon.com.au |
| `amz_mx` | Mexico | amazon.com.mx |
| `amz_sa` | Saudi Arabia | amazon.sa |
| `amz_ae` | UAE | amazon.ae |
| `amz_br` | Brazil | amazon.com.br |

## Parser Types

| Parser | Use Case |
|--------|----------|
| `amzProductDetail` | Single product page |
| `amzKeyword` | Keyword search results |
| `amzProductOfCategory` | Category listing |
| `amzProductOfSeller` | Seller's products |
| `amzBestSellers` | Best sellers ranking |
| `amzNewReleases` | New releases ranking |
| `amzFollowSeller` | Variants / other sellers |

## Response

```json
{
  "code": 0,
  "message": "ok",
  "data": {
    "json": [
      {
        "metadata": {
          "executionTime": 1791,
          "parserType": "amzProductDetail",
          "parsedAt": "2026-01-13T06:42:01.861Z"
        },
        "code": 0,
        "data": {
          "results": [
            {
              "asin": "B0DYTF8L2W",
              "title": "Product Title",
              "brand": "Brand Name",
              "price": "$29.99",
              "star": "4.5",
              "rating": "1,234",
              "image": "https://m.media-amazon.com/images/...",
              "images": ["url1", "url2"],
              "inStock": "In Stock",
              "seller": "Seller Name",
              "features": ["Feature 1", "Feature 2"],
              "category_name": "Electronics",
              "bestSellersRank": "#1,234",
              "deliveryTime": "Tomorrow",
              "sales": "10K+ bought in past month"
            }
          ]
        }
      }
    ],
    "url": "https://www.amazon.com/dp/B0DYTF8L2W",
    "taskId": "02b3e90810f0450ca6d41244d6009d0f"
  }
}
```

### Response Metadata

| Field | Type | Description |
|-------|------|-------------|
| metadata.executionTime | int | Execution time in milliseconds |
| metadata.parserType | string | Parser used for this request |
| metadata.parsedAt | string | ISO timestamp of when parsing completed |

### Product Detail Fields (amzProductDetail)

**Core:**
`asin`, `title`, `brand`, `price`, `strikethroughPrice`, `star`, `rating`, `image`, `images`, `highResolutionImages`, `galleryThumbnails`

**Inventory & Seller:**
`inStock`, `merchant_id`, `seller`, `shipper`, `has_cart`

**Attributes:**
`color`, `size`, `features`, `badge`, `acBadge` (Amazon's Choice)

**Category & Ranking:**
`category_id`, `category_name`, `bestSellersRank`

**Dimensions:**
`pkg_dims`, `pkg_weight`, `product_dims`, `product_weight`

**Reviews & AI:**
`ratingDistribution`, `reviews`, `aiReviewsSummary`

**Structured Data:**
`productDescription`, `productOverview`, `attributes`, `otherAsins`

**Additional:**
`deliveryTime`, `first_date`, `coupon`, `sales`

### Keyword Search Fields (amzKeyword)

`asin`, `title`, `price`, `star`, `rating`, `image`, `sales`, `rank`, `sponsored`, `spRank`, `badge`, `delivery`

### Category/Seller Product Fields (amzProductOfCategory / amzProductOfSeller)

`asin`, `title`, `price`, `star`, `rating`, `image`

### Ranking Fields (amzBestSellers / amzNewReleases)

`asin`, `rank`, `title`, `price`, `star`, `rating`, `image`

### Variant Fields (amzFollowSeller)

`options`, `price`, `delivery`, `shipsFrom`, `soldBy`

## Amazon Review API

Fetch product reviews with star filtering and sorting. Uses the same v1 endpoint.

### Request Body

```json
{
  "url": "https://www.amazon.com",
  "format": "json",
  "parserName": "amzReviewV2",
  "bizContext": {
    "bizKey": "review",
    "asin": "B00163U4LK",
    "pageCount": 1,
    "filterByStar": "critical",
    "sortBy": "recent"
  }
}
```

### bizContext Parameters

| Field | Type | Description |
|-------|------|-------------|
| bizKey | string | Must be `"review"` |
| asin | string | Amazon product ASIN |
| pageCount | int | Number of pages to fetch (5 credits per page) |
| filterByStar | string | Star filter (see below) |
| sortBy | string | `"recent"` or `"helpful"` |

### filterByStar Options

| Value | Description |
|-------|-------------|
| `all_stars` | All reviews |
| `five_star` | 5-star reviews only |
| `four_star` | 4-star reviews only |
| `three_star` | 3-star reviews only |
| `two_star` | 2-star reviews only |
| `one_star` | 1-star reviews only |
| `positive` | Positive reviews (4-5 stars) |
| `critical` | Critical reviews (1-3 stars) |

### Review Response Fields

Each review object contains:

`date`, `country`, `star`, `reviewLink`, `author`, `title`, `authorId`, `content`, `purchased`, `vineVoice`, `authorLink`, `attributes`, `helpful`, `reviewId`

### Review Response Example

```json
{
  "code": 0,
  "message": "ok",
  "data": {
    "json": [
      {
        "metadata": {"executionTime": 153, "parserType": "amzReviewV2", "parsedAt": "..."},
        "code": 0,
        "data": {
          "results": [
            {
              "date": "2026-01-13",
              "country": "United States",
              "star": "5.0",
              "author": "Tucker&Emily",
              "title": "Perfect",
              "content": "Great coat. It fits well and is very attractive.",
              "purchased": true,
              "vineVoice": false,
              "helpful": "",
              "reviewId": "R1IIVEFNGPSLW3"
            }
          ]
        }
      }
    ],
    "url": "https://www.amazon.com",
    "taskId": "..."
  }
}
```

## Cost

| Format | Credits |
|--------|---------|
| Scrape `json` | 1 per request |
| Scrape `rawHtml` / `markdown` | 0.75 per request |
| Reviews | 5 per page |

Average response time: ~5 seconds.


path: references/error-codes.md
content: # Error Codes and Troubleshooting

## Error Code Reference

| Code | Meaning | Resolution |
|------|---------|------------|
| 0 | Success | No action needed |
| 1004 | Invalid or expired token | Re-authenticate with email/password. The script does this automatically. |
| 2001 | Insufficient credits | Top up credits at pangolinfo.com |
| 2007 | Account expired | Renew account subscription at pangolinfo.com |
| 10000 | Task execution failed | Retry the request. If persistent, check query/URL format. |
| 10001 | Task execution failed | Retry the request. May indicate a temporary server issue. |
| 404 | Incorrect URL address | Verify the target URL format |

## Authentication

### Token Lifecycle

- Tokens are **permanent** -- they do not expire on their own
- A token becomes invalid only if the account is deactivated
- Error code `1004` indicates the token needs to be refreshed
- The script caches tokens at `~/.pangolin_token`

### Token Resolution Order

1. `PANGOLIN_TOKEN` environment variable
2. Cached token at `~/.pangolin_token`
3. Fresh login using `PANGOLIN_EMAIL` + `PANGOLIN_PASSWORD`

### Auth Endpoint

```
POST https://scrapeapi.pangolinfo.com/api/v1/auth
Body: {"email": "<email>", "password": "<password>"}
Response: {"code": 0, "message": "ok", "data": "<token>"}
```

## Credit Management

- AI Mode API: **2 credits** per request
- AI Overview SERP API: **2 credits** per request
- Amazon Scrape API (json): **1 credit** per request
- Amazon Scrape API (rawHtml/markdown): **0.75 credits** per request
- Credits are only consumed on successful requests (code 0)
- Check your credit balance at [pangolinfo.com](https://www.pangolinfo.com)

## Common Issues

**"No authentication credentials" error**
Set environment variables: `export PANGOLIN_EMAIL=... PANGOLIN_PASSWORD=...`

**Empty AI overview in response**
Not all queries trigger an AI overview. Check `ai_overview` field in response. Try a more informational query.

**Timeout or network errors**
The script retries 3 times with exponential backoff. If all retries fail, check your network connection and the API status.

**Screenshot URL not returned**
Ensure `--screenshot` flag is passed (or `"screenshot": true` in request body).


path: references/examples/keyword-search-example.json
content: {
  "success": true,
  "task_id": "a1b2c3d4e5f6470890abcdef12345678",
  "url": "https://www.amazon.com/s?k=wireless+mouse",
  "metadata": {
    "executionTime": 2134,
    "parserType": "amzKeyword",
    "parsedAt": "2026-01-15T10:23:45.123Z"
  },
  "results": [
    {
      "asin": "B09HMJ5LRZ",
      "title": "Logitech Pebble Mouse 2 M750 Wireless Bluetooth Mouse - Portable, Customizable, Quiet Clicks, Multi-Device",
      "price": "$29.99",
      "star": "4.6",
      "rating": "8,432",
      "image": "https://m.media-amazon.com/images/I/61example1.jpg",
      "sales": "10K+ bought in past month",
      "rank": 1,
      "sponsored": false,
      "spRank": null,
      "badge": "Best Seller",
      "delivery": "FREE delivery Wed, Jan 17"
    },
    {
      "asin": "B0CX4TM3K7",
      "title": "Amazon Basics Ergonomic Wireless Mouse - DPI Adjustable - Black",
      "price": "$14.99",
      "star": "4.3",
      "rating": "2,156",
      "image": "https://m.media-amazon.com/images/I/71example2.jpg",
      "sales": "5K+ bought in past month",
      "rank": 2,
      "sponsored": false,
      "spRank": null,
      "badge": "",
      "delivery": "FREE delivery Thu, Jan 18"
    },
    {
      "asin": "B0DPR4EXAMPLE",
      "title": "Wireless Mouse, 2.4G Slim Portable Optical Mouse with USB Receiver, 1600 DPI, 6 Buttons, for Laptop Desktop",
      "price": "$9.99",
      "star": "4.1",
      "rating": "15,678",
      "image": "https://m.media-amazon.com/images/I/51example3.jpg",
      "sales": "20K+ bought in past month",
      "rank": 3,
      "sponsored": true,
      "spRank": 1,
      "badge": "Amazon's Choice",
      "delivery": "FREE delivery Fri, Jan 19"
    }
  ],
  "results_count": 3
}


path: references/examples/product-detail-example.json
content: {
  "success": true,
  "task_id": "f8e7d6c5b4a3021098fedcba87654321",
  "url": "https://www.amazon.com/dp/B0DYTF8L2W",
  "metadata": {
    "executionTime": 1791,
    "parserType": "amzProductDetail",
    "parsedAt": "2026-01-13T06:42:01.861Z"
  },
  "results": [
    {
      "asin": "B0DYTF8L2W",
      "title": "Wireless Bluetooth Mouse, Multi-Device Ergonomic Mouse with Silent Clicks, 2.4G + BT5.0 Dual Mode, USB-C Rechargeable, 4 DPI Levels",
      "brand": "TechComfort",
      "price": "$24.99",
      "strikethroughPrice": "$39.99",
      "star": "4.5",
      "rating": "3,456",
      "image": "https://m.media-amazon.com/images/I/71detailmain.jpg",
      "images": [
        "https://m.media-amazon.com/images/I/71detailmain.jpg",
        "https://m.media-amazon.com/images/I/71detailside.jpg",
        "https://m.media-amazon.com/images/I/71detailback.jpg"
      ],
      "inStock": "In Stock",
      "seller": "TechComfort Official",
      "shipper": "Amazon",
      "merchant_id": "A3EXAMPLE1234",
      "has_cart": true,
      "features": [
        "Dual-mode connectivity: seamlessly switch between Bluetooth 5.0 and 2.4G USB receiver",
        "Ergonomic design with thumb rest reduces wrist strain during extended use",
        "Silent click technology for quiet operation in offices and libraries",
        "USB-C rechargeable battery lasts up to 60 days on a single charge",
        "4 adjustable DPI levels (800/1200/1600/2400) for precise cursor control"
      ],
      "badge": "",
      "acBadge": "Amazon's Choice",
      "color": "Space Gray",
      "size": "Standard",
      "category_id": "172282",
      "category_name": "Computer Mice",
      "bestSellersRank": "#1,234 in Computer Mice",
      "deliveryTime": "Tomorrow",
      "first_date": "2025-11-01",
      "coupon": "Save 10% with coupon",
      "sales": "10K+ bought in past month",
      "pkg_dims": "5.91 x 3.54 x 1.97 inches",
      "pkg_weight": "3.53 ounces",
      "product_dims": "4.72 x 2.76 x 1.57 inches",
      "product_weight": "2.82 ounces",
      "ratingDistribution": {
        "5star": "68%",
        "4star": "18%",
        "3star": "8%",
        "2star": "3%",
        "1star": "3%"
      },
      "reviews": [
        {
          "title": "Best mouse I've ever owned",
          "content": "Incredibly comfortable and the silent clicks are a game changer.",
          "star": "5.0",
          "author": "TechReviewer42"
        }
      ],
      "aiReviewsSummary": "Customers praise the ergonomic design and silent clicks. Some note the Bluetooth connection can be slow to wake from sleep mode.",
      "productDescription": "Experience premium wireless mouse technology with TechComfort's dual-mode Bluetooth and 2.4G connectivity...",
      "productOverview": {
        "Brand": "TechComfort",
        "Connectivity": "Bluetooth 5.0, 2.4G Wireless",
        "Battery": "USB-C Rechargeable, 60-day life",
        "DPI": "800/1200/1600/2400"
      },
      "attributes": [
        {"name": "Color", "value": "Space Gray"},
        {"name": "Connectivity Technology", "value": "Bluetooth, USB"},
        {"name": "Special Feature", "value": "Ergonomic, Silent Click, Rechargeable"}
      ],
      "otherAsins": ["B0DYTF8M3X", "B0DYTF8N4Y"]
    }
  ],
  "results_count": 1
}


path: references/output-schema.md
content: # Output Schema Reference

This document describes the JSON output structure produced by `scripts/pangolin.py`.

## Success Envelope

All successful responses follow this structure:

```json
{
  "success": true,
  "task_id": "02b3e90810f0450ca6d41244d6009d0f",
  "url": "https://www.amazon.com/dp/B0DYTF8L2W",
  "metadata": {
    "executionTime": 1791,
    "parserType": "amzProductDetail",
    "parsedAt": "2026-01-13T06:42:01.861Z"
  },
  "results": [ ... ],
  "results_count": 1
}
```

| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Always `true` for successful responses |
| `task_id` | string | Unique task identifier for this request |
| `url` | string | The URL that was scraped |
| `metadata` | object | Execution metadata (may be absent in legacy format) |
| `metadata.executionTime` | int | Execution time in milliseconds |
| `metadata.parserType` | string | Parser used for this request |
| `metadata.parsedAt` | string | ISO 8601 timestamp of when parsing completed |
| `results` | array | Array of result objects (structure varies by parser) |
| `results_count` | int | Number of items in `results` |

## Error Envelope

All errors (both script-level and API-level) follow this structure:

```json
{
  "success": false,
  "error": {
    "code": "ERROR_CODE",
    "message": "Human-readable error description",
    "hint": "Actionable suggestion to resolve the issue"
  }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Always `false` for errors |
| `error.code` | string | Machine-readable error code (see table below) |
| `error.message` | string | Human-readable error description |
| `error.hint` | string | Actionable suggestion (optional for some errors) |

### Error Codes

| Code | Meaning | Typical Exit Code |
|------|---------|-------------------|
| `MISSING_ENV` | No credentials configured | 4 |
| `AUTH_FAILED` | Email/password incorrect | 4 |
| `RATE_LIMIT` | HTTP 429 rate limiting | 3 |
| `NETWORK` | Connection or HTTP error | 3 |
| `SSL_CERT` | SSL certificate verification failed | 3 |
| `API_ERROR` | Pangolin API returned non-zero code | 1 |
| `USAGE_ERROR` | Invalid CLI arguments or missing required inputs | 2 |

## Per-Parser Result Fields

### amzProductDetail

Single product detail page. Returns 1 result object.

**Guaranteed fields:**

| Field | Type | Example |
|-------|------|---------|
| `asin` | string | `"B0DYTF8L2W"` |
| `title` | string | `"Wireless Mouse, 2.4G Ergonomic..."` |
| `price` | string | `"$29.99"` |
| `image` | string | `"https://m.media-amazon.com/images/I/..."` |

**Common fields (usually present):**

| Field | Type | Description |
|-------|------|-------------|
| `brand` | string | Brand name |
| `star` | string | Star rating, e.g., `"4.5"` |
| `rating` | string | Number of ratings, e.g., `"1,234"` |
| `strikethroughPrice` | string | Original price before discount |
| `images` | array[string] | Additional product images |
| `highResolutionImages` | array[string] | High-resolution image URLs |
| `galleryThumbnails` | array[string] | Gallery thumbnail URLs |
| `inStock` | string | Stock status, e.g., `"In Stock"` |
| `seller` | string | Seller name |
| `shipper` | string | Shipping provider |
| `merchant_id` | string | Merchant identifier |
| `has_cart` | boolean | Whether "Add to Cart" is available |
| `features` | array[string] | Bullet-point feature list |
| `badge` | string | Product badge text |
| `acBadge` | string | Amazon's Choice badge text |
| `color` | string | Selected color variant |
| `size` | string | Selected size variant |
| `category_id` | string | Category node ID |
| `category_name` | string | Category name |
| `bestSellersRank` | string | BSR string, e.g., `"#1,234"` |
| `deliveryTime` | string | Delivery estimate, e.g., `"Tomorrow"` |
| `first_date` | string | First available date |
| `coupon` | string | Active coupon text |
| `sales` | string | Sales volume, e.g., `"10K+ bought in past month"` |
| `pkg_dims` | string | Package dimensions |
| `pkg_weight` | string | Package weight |
| `product_dims` | string | Product dimensions |
| `product_weight` | string | Product weight |
| `ratingDistribution` | object | Star distribution breakdown |
| `reviews` | array | Embedded review snippets |
| `aiReviewsSummary` | string | AI-generated review summary |
| `productDescription` | string | Full product description |
| `productOverview` | object | Structured product overview |
| `attributes` | array | Product attribute key-value pairs |
| `otherAsins` | array | Related/variant ASINs |

### amzKeyword

Keyword search results. Returns multiple result objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `asin` | string | Yes | Product ASIN |
| `title` | string | Yes | Product title |
| `price` | string | Yes | Price string |
| `star` | string | No | Star rating |
| `rating` | string | No | Number of ratings |
| `image` | string | Yes | Product image URL |
| `sales` | string | No | Sales volume text |
| `rank` | int | No | Position in search results |
| `sponsored` | boolean | No | Whether the listing is sponsored |
| `spRank` | int | No | Sponsored rank position |
| `badge` | string | No | Badge text (e.g., "Best Seller") |
| `delivery` | string | No | Delivery info |

### amzProductOfCategory

Category listing results. Returns multiple result objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `asin` | string | Yes | Product ASIN |
| `title` | string | Yes | Product title |
| `price` | string | Yes | Price string |
| `star` | string | No | Star rating |
| `rating` | string | No | Number of ratings |
| `image` | string | Yes | Product image URL |

### amzProductOfSeller

Seller product listing. Returns multiple result objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `asin` | string | Yes | Product ASIN |
| `title` | string | Yes | Product title |
| `price` | string | Yes | Price string |
| `star` | string | No | Star rating |
| `rating` | string | No | Number of ratings |
| `image` | string | Yes | Product image URL |

### amzBestSellers

Bestseller ranking results. Returns multiple result objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `asin` | string | Yes | Product ASIN |
| `rank` | int | Yes | Bestseller rank position |
| `title` | string | Yes | Product title |
| `price` | string | Yes | Price string |
| `star` | string | No | Star rating |
| `rating` | string | No | Number of ratings |
| `image` | string | Yes | Product image URL |

### amzNewReleases

New releases ranking. Returns multiple result objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `asin` | string | Yes | Product ASIN |
| `rank` | int | Yes | New release rank position |
| `title` | string | Yes | Product title |
| `price` | string | Yes | Price string |
| `star` | string | No | Star rating |
| `rating` | string | No | Number of ratings |
| `image` | string | Yes | Product image URL |

### amzFollowSeller

Seller/variant options for a product. Returns 1 result object with nested data.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `options` | array | Yes | Available variant/seller options |
| `price` | string | Yes | Price for the current option |
| `delivery` | string | No | Delivery estimate |
| `shipsFrom` | string | No | Shipping origin |
| `soldBy` | string | No | Seller name |

### amzReviewV2

Product reviews. Returns multiple review objects.

| Field | Type | Guaranteed | Description |
|-------|------|------------|-------------|
| `date` | string | Yes | Review date (e.g., `"2026-01-13"`) |
| `country` | string | Yes | Reviewer's country |
| `star` | string | Yes | Star rating (e.g., `"5.0"`) |
| `author` | string | Yes | Reviewer name |
| `title` | string | Yes | Review title |
| `content` | string | Yes | Review body text |
| `purchased` | boolean | Yes | Whether the reviewer made a verified purchase |
| `vineVoice` | boolean | Yes | Whether the reviewer is a Vine Voice |
| `helpful` | string | No | Helpful vote text (e.g., `"5 people found this helpful"`) |
| `reviewId` | string | Yes | Unique review identifier |
| `reviewLink` | string | No | URL to the review |
| `authorId` | string | No | Author's Amazon ID |
| `authorLink` | string | No | URL to the author's profile |
| `attributes` | array | No | Additional review attributes |

## Field Presence Notes

- **Guaranteed** fields are always present in the result object when the API returns successfully.
- **Optional/Common** fields depend on the product listing, availability of data on Amazon, and the specific Amazon marketplace.
- Fields may be empty strings (`""`) or `null` even when present.
- Price formats vary by region (e.g., `"$29.99"`, `"EUR 24,99"`, `"2,980"` for JPY).
- Star ratings are returned as strings, not numbers.
- The `results` array may be empty (`[]`) if no matching data was found.


path: scripts/pangolin.py
content: #!/usr/bin/env python3
"""
Pangolin Amazon Scrape API Client

Zero-dependency Python client for Pangolin's Amazon Scrape APIs.
Supports product detail lookup, keyword search, bestsellers, new releases,
category browsing, seller products, variant options, and product reviews.

Semantic usage (no URL required):
    pangolin.py --content B0DYTF8L2W --mode amazon --site amz_us
    pangolin.py --q "wireless mouse" --mode amazon --site amz_us
    pangolin.py --content B00163U4LK --mode review --site amz_us

Legacy usage (URL-based):
    pangolin.py --url "https://amazon.com/dp/..." --mode amazon

Environment:
    PANGOLIN_API_KEY  - API key (skips login)
    PANGOLIN_EMAIL    - Account email (for login)
    PANGOLIN_PASSWORD - Account password (for login)
"""

import argparse
import io
import json
import os
import re
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path

# ---------------------------------------------------------------------------
# Fix Windows / macOS console encoding for Unicode output
# ---------------------------------------------------------------------------
if sys.stdout.encoding and sys.stdout.encoding.lower() != "utf-8":
    sys.stdout = io.TextIOWrapper(
        sys.stdout.buffer, encoding="utf-8", errors="replace"
    )
if sys.stderr.encoding and sys.stderr.encoding.lower() != "utf-8":
    sys.stderr = io.TextIOWrapper(
        sys.stderr.buffer, encoding="utf-8", errors="replace"
    )

# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
API_BASE = "https://scrapeapi.pangolinfo.com"
AUTH_ENDPOINT = f"{API_BASE}/api/v1/auth"
SCRAPE_V1_ENDPOINT = f"{API_BASE}/api/v1/scrape"
FOLLOW_SELLER_ENDPOINT = f"{API_BASE}/api/v1/scrape/follow-seller"
API_KEY_CACHE_PATH = Path.home() / ".pangolin_api_key"

EXIT_SUCCESS = 0
EXIT_API_ERROR = 1
EXIT_USAGE_ERROR = 2
EXIT_NETWORK_ERROR = 3
EXIT_AUTH_ERROR = 4

AMAZON_PARSERS = [
    "amzProductDetail",
    "amzKeyword",
    "amzProductOfCategory",
    "amzProductOfSeller",
    "amzBestSellers",
    "amzNewReleases",
    "amzFollowSeller",
    "amzReviewV2",
]

# Review filter and sort options
REVIEW_STAR_FILTERS = [
    "all_stars",
    "five_star",
    "four_star",
    "three_star",
    "two_star",
    "one_star",
    "positive",
    "critical",
]

REVIEW_SORT_OPTIONS = [
    "recent",
    "helpful",
]

# Amazon site codes mapped to region domains
AMAZON_SITES = {
    "amz_us": "amazon.com",
    "amz_uk": "amazon.co.uk",
    "amz_ca": "amazon.ca",
    "amz_de": "amazon.de",
    "amz_fr": "amazon.fr",
    "amz_jp": "amazon.co.jp",
    "amz_it": "amazon.it",
    "amz_es": "amazon.es",
    "amz_au": "amazon.com.au",
    "amz_mx": "amazon.com.mx",
    "amz_sa": "amazon.sa",
    "amz_ae": "amazon.ae",
    "amz_br": "amazon.com.br",
}

# Regex to detect ASIN pattern (10-character alphanumeric, uppercase)
ASIN_PATTERN = re.compile(r"^[A-Z0-9]{10}$", re.IGNORECASE)


# ---------------------------------------------------------------------------
# Unified error helpers
# ---------------------------------------------------------------------------
def _error_exit(code, message, hint=None, exit_code=EXIT_API_ERROR):
    """Print a structured error envelope to stderr and exit."""
    err = {"success": False, "error": {"code": code, "message": message}}
    if hint:
        err["error"]["hint"] = hint
    print(json.dumps(err, ensure_ascii=False), file=sys.stderr)
    sys.exit(exit_code)


# ---------------------------------------------------------------------------
# API key management
# ---------------------------------------------------------------------------
def load_cached_api_key():
    """Load API key from cache file if it exists."""
    if API_KEY_CACHE_PATH.exists():
        [REDACTED_SECRET]).strip()
        if api_key and len(api_key.split(".")) == 3:  # Basic JWT format check
            return api_key
    return None


def save_cached_api_key(api_key):
    """Save API key to cache file."""
    API_KEY_CACHE_PATH.write_text(api_key)
    try:
        API_KEY_CACHE_PATH.chmod(0o600)
    except OSError:
        pass
    # Windows: restrict file access to current user
    if sys.platform == "win32":
        try:
            import subprocess
            subprocess.run(
                ["icacls", str(API_KEY_CACHE_PATH), "/inheritance:r",
                 "/grant:r", f"{os.environ.get('USERNAME', '')}:F"],
                capture_output=True, check=False,
            )
        except Exception:
            pass


def authenticate(email, password):
    """Authenticate with Pangolin API and return an API key."""
    body = json.dumps({"email": email, "password": password}).encode()
    req = urllib.request.Request(
        AUTH_ENDPOINT,
        data=body,
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            result = json.loads(resp.read())
    except urllib.error.URLError as e:
        # Improvement 1: macOS SSL certificate handling
        if "CERTIFICATE_VERIFY_FAILED" in str(e) or "SSL" in str(e):
            _error_exit(
                "SSL_CERT",
                "SSL certificate verification failed",
                hint=(
                    "macOS users: run '/Applications/Python 3.x/Install Certificates.command' "
                    "or set SSL_CERT_FILE env var. "
                    "See: python3 -c \"import certifi; print(certifi.where())\""
                ),
                exit_code=EXIT_NETWORK_ERROR,
            )
        _error_exit(
            "NETWORK",
            f"Network error during authentication: {e.reason if hasattr(e, 'reason') else e}",
            hint="Check your internet connection and try again.",
            exit_code=EXIT_NETWORK_ERROR,
        )

    if result.get("code") != 0:
        _error_exit(
            "AUTH_FAILED",
            result.get("message", "Authentication failed"),
            hint="Verify PANGOLIN_EMAIL and PANGOLIN_PASSWORD are correct.",
            exit_code=EXIT_AUTH_ERROR,
        )

    [REDACTED_SECRET]"data"]
    save_cached_api_key(api_key)
    return api_key


def get_api_key():
    """Resolve API key from env var, cache file, or fresh login."""
    [REDACTED_SECRET]"PANGOLIN_API_KEY")
    if [REDACTED_SECRET])  # Cache for future calls without env var
        return api_key

    [REDACTED_SECRET])
    if [REDACTED_SECRET] api_key

    email = os.environ.get("PANGOLIN_EMAIL")
    [REDACTED_SECRET]"PANGOLIN_PASSWORD")
    if not email or not [REDACTED_SECRET]
            "MISSING_ENV",
            "No authentication credentials found.",
            hint=(
                "Set PANGOLIN_API_KEY, or both PANGOLIN_EMAIL and PANGOLIN_PASSWORD "
                "environment variables."
            ),
            exit_code=EXIT_AUTH_ERROR,
        )

    return authenticate(email, password)


def refresh_api_key():
    """Force re-authentication using email/password."""
    email = os.environ.get("PANGOLIN_EMAIL")
    [REDACTED_SECRET]"PANGOLIN_PASSWORD")
    if not email or not [REDACTED_SECRET]
            "MISSING_ENV",
            "Cannot refresh API key without credentials.",
            hint="Set PANGOLIN_EMAIL and PANGOLIN_PASSWORD environment variables.",
            exit_code=EXIT_AUTH_ERROR,
        )
    return authenticate(email, password)


# ---------------------------------------------------------------------------
# Amazon-specific helpers
# ---------------------------------------------------------------------------
def infer_amazon_parser(content):
    """Infer the best Amazon parser based on content pattern."""
    if not content:
        return None
    if ASIN_PATTERN.match(content):
        return "amzProductDetail"
    return "amzKeyword"


def infer_site_from_url(url):
    """Extract Amazon site code from a URL."""
    # Sort by domain length descending to avoid amazon.com matching amazon.com.au
    for site_code, domain in sorted(AMAZON_SITES.items(), key=lambda x: len(x[1]), reverse=True):
        if domain in url:
            return site_code
    return None


def build_review_body(asin, site, filter_by_star, sort_by, page_count, fmt):
    """Build request body for Amazon Review API.

    Uses bizContext with bizKey=review to fetch product reviews.
    """
    if not asin:
        _error_exit(
            "USAGE_ERROR",
            "Review mode requires an ASIN.",
            hint="Provide --content <ASIN> or --q <ASIN>.",
            exit_code=EXIT_USAGE_ERROR,
        )

    domain = AMAZON_SITES.get(site or "amz_us", "amazon.com")

    body = {
        "url": f"https://www.{domain}",
        "format": fmt,
        "parserName": "amzReviewV2",
        "bizContext": {
            "bizKey": "review",
            "asin": asin,
            "pageCount": page_count,
            "filterByStar": filter_by_star,
            "sortBy": sort_by,
        },
    }
    return body


def build_follow_seller_body(asin, site, zipcode):
    """Build request body for Amazon Follow Seller API."""
    if not asin:
        _error_exit(
            "USAGE_ERROR",
            "Follow Seller mode requires an ASIN.",
            hint="Provide --content <ASIN> or --asin <ASIN>.",
            exit_code=EXIT_USAGE_ERROR,
        )
    domain = AMAZON_SITES.get(site or "amz_us", "amazon.com")
    return {
        "url": f"https://www.{domain}",
        "bizContext": {
            "zipcode": zipcode,
            "asin": asin,
        },
    }


def build_amazon_body(url, query, content, site, parser, zipcode, fmt):
    """Build request body for Amazon Scrape API.

    Supports three input modes:
    1. URL mode: provide url directly (legacy)
    2. Content mode: provide site + content (no URL needed)
    3. Query mode: provide query, auto-infer content and parser
    """
    body = {
        "format": fmt,
        "parserName": parser,
        "bizContext": {
            "zipcode": zipcode,
        },
    }

    # Mode 1: URL provided directly
    if url:
        body["url"] = url
        if site:
            body["site"] = site
        else:
            inferred = infer_site_from_url(url)
            if inferred:
                body["site"] = inferred
        if content:
            body["content"] = content
        return body

    # Mode 2: content + site (semantic mode)
    if content:
        if not site:
            site = "amz_us"
        body["site"] = site
        body["content"] = content
        # Auto-detect parser if user didn't override
        if parser == "amzProductDetail":
            inferred = infer_amazon_parser(content)
            if inferred:
                body["parserName"] = inferred
        return body

    # Mode 3: query as content (most semantic)
    if query:
        if not site:
            site = "amz_us"
        body["site"] = site
        body["content"] = query
        # Query implies keyword search unless parser was explicitly set
        if parser == "amzProductDetail":
            inferred = infer_amazon_parser(query)
            if inferred:
                body["parserName"] = inferred
        return body

    _error_exit(
        "USAGE_ERROR",
        "Amazon mode requires --url, --content, or --q.",
        hint="Provide at least one of: --url <URL>, --content <ASIN/keyword>, --q <keyword>.",
        exit_code=EXIT_USAGE_ERROR,
    )


# ---------------------------------------------------------------------------
# API call with retry
# ---------------------------------------------------------------------------
def call_api(api_key, body, endpoint, max_retries=3, timeout=120):
    """Call the scrape API with retry and exponential backoff."""
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {api_key}",
        "User-Agent": "Pangolin-CLI/1.0",
    }
    payload = json.dumps(body).encode()

    for attempt in range(max_retries):
        try:
            req = urllib.request.Request(
                endpoint,
                data=payload,
                headers=headers,
                method="POST",
            )
            with urllib.request.urlopen(req, timeout=timeout) as resp:
                return json.loads(resp.read())

        except urllib.error.HTTPError as e:
            error_body = e.read().decode() if e.fp else ""
            if e.code == 429:
                if attempt < max_retries - 1:
                    time.sleep(2 ** attempt)
                    continue
                _error_exit(
                    "RATE_LIMIT",
                    "Too many requests. Rate limited by the API.",
                    hint="Wait a moment and retry, or reduce request frequency.",
                    exit_code=EXIT_NETWORK_ERROR,
                )
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)
                continue
            _error_exit(
                "NETWORK",
                f"HTTP {e.code} error from API.",
                hint="Check your request parameters and try again.",
                exit_code=EXIT_NETWORK_ERROR,
            )

        except urllib.error.URLError as e:
            # Improvement 1: macOS SSL certificate handling
            if "CERTIFICATE_VERIFY_FAILED" in str(e) or "SSL" in str(e):
                _error_exit(
                    "SSL_CERT",
                    "SSL certificate verification failed",
                    hint=(
                        "macOS users: run '/Applications/Python 3.x/Install Certificates.command' "
                        "or set SSL_CERT_FILE env var. "
                        "See: python3 -c \"import certifi; print(certifi.where())\""
                    ),
                    exit_code=EXIT_NETWORK_ERROR,
                )
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)
                continue
            _error_exit(
                "NETWORK",
                f"Network error: {e.reason if hasattr(e, 'reason') else e}",
                hint="Check your internet connection and try again.",
                exit_code=EXIT_NETWORK_ERROR,
            )

    return None


def handle_response(result, api_key, body, endpoint, timeout=120):
    """Handle API response, retrying auth on 1004 error."""
    if result.get("code") == 1004:
        new_api_key = refresh_api_key()
        return call_api(new_api_key, body, endpoint, timeout=timeout)
    return result


# ---------------------------------------------------------------------------
# Output extraction
# ---------------------------------------------------------------------------
def extract_amazon_output(result):
    """Extract structured output from Amazon Scrape API response.

    Handles both legacy flat format and new nested format:
    New: data.json[].metadata + data.json[].data.results[]
    Legacy: data.json as flat list/dict
    """
    code = result.get("code")
    if code != 0:
        msg = result.get("message", "Unknown error")
        # Map known API error codes
        if code == 2001:
            return {
                "success": False,
                "error": {
                    "code": "API_ERROR",
                    "message": f"Insufficient credits (code {code})",
                    "hint": "Top up credits at pangolinfo.com.",
                },
            }
        if code == 2007:
            return {
                "success": False,
                "error": {
                    "code": "API_ERROR",
                    "message": f"Account expired (code {code})",
                    "hint": "Renew your subscription at pangolinfo.com.",
                },
            }
        return {
            "success": False,
            "error": {
                "code": "API_ERROR",
                "message": f"API error (code {code}): {msg}",
                "hint": "Retry the request. If persistent, check query/URL format.",
            },
        }

    data = result.get("data", {})
    output = {
        "success": True,
        "task_id": data.get("taskId"),
        "url": data.get("url"),
    }

    json_data = data.get("json")

    if isinstance(json_data, list) and len(json_data) > 0:
        first = json_data[0]
        # New nested format: {metadata, code, data: {results: [...]}}
        if isinstance(first, dict) and "metadata" in first:
            output["metadata"] = first["metadata"]
            inner_data = first.get("data", {})
            results = inner_data.get("results", [])
            output["results"] = results
            output["results_count"] = len(results)
        else:
            # Legacy flat format
            output["results"] = json_data
            output["results_count"] = len(json_data)
    elif isinstance(json_data, dict):
        output["results"] = [json_data]
        output["results_count"] = 1
    else:
        output["results"] = []
        output["results_count"] = 0

    return output


# ---------------------------------------------------------------------------
# CLI entry point
# ---------------------------------------------------------------------------
def main():
    parser = argparse.ArgumentParser(
        description="Pangolin Amazon Scrape API Client",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=(
            "Examples:\n"
            "  # Product detail by ASIN\n"
            "  python3 scripts/pangolin.py --content B0DYTF8L2W --mode amazon --site amz_us\n"
            "\n"
            "  # Keyword search\n"
            '  python3 scripts/pangolin.py --q "wireless mouse" --mode amazon --site amz_us\n'
            "\n"
            "  # Bestsellers\n"
            '  python3 scripts/pangolin.py --content "electronics" --mode amazon --parser amzBestSellers --site amz_us\n'
            "\n"
            "  # Critical reviews\n"
            "  python3 scripts/pangolin.py --content B00163U4LK --mode review --site amz_us --filter-star critical\n"
            "\n"
            "  # Auth check\n"
            "  python3 scripts/pangolin.py --auth-only\n"
            "\n"
            "Environment variables:\n"
            "  PANGOLIN_API_KEY    API key (skips login)\n"
            "  PANGOLIN_EMAIL      Account email\n"
            "  PANGOLIN_PASSWORD   Account password\n"
            "\n"
            "Amazon parsers:\n"
            "  amzProductDetail       Product detail page\n"
            "  amzKeyword             Keyword search results\n"
            "  amzProductOfCategory   Category listing\n"
            "  amzProductOfSeller     Seller's products\n"
            "  amzBestSellers         Best sellers ranking\n"
            "  amzNewReleases         New releases ranking\n"
            "  amzFollowSeller        Product variants/seller options\n"
            "\n"
            "Amazon sites:\n"
            "  amz_us  amz_uk  amz_ca  amz_de  amz_fr  amz_jp\n"
            "  amz_it  amz_es  amz_au  amz_mx  amz_sa  amz_ae  amz_br"
        ),
    )
    parser.add_argument("--q", dest="query", help="Search query or keyword")
    parser.add_argument(
        "--url",
        dest="target_url",
        help="Target Amazon URL (optional if --site + --content provided)",
    )
    parser.add_argument(
        "--content",
        dest="content",
        help="Content identifier: ASIN, keyword, category Node ID, seller ID, etc.",
    )
    parser.add_argument(
        "--site", "--region",
        dest="site",
        choices=list(AMAZON_SITES.keys()),
        help="Amazon site/region code (e.g. amz_us, amz_uk). Default: amz_us",
    )
    parser.add_argument(
        "--mode",
        choices=["amazon", "review"],
        default="amazon",
        help="API mode: amazon (default) | review",
    )
    parser.add_argument(
        "--parser",
        choices=AMAZON_PARSERS,
        default="amzProductDetail",
        help="Amazon parser name (default: amzProductDetail, auto-inferred when possible)",
    )
    parser.add_argument(
        "--zipcode",
        default="10041",
        help="Amazon zipcode for localized pricing (default: 10041)",
    )
    parser.add_argument(
        "--format",
        dest="fmt",
        choices=["json", "rawHtml", "markdown"],
        default="json",
        help="Amazon response format (default: json)",
    )
    parser.add_argument(
        "--filter-star",
        dest="filter_star",
        choices=REVIEW_STAR_FILTERS,
        default="all_stars",
        help="Review star filter (review mode only): all_stars, critical, positive, five_star, etc.",
    )
    parser.add_argument(
        "--sort-by",
        dest="sort_by",
        choices=REVIEW_SORT_OPTIONS,
        default="recent",
        help="Review sort order (review mode only): recent | helpful",
    )
    parser.add_argument(
        "--pages",
        dest="page_count",
        type=int,
        default=1,
        help="Number of review pages to fetch (review mode only, 5 credits per page)",
    )
    parser.add_argument(
        "--auth-only",
        action="store_true",
        help="Only authenticate and print API key info",
    )
    parser.add_argument(
        "--asin",
        dest="asin",
        help="Amazon ASIN (shortcut for --content <ASIN> --parser amzProductDetail)",
    )
    parser.add_argument(
        "--raw",
        action="store_true",
        help="Output raw API response instead of extracted data",
    )
    parser.add_argument(
        "--timeout",
        type=int,
        default=120,
        help="API request timeout in seconds (default: 120)",
    )

    args = parser.parse_args()

    # --asin is a convenience shortcut
    if args.asin:
        args.content = args.asin.upper()
        if args.parser == "amzProductDetail":
            pass  # already default
        if not args.mode or args.mode == "amazon":
            args.mode = "amazon"

    if args.page_count < 1:
        parser.error("--pages must be at least 1")

    # Auto-detect review mode from input signals
    if args.mode == "amazon":
        if args.parser == "amzReviewV2" or args.filter_star != "all_stars":
            args.mode = "review"

    if not args.query and not args.target_url and not args.content and not args.auth_only:
        parser.error("--q, --url, or --content is required unless using --auth-only")

    [REDACTED_SECRET])

    if args.auth_only:
        print(
            json.dumps(
                {
                    "success": True,
                    "message": "Authentication successful",
                    "api_key_preview": (
                        f"{api_key[:8]}...{api_key[-4:]}"
                        if len(api_key) > 12
                        else "***"
                    ),
                },
                indent=2,
            )
        )
        sys.exit(EXIT_SUCCESS)

    if args.parser == "amzFollowSeller":
        asin = args.content or args.query
        body = build_follow_seller_body(asin, args.site, args.zipcode)
        endpoint = FOLLOW_SELLER_ENDPOINT
    elif args.mode == "review":
        asin = args.content or args.query
        body = build_review_body(
            asin,
            args.site,
            args.filter_star,
            args.sort_by,
            args.page_count,
            args.fmt,
        )
        endpoint = SCRAPE_V1_ENDPOINT
    else:
        body = build_amazon_body(
            args.target_url,
            args.query,
            args.content,
            args.site,
            args.parser,
            args.zipcode,
            args.fmt,
        )
        endpoint = SCRAPE_V1_ENDPOINT
    result = call_api(api_key, body, endpoint, timeout=args.timeout)

    if result is None:
        _error_exit(
            "NETWORK",
            "API call failed after all retries.",
            hint="Check your internet connection and try again.",
            exit_code=EXIT_NETWORK_ERROR,
        )

    result = handle_response(result, api_key, body, endpoint, timeout=args.timeout)

    if result is None:
        _error_exit(
            "NETWORK",
            "API call failed after API key refresh.",
            hint="Check your internet connection and try again.",
            exit_code=EXIT_NETWORK_ERROR,
        )

    if args.raw:
        print(json.dumps(result, indent=2, ensure_ascii=False))
    else:
        output = extract_amazon_output(result)
        print(json.dumps(output, indent=2, ensure_ascii=False))

    if result.get("code") != 0:
        sys.exit(EXIT_API_ERROR)

    sys.exit(EXIT_SUCCESS)


if __name__ == "__main__":
    main()


path: scripts/self_test.sh
content: #!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PANGOLIN="$SCRIPT_DIR/pangolin.py"

# Cross-platform Python detection
# On Windows, python3 may exist but exit non-zero; test with --version
if python3 --version &>/dev/null 2>&1; then
  PYTHON="python3"
else
  PYTHON="python"
fi

echo "=== Pangolinfo Amazon Scraper Self-Test ==="

# Test 1: Auth check
echo "[1/2] Testing authentication..."
if $PYTHON "$PANGOLIN" --auth-only; then
  echo "  PASS: Auth OK"
else
  echo "  FAIL: Auth failed (exit code $?)"
  exit 1
fi

# Test 2: Minimal Amazon query
echo "[2/2] Testing Amazon keyword search..."
if $PYTHON "$PANGOLIN" --q "test" --mode amazon --site amz_us 2>/dev/null | $PYTHON -c "import sys,json; d=json.load(sys.stdin); assert d.get('success')==True; print(f'  PASS: Got {d.get(\"results_count\",0)} results')"; then
  :
else
  echo "  FAIL: Amazon query failed"
  exit 2
fi

# Error path tests
echo "[E1] Testing missing input error..."
if $PYTHON "$PANGOLIN" --mode amazon 2>/dev/null; then
  echo "  FAIL: Should have errored on missing input"
  exit 10
else
  echo "  PASS: Correctly rejected missing input"
fi

echo "[E2] Testing invalid --pages..."
if $PYTHON "$PANGOLIN" --q "test" --mode review --pages 0 2>/dev/null; then
  echo "  FAIL: Should have errored on --pages 0"
  exit 11
else
  echo "  PASS: Correctly rejected --pages 0"
fi

echo "=== All tests passed ==="


Security scan verdicts — ClawScan: suspicious | static analysis: clean | VirusTotal: clean | SkillSpector: suspicious (HIGH)