n8n workflow: Vision-Based AI Agent Scraper - with Google Sheets, ScrapingBee, and Gemini

Nodes:
name: When clicking ‘Test workflow’
type: n8n-nodes-base.manualTrigger
name: ScrapingBee- Get page HTML
type: n8n-nodes-base.httpRequest
parameters: url: https://app.scrapingbee.com/api/v1
sendQuery: yes
queryParameters: parameters: name: api_key
value: <your_scrapingbee_apikey>
name: url
value: ={{$json.url}}
name: Structured Output Parser
type: @n8n/n8n-nodes-langchain.outputParserStructured
parameters: jsonSchemaExample: product_title: The title of the product
product_price: The price of the product
product_brand: The brand of the product
promo: true or false
promo_percentage: NUM %
name: Google Gemini Chat Model
type: @n8n/n8n-nodes-langchain.lmChatGoogleGemini
parameters: modelName: models/gemini-1.5-pro-latest
name: Split Out
type: n8n-nodes-base.splitOut
parameters: fieldToSplitOut: output
name: Google Sheets - Get list of URLs
type: n8n-nodes-base.googleSheets
parameters: sheetName: __rl: yes
mode: list
value: gid=0
cachedResultName: List of URLs
documentId: __rl: yes
mode: list
cachedResultName: Google Sheets - Workflow Vision-Based Scraping
authentication: serviceAccount
name: Sticky Note
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 652.6453488372096
content: ## Trigger
The default trigger is **When clicking ‘Test workflow’**, meaning the workflow will **need to be triggered manually**. 

You can replace this by selecting a **trigger of your choice**.

name: Sticky Note1
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 664.2441860465121
content: ## Google Sheets - List of URLs

The Google Sheet will contain two sheets:  
- **List of URLs to** scrape  
- **Results** page, populated with the scraping results and AI-extracted data.

Here is an **[example Google Sheet](https://docs.google.com/spreadsheets/d/10Gc7ooUeTBbOOE6bgdNe5vSKRkkcAamonsFSjFevkOE/)** you can use. The "Results" sheet is pre-configured for e-commerce website scraping. You can adapt it to your specific needs, but remember to adjust the `Structured Output Parser` node accordingly.

name: Sticky Note2
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 667.2965116279074
content: ## Set Fields

This node allows you to **define the fields** that will be sent to the **ScrapingBee HTTP Node** and the AI Agent. 

In this template, **only one field** is pre-configured: **url**. You can customize it by adding additional fields as needed.

name: Set fields
type: n8n-nodes-base.set
parameters: assignments: assignments: name: url
type: string
value: ={{ $json.url }}
name: Sticky Note3
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 671.8750000000002
content: ## ScrapingBee - Get Page Screenshot

This node uses ScrapingBee, a powerful scraping tool, to capture a screenshot of the desired URL.  
You can [try ScrapingBee](https://www.scrapingbee.com/) and enjoy 1,000 free requests (non-affiliate link).  

Ensure the `screenshot_full_page` parameter is set to *`true`* for a full-page screenshot. This is crucial for vision-based scraping with the AI Agent.  

Alternatively, you can **choose to screenshot only a specific part of the page**. However, keep in mind that the **AI Agent will extract data only from the visible section—it has vision**, but not a crystal ball 🔮!

name: ScrapingBee - Get page screenshot
type: n8n-nodes-base.httpRequest
parameters: url: https://app.scrapingbee.com/api/v1
sendQuery: yes
sendHeaders: yes
queryParameters: parameters: name: api_key
value: <your_scrapingbee_apikey>
name: url
value: ={{ $json.url }}
name: screenshot_full_page
value: true
headerParameters: parameters: name: User-Agent
value: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
name: Sticky Note4
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 1000.334302325581
height: 679.5058139534889
content: ## Vision-Based Scraping AI Agent

This is the central node of the workflow, powered by an AI Agent with two key prompts:

- **System Prompt**: Instructs the AI on how and what data to extract from the screenshot. You can customize this to suit your needs. It also includes fallback instructions to call a tool for retrieving the HTML page if data extraction from the screenshot fails.  
- **User Message**: Provides the page URL for context.

### Sub-Nodes

1. **Google Gemini Chat Model**  
   Chosen because tests show that **Gemini-1.5-Pro** outperforms GPT-4 and GPT-4-Vision in visual tasks. *Either my prompt wasn’t optimized for GPT models, or GPT might need glasses 👓*. 
**Other multimodal LLMs haven’t been tested yet**.

2. **HTML-Based Scraping Tool**  
   A **fallback tool** the agent **uses if it cannot extract data directly from the screenshot**.

3. **Structured Output Parser**  
   Formats the **extracted data into an easy-to-use structure**, ready to be added to the **results page in Google Sheets**.
name: HTML-based Scraping Tool
type: @n8n/n8n-nodes-langchain.toolWorkflow
parameters: name: HTMLScrapingTool
workflowId: __rl: yes
mode: list
value: PpFVCrTiYoa35q1m
cachedResultName: vb-scraping
description: =Call this tool ONLY when you need to retrieve the HTML content of a webpage.
responsePropertyName: data
name: Sticky Note5
type: n8n-nodes-base.stickyNote
parameters: color: 6
width: 305.625
height: 337.03488372093034
content: ### Google Gemini Chat Model

The **default model is gemini-1.5-pro**. It offers excellent performance for this use case, but **it’s not the most cost-effective option—use it judiciously**.


name: Sticky Note6
type: n8n-nodes-base.stickyNote
parameters: color: 6
width: 305.625
height: 337.03488372093034
content: ### HTML-Based Scraping Tool

This tool is **invoked when the AI Agent requires the HTML** (*converted to Markdown*) to extract data because the **screenshot alone wasn’t sufficient**.

name: Sticky Note7
type: n8n-nodes-base.stickyNote
parameters: color: 6
width: 305.625
height: 337.03488372093034
content: ### Structured Output Parser

This node **organizes the extracted data into an easy-to-use JSON format**.  

In this template, the JSON is **designed for an e-commerce webpage**. Customize it to fit your specific needs.

name: Sticky Note8
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 671.8750000000002
content: ## Split Out

This node **splits the array** created by the `Structured Output Parser` into **individual rows**, making them easy to append to the **subsequent Google Sheets node**.

name: Sticky Note9
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 671.8750000000002
content: ## Google Sheets - Create Rows

This node **creates rows** in the **Results** sheet using the extracted data.  

You can use the **[example Google Sheet](https://docs.google.com/spreadsheets/d/10Gc7ooUeTBbOOE6bgdNe5vSKRkkcAamonsFSjFevkOE/)** as a template. However, ensure that the **columns in the Results sheet are aligned with the structure of the output** from the `Structured Output Parser node`.

name: Google Sheets - Create Rows
type: n8n-nodes-base.googleSheets
parameters: columns: value: promo: ={{ $json.promo }}
category: ={{ $('Set fields').item.json.url }}
product_url: ={{ $json.product_title }}
product_brand: ={{ $json.product_brand }}
product_price: ={{ $json.product_price }}
promo_percent: ={{ $json.promo_percentage }}
schema: type: string
display: yes
required: no
displayName: category
defaultMatch: no
canBeUsedToMatch: yes
type: string
display: yes
required: no
displayName: product_url
defaultMatch: no
canBeUsedToMatch: yes
type: string
display: yes
required: no
displayName: product_price
defaultMatch: no
canBeUsedToMatch: yes
type: string
display: yes
required: no
displayName: product_brand
defaultMatch: no
canBeUsedToMatch: yes
type: string
display: yes
required: no
displayName: promo
defaultMatch: no
canBeUsedToMatch: yes
type: string
display: yes
required: no
displayName: promo_percent
defaultMatch: no
canBeUsedToMatch: yes
mappingMode: defineBelow
operation: append
sheetName: __rl: yes
mode: list
value: 648398171
cachedResultName: Results
documentId: __rl: yes
mode: list
value: 1g81_39MJUlwnInX30ZuBtHUb-Y80WrYyF5lccaRtcu0
cachedResultName: Google Sheets - Workflow Vision-Based Scraping
authentication: serviceAccount
name: Vision-based Scraping Agent
type: @n8n/n8n-nodes-langchain.agent
parameters: text: =Here is the screenshot you need to use to extract data about the page:

{{ $json.url }}
options: systemMessage: =Extract the following details from the input screenshot:

- Product Titles
- Product Prices
- Brands
- Promotional Information (e.g., if the product is on promo)

Step 1: Image-Based Extraction
Analyze the provided screenshot to identify and extract all the required details: product titles, prices, brands, and promotional information.
Ensure the extraction is thorough and validate the completeness of the information.
Cross-check all products for missing or unclear details.
Highlight any limitations (e.g., text is unclear, partially cropped, or missing) in the extraction process.

Step 2: HTML-Based Extraction (If Needed)
If you determine that any required information is:

Incomplete or missing (e.g., not all titles, prices, or brands could be retrieved).
Ambiguous or uncertain (e.g., unclear text or potential errors in OCR).
Unavailable due to the limitations of image processing (e.g., product links).

Then:

Call the HTML-based tool with the input URL to access the page content.
Extract the required details from the HTML to supplement or replace the image-based results.
Combine data from both sources (if applicable) to ensure the final result is comprehensive and accurate.

Additional Notes
Avoid redundant HTML tool usage—confirm deficiencies in image-based extraction before proceeding.
For products on promotion, explicitly label this status in the output.
Report extraction errors or potential ambiguities (e.g., text illegibility).

In your output, include all these fields as shown in the example below. If there is no promotion, set "promo" to false and "promo_percent" to 0.

json
Copy code
[{
  "product_title": "The title of the product",
  "product_price": "The price of the product",
  "product_brand": "The brand of the product",
  "promo": true,
  "promo_percent": 25
}]
passthroughBinaryImages: yes
promptType: define
hasOutputParser: yes
name: Sticky Note10
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 357.10392441860495
content: ## HTML-Scraping Tool Trigger

This **node serves as the entry point for the HTML scraping tool.  

It is triggered by the **AI Agent only when it fails to extract data** from the screenshot. The **URL** is sent as a **parameter for the query**.
name: HTML-Scraping Tool
type: n8n-nodes-base.executeWorkflowTrigger
name: Sticky Note11
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 357.10392441860495
content: ## Set Fields - From AI Agent Query

This node sets the fields from the AI Agent’s query.  

In this template, the only field configured is **url**.

name: Set fields - from AI agent query
type: n8n-nodes-base.set
parameters: assignments: assignments: name: url
type: string
value: ={{ $json.query }}
name: Sticky Note12
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 357.10392441860495
content: ## ScrapingBee - Get Page HTML

This node utilizes the ScrapingBee API to **retrieve the HTML of the webpage**.

name: Sticky Note13
type: n8n-nodes-base.stickyNote
parameters: color: 7
width: 364.53488372093034
height: 357.10392441860495
content: ## HTML to Markdown

This node **converts the HTML from the previous node** into Markdown format, **helping to save tokens**.  

The converted **Markdown is then automatically sent to the AI Agent** through this node.

name: HTML to Markdown
type: n8n-nodes-base.markdown
parameters: html: ={{ $json.data }}
name: Sticky Note14
type: n8n-nodes-base.stickyNote
parameters: width: 829.9937466197946
height: 646.0101744186061
content: # ✨ Vision-Based AI Agent Scraper - with Google Sheets, ScrapingBee, and Gemini

## Important notes :
### Check legal regulations: 
This workflow involves scraping, so make sure to check the legal regulations around scraping in your country before getting started. Better safe than sorry!

## Workflow description
This workflow leverages a **vision-based AI Agent**, integrated with Google Sheets, ScrapingBee, and the Gemini-1.5-Pro model, to **extract structured data from webpages**. The AI Agent primarily **uses screenshots for data extraction** but switches to HTML scraping when necessary, ensuring high accuracy. 

Key features include:  
- **Google Sheets Integration**: Manage URLs to scrape and store structured results.  
- **ScrapingBee**: Capture full-page screenshots and retrieve HTML data for fallback extraction.  
- **AI-Powered Data Parsing**: Use Gemini-1.5-Pro for vision-based scraping and a Structured Output Parser to format extracted data into JSON.  
- **Token Efficiency**: HTML is converted to Markdown to optimize processing costs.

This template is designed for e-commerce scraping but can be customized for various use cases.  


Connections:
Split Out: main: node: Google Sheets - Create Rows
type: main
Set fields: main: node: ScrapingBee - Get page screenshot
type: main
HTML-Scraping Tool: main: node: Set fields - from AI agent query
type: main
Google Gemini Chat Model: ai_languageModel: node: Vision-based Scraping Agent
type: ai_languageModel
HTML-based Scraping Tool: ai_tool: node: Vision-based Scraping Agent
type: ai_tool
Structured Output Parser: ai_outputParser: node: Vision-based Scraping Agent
type: ai_outputParser
ScrapingBee- Get page HTML: main: node: HTML to Markdown
type: main
Vision-based Scraping Agent: main: node: Split Out
type: main
Google Sheets - Get list of URLs: main: node: Set fields
type: main
Set fields - from AI agent query: main: node: ScrapingBee- Get page HTML
type: main
ScrapingBee - Get page screenshot: main: node: Vision-based Scraping Agent
type: main
When clicking ‘Test workflow’: main: node: Google Sheets - Get list of URLs
type: main