The easiest way to scrape eBay products
Tags: scraping, tutorial
Published: 2021-11-01 09:58:11

## Introduction 
‎
### This article will describe the easiest way to scrape eBay products with  [Page2API](https://www.page2api.com/) 

---


**eBay** is an online shopping site that's best known for its auctions and consumer-to-consumer sales.


You can scrape eBay if you intend to perform tasks such as:


- analyzing products
- predicting market trends
- price monitoring



Compared to **Amazon**, scraping **eBay** can be a little more challenging, but the product page doesn't require a real browser, which leads to faster scraping speed.


For scraping **eBay**, we will use **Page2API** - a powerful and delightful API that makes web scraping easy and fun.


We will scrape 2 types of eBay pages:

- Products list page
- Product page    

‎  
## Prerequisites
‎
To perform this simple task, you will need the following things:


-  [A Page2API account](https://www.page2api.com/users/sign_up/)    

The free trial offers a credit that covers up to **1000** web pages to scrape.

- A product or a category of products that we are about to scrape.   

In our case, we will search for **'fitbit 4 silicone strap'**
and then scrape the product page for a random product.    
‎
## Scraping eBay Products List Page  
‎
First what we need is to type **'fitbit 4 silicone strap'** into the search input from eBay's search page change the view type to **Gallery View**.

This will change the browser URL to something similar to:
  
```
https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc
``` 

The URL is the first parameter we need to perform the scraping.

The page that you see must look like the following one:

![eBay results page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8i3lwkap7dz7yju8x7qg.png)   
‎
If you inspect the page HTML, you will find out that a single result is wrapped into a div that looks like the following:


![eBay result component](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a7q18jr3aak229fkm41w.png)
‎

The HTML for a single result element will look like this:


![eBay result HTML](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sbuxgvgp7c2jolaprr1y.png)
‎
The last part is the pagination handling.

In our case, we must click on the **Next (→)** button while the list item's class will be active:


![Next page available](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2p8rolwg2csvkobewsqe.png)
‎


And stop our scraping request when the **Next (→)** button became disabled.

In our case, a new attribute (**aria-disabled="true"**) is assigned to the Next (→) button:


![Next page disabled](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eo02adq59ucwxwla7hb4.png)
‎

Now it's time to prepare the request that will scrape all products that the search page returned.
   


**Setting the api_key as an environment variable**
```bash
export API_KEY=YOUR_PAGE2API_KEY
```     
  

**Running the scraping request with cURL**
```json
    curl -v -XPOST -H "Content-type: application/json" -d '{
      "api_key": "'"$API_KEY"'",
      "url": "https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc",
      "real_browser": true,
      "merge_loops": true,
      "scenario": [
        {
          "loop" : [
            { "wait_for": ".pagination__next" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\"a.pagination__next\").click()" }
          ],
          "stop_condition": "document.querySelector(\"a.pagination__next\").ariaDisabled == \"true\""
        }
      ],
      "parse": {
        "items": [
          {
            "_parent":"ul.srp-grid li.s-item",
            "title":"h3.s-item__title >> text",
            "link":"a.s-item__link >> href",
            "price":".s-item__price >> text",
            "shipping":".s-item__shipping >> text"
          }
        ]
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool
  ```

**The result**
  ```json
    {
      "result": {
        "items": [
          {
            "title": "For Fitbit Charge 3/4 Replacement Silicone Wristband Straps Sports Watch Band",
            "link": "https://www.ebay.com/itm/174485572199?hash=item28a0268a67:g:1QMAAOSwLA9fjTwQ",
            "price": "$3.89",
            "shipping": "Free shipping"
          },
          {
            "title": "Unisex Silicone Sports Bracelet Strap Fashion Band for Fitbit Inspire/Inspire HR",
            "link": "https://www.ebay.com/itm/324593039446?hash=item4b9340b856:g:5jsAAOSwRLJgh62j",
            "price": "$3.49",
            "shipping": "Free shipping"
          },
          ...
        ]
      },
      ...
    }
```    
‎
## Scraping eBay Product Page    
‎
From the products list page, we need to click on any product.


The browser URL will look like this:

```
 https://www.ebay.com/itm/114503614713
```

The URL is the first parameter we need to perform the product data scraping.

Let's prepare the request that will scrape the needed information from the page.


**Setting the api_key as an environment variable (if needed)**
```bash
export API_KEY=YOUR_PAGE2API_KEY
```

**Running the scraping request with cURL**
  ```json
    curl -v -XPOST -H "Content-type: application/json" -d '{
      "api_key": "'"$API_KEY"'",
      "url": "https://www.ebay.com/itm/114503614713",
      "parse": {
        "name": "h1 >> text",
        "price": "span[itemprop=price] >> text",
        "condition": "div[itemprop=itemCondition] >> text",
        "categories": [
          "#vi-VR-brumb-lnkLst span >> text"
        ],
        "amount_sold": ".soldwithfeedback a >> text",
        "seller_name": "#RightSummaryPanel span.mbg-nw >> text",
        "item_location": "span[itemprop=availableAtOrFrom] >> text",
        "feedback_score": "#RightSummaryPanel span.mbg-l a >> text",
        "approximate_price": "#convbidPrice >> text"
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool
  ```


**The result**
```json 
    {
      "result": {
        "name": "Details about Fitbit Charge 2 3 4 Bracelet Stainless Steel Spare Band Nylon Milanese Sport",
        "price": "GBP 5.66",
        "condition": "New with tags",
        "categories": [
          "Jewelry & Watches",
          "Watches, Parts & Accessories",
          "Watch Accessories",
          "Wristwatch Bands"
        ],
        "amount_sold": "",
        "seller_name": "fro-shop",
        "item_location": "Mülheim, Germany",
        "feedback_score": "28166",
        "approximate_price": "US $7.76(including shipping)"
      },
      ...
    }
  
```   
‎
## Conclusion
‎
That's it!

Scraping eBay product data is a bit tricky because of the HTML structure, but that is not a problem if you have the proper scraping tool, such as Page2API that makes web scraping something you can enjoy. 

The original article can be found here:      
[https://www.page2api.com/blog/scrape-ebay-products/](https://www.page2api.com/blog/scrape-ebay-products/)