How to Extract Bing Images Data with SerpApi and Python
Tags: webscraping, python, tutorial, programming
Published: 2023-04-12 19:12:30

- <a href="#intro">Intro</a>
- <a href="#what_will_be_scraped">What will be scraped</a>
- <a href="#why_use_api">Why using API?</a>
- <a href="#fullcode">Full Code</a>
- <a href="#preparation">Preparation</a>
- <a href="#code_explanation">Code Explanation</a>
- <a href="#output">Output</a>
- <a href="#links">Links</a>


<h2 id="intro">Intro</h2>

In this blog post, we'll go through the process of extracting Bing Images using the [Bing Images API](https://serpapi.com/bing-images-api) and the Python programming language. You can [look at the complete code in the online IDE (Replit)](https://replit.com/@chukhraiartur/blog-bing-images-api#main.py).

<h2 id="what_will_be_scraped">What will be scraped</h2>

![wwbs-bing-images](https://user-images.githubusercontent.com/81998012/227189835-3c7d62b0-8299-42e0-9d30-3693b3df3df0.png)


<h2 id="why_use_api">Why using API?</h2>

There're a couple of reasons that may use API, ours in particular:

- No need to create a parser from scratch and maintain it.
- Bypass blocks from Google: solve CAPTCHA or solve IP blocks.
- Pay for proxies, and CAPTCHA solvers.
- Figure out the legal part of scraping data.

SerpApi handles everything on the backend with fast response times under ~1.7 seconds per request and without browser automation, which becomes much faster. Response times and status rates are shown under [SerpApi Status page](https://serpapi.com/status).


<h2 id="fullcode">Full Code</h2>

This code retrieves all the data with pagination:

```python
from serpapi import BingSearch
import json

params = {
    'api_key': '...',               # https://serpapi.com/manage-api-key
    'q': 'Coffee',                  # search query
    'engine': 'bing_images',        # search engine
    'cc': 'US',                     # country of the search
    'first': 1,                     # pagination
    'count': 50,                    # number of results per page
    # 'imagesize': 'wallpaper',     # filtering by size
    # 'color2': 'color',            # filtering by color
    # 'photo': 'photo',             # filtering by image type
    # 'aspect': 'wide',             # filtering by layout
    # 'face': 'portrait',           # filtering by people
    # 'age': 'lt525600',            # filtering by date
    # 'license': 'Type-Any'         # filtering by license
}

search = BingSearch(params)         # data extraction on the SerpApi backend
results = search.get_dict()         # JSON -> Python dict

bing_images_results = {
    'images_results': [],
    'suggested_searches': [],
    'refined_searches': results.get('refined_searches', []),
    'related_searches': results.get('related_searches', []),
    'shopping_results': results.get('shopping_results', [])
}

page_count = 0
page_limit = 10

while 'error' not in results and page_count < page_limit:
    bing_images_results['images_results'].extend(results.get('images_results', []))
    bing_images_results['suggested_searches'].extend(results.get('suggested_searches', []))

    params['first'] += params['count']
    page_count += 1
    results = search.get_dict()

print(json.dumps(bing_images_results, indent=2, ensure_ascii=False))
```


<h2 id="preparation">Preparation</h2>

Install library:

```lang-none
pip install google-search-results
```

[`google-search-results`](https://github.com/serpapi/google-search-results-python) is a SerpApi API package.


<h2 id="code_explanation">Code Explanation</h2>

Import libraries:

```python
from serpapi import BingSearch
import json
```

| Library       | Purpose                                                            |
|---------------|--------------------------------------------------------------------|
| `BingSearch` | to scrape and parse Bing results using [SerpApi](https://serpapi.com/) web scraping library. |
| [`json`](https://docs.python.org/3/library/json.html)| to convert extracted data to a JSON object. |


The parameters are defined for generating the URL. If you want to pass other parameters to the URL, you can do so using the `params` dictionary:

```python
params = {
    'api_key': '...',               # https://serpapi.com/manage-api-key
    'q': 'Coffee',                  # search query
    'engine': 'bing_images',        # search engine
    'cc': 'US',                     # country of the search
    'first': 1,                     # pagination
    'count': 50,                    # number of results per page
    # 'imagesize': 'wallpaper',     # filtering by size
    # 'color2': 'color',            # filtering by color
    # 'photo': 'photo',             # filtering by image type
    # 'aspect': 'wide',             # filtering by layout
    # 'face': 'portrait',           # filtering by people
    # 'age': 'lt525600',            # filtering by date
    # 'license': 'Type-Any'         # filtering by license
}
```

|Parameters|Explanation|
|----------|-----------|
|[`api_key`](https://serpapi.com/bing-images-api#api-parameters-serpapi-parameters-api-key)| Parameter defines the SerpApi private key to use.|
|[`q`](https://serpapi.com/bing-images-api#api-parameters-search-query-q)| Parameter defines the search query. You can use anything that you would use in a regular Bing Images search.|
|[`engine`](https://serpapi.com/bing-images-api#api-parameters-serpapi-parameters-engine)| Set parameter to `bing_images` to use the Bing Images API engine.|
|[`cc`](https://serpapi.com/bing-images-api#api-parameters-localization-cc)| Parameter defines the country to search from. It follows the 2-character [ISO_3166-1](https://en.wikipedia.org/wiki/ISO_3166-1) format. (e.g., `us` for United States, `de` for Germany, `gb` for United Kingdom, etc.).|
|[`first`](https://serpapi.com/bing-images-api#api-parameters-pagination-first)| Parameter controls the offset of the organic results. This parameter defaults to `1`. (e.g., `first=10` will move the 10th organic result to the first position).|
|[`count`](https://serpapi.com/bing-images-api#api-parameters-pagination-count)| Parameter controls the number of results per page. This parameter is only a suggestion and might not reflect the returned results.|
|[`imagesize`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-imagesize)| Parameter is used for filtering images by size. It can be set to: `small`, `medium`, `large`, `wallpaper`.|
|[`color2`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-color2)| Parameter is used for filtering images by color. It can be set to: `color` - Color Only, `bw` - Black & white, `FGcls_GREEN` - Green, etc.|
|[`photo`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-photo)| Parameter is used for filtering images by image type. It can be set to: `photo`, `clipart`, `linedrawing`, `animatedgif`, `transparent`.|
|[`aspect`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-aspect)| Parameter is used for filtering images by layout. It can be set to: `square`, `wide`, `tall`.|
|[`face`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-face)| Parameter is used for filtering images by people. It can be set to: `face` - Faces Only, `portrait` - Head & Shoulders.|
|[`age`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-age)| Parameter is used for filtering images by date. It can be set to: `lt1440` - Past 24 hours, `lt10080` - Past week, `lt43200` - Past month, `lt525600` - Past year.|
|[`license`](https://serpapi.com/bing-images-api#api-parameters-advanced-bing-images-parameters-license)| Parameter is used for filtering images by license. It can be set to: `Type-Any` - All Creative Commons, `L1` - Public Domain, `L2_L3_L4_L5_L6_L7` - Free to share and use, `L2_L3_L4` - Free to share and use commercially, `L2_L3_L5_L6` - Free to modify, share and use, `L2_L3` - Free to modify, share, and use commercially.|

📌Note: You can also add other [API Parameters](https://serpapi.com/bing-images-api#api-parameters).

Then, we create a `search` object where the data is retrieved from the SerpApi backend. In the `results` dictionary we get data from JSON:

```python
search = BingSearch(params)         # data extraction on the SerpApi backend
results = search.get_dict()         # JSON -> Python dict
```

At the moment, the `results` dictionary only stores data from 1 page. Before extracting data, the `bing_images_results` dictionary is created where this data will be added later. Since some of the data is only displayed on the first page, you can extract them immediately:

```python
bing_images_results = {
    'images_results': [],
    'suggested_searches': [],
    'refined_searches': results.get('refined_searches', []),
    'related_searches': results.get('related_searches', []),
    'shopping_results': results.get('shopping_results', [])
}
```

The `page_limit` variable defines the page limit. If you want to extract data from a different number of pages, then simply write the required number into this variable.

```python
page_limit = 10
```

To get all results, you need to apply pagination. This is achieved by the following check: while there is no `error` in the `results` and the current `page_count` value is less than the specified `page_limit` value, we extract the data, increase the `first` parameter by the value of the `count` parameter to get the results from next page and update the `results` object with the new page data:

```python
page_count = 0

while 'error' not in results and page_count < page_limit:
    # data extraction from current page will be here

    params['first'] += params['count']
    page_count += 1
    results = search.get_dict()
```

Lists by corresponding keys are extended with new data from each page:

```python
bing_images_results['images_results'].extend(results.get('images_results', []))
bing_images_results['suggested_searches'].extend(results.get('suggested_searches', []))
# thumbnail= results['images_results'][0]['thumbnail']
# link = results['images_results'][0]['link']
# title = results['images_results'][0]['title']
```

📌Note: In the comments above, I showed how to extract specific fields. You may have noticed the `results['images_results'][0]`. This is the index of a organic result, which means that we are extracting data from the first organic result. The `results['images_results'][1]` is from the second organic result and so on.

After the all data is retrieved, it is output in JSON format:

```python
print(json.dumps(bing_images_results, indent=2, ensure_ascii=False))
```


<h2 id="output">Output</h2>

```json
{
  "images_results": [
    {
      "thumbnail": "https://th.bing.com/th/id/OIP.-ACVKbQLg-aRb2bVxy1jfAHaE8?w=237&h=180&c=7&r=0&o=5&pid=1.7",
      "link": "https://www.bing.com/images/search?view=detailV2&ccid=%2bACVKbQL&id=16A3CD2F2C1C0D71AE4EA5B08475E79A9A2D543C&thid=OIP.-ACVKbQLg-aRb2bVxy1jfAHaE8&mediaurl=https%3a%2f%2fimages.freeimages.com%2fimages%2flarge-previews%2fbc0%2fcoffee-1317648.jpg&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.f8009529b40b83e6916f66d5c72d637c%3frik%3dPFQtmprndYSwpQ%26pid%3dImgRaw%26r%3d0&exph=1066&expw=1599&q=Coffee&simid=608042879963195855&FORM=IRPRST&ck=733C3F62E8891EC1D9ECD99852D9CB5E&selectedIndex=0",
      "title": "coffee Free Photo Download | FreeImages",
      "size": "1599 x 1066 · jpeg",
      "source": "https://www.freeimages.com/photo/coffee-1317648",
      "domain": "freeimages.com",
      "original": "https://images.freeimages.com/images/large-previews/bc0/coffee-1317648.jpg",
      "description": "coffee freeimages",
      "position": 1
    },
    {
      "thumbnail": "https://th.bing.com/th/id/OIP.ceGbTJIrj6ajM2XlsyFXAwAAAA?w=238&h=180&c=7&r=0&o=5&pid=1.7",
      "link": "https://www.bing.com/images/search?view=detailV2&ccid=ceGbTJIr&id=A7A5C7AA478FAAAE30E29D2927035138A7ED4F86&thid=OIP.ceGbTJIrj6ajM2XlsyFXAwAAAA&mediaurl=https%3a%2f%2fwww.indigofinance.com.au%2fwp-content%2fuploads%2f2018%2f07%2fistock-157528129.jpg&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.71e19b4c922b8fa6a33365e5b3215703%3frik%3dhk%252ftpzhRAycpnQ%26pid%3dImgRaw%26r%3d0&exph=315&expw=474&q=Coffee&simid=608037970811556740&FORM=IRPRST&ck=03F56E55E9A229798D3F9E0E47710CF6&selectedIndex=1",
      "title": "Coffee Addiction! Can it be tamed? - Indigo Finance",
      "size": "474 x 315 · jpeg",
      "source": "https://www.indigofinance.com.au/coffee-addiction-can-tamed/",
      "domain": "indigofinance.com.au",
      "original": "https://www.indigofinance.com.au/wp-content/uploads/2018/07/istock-157528129.jpg",
      "description": "coffee addiction",
      "position": 2
    },
    {
      "thumbnail": "https://serpapi.com/searches/641c636f47ccf1fac0af6454/images/d8b77d8a83f199ee4f68a702661ae07622e8f07644945d882e253318c4e63e4d.gif",
      "link": "https://www.bing.com/images/search?view=detailV2&ccid=%2fO2T2GEH&id=CDAE3AA25A9EAAC5AAFCFEAE226C4E400B130445&thid=OIP._O2T2GEHLIPfkjeW55c4fAHaEo&mediaurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.fced93d861072c83df923796e797387c%3frik%3dRQQTC0BObCKu%252fg%26riu%3dhttp%253a%252f%252fi.slimg.com%252fsc%252fsl%252fphoto%252ff%252ffo%252ffood-coffeeandcoffeebeans-dd.jpg%26ehk%3dd3WNdg6Q8xrjnwbPLfOtvTsmTGRas1ZKp81pa7PKQqw%253d%26risl%3d%26pid%3dImgRaw%26r%3d0&exph=382&expw=610&q=Coffee&simid=607997220165786271&FORM=IRPRST&ck=03A478D663783F2B09856F5B2AAAE68B&selectedIndex=2",
      "title": "America's Best Coffee Shops - SmarterTravel.com",
      "size": "610 x 382 · jpeg",
      "source": "http://www.smartertravel.com/blogs/today-in-travel/america-best-coffee-shops.html?id=10909975",
      "domain": "smartertravel.com",
      "original": "http://i.slimg.com/sc/sl/photo/f/fo/food-coffeeandcoffeebeans-dd.jpg",
      "description": "coffee shops america cafe coffe food cup foods wine facts espresso coffees tweet caffeine un good healthy give con flavors",
      "position": 3
    },
    ... other images results
  ],
  "suggested_searches": [
    {
      "thumbnail": "https://th.bing.com/th?q=Coffee+Maker&w=42&h=42&c=7&rs=1&p=0&o=5&pid=1.7&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1",
      "link": "https://www.bing.com/images/search?q=Coffee+Maker&FORM=RESTAB",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Coffee+Maker",
      "name": "Coffee Maker"
    },
    {
      "thumbnail": "https://th.bing.com/th?q=Best+Coffee+Beans&w=42&h=42&c=7&rs=1&p=0&o=5&pid=1.7&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1",
      "link": "https://www.bing.com/images/search?q=Best+Coffee+Beans&FORM=RESTAB",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Best+Coffee+Beans",
      "name": "Best Coffee Beans"
    },
    {
      "thumbnail": "https://th.bing.com/th?q=Coffee+Flavors&w=42&h=42&c=7&rs=1&p=0&o=5&pid=1.7&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1",
      "link": "https://www.bing.com/images/search?q=Coffee+Flavors&FORM=RESTAB",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Coffee+Flavors",
      "name": "Coffee Flavors"
    },
    ... other suggested searches
  ],
  "refined_searches": [
    {
      "thumbnail": "https://th.bing.com/th?q=Coffee+Cart&w=120&h=120&c=1&rs=1&qlt=90&cb=1&pid=InlineBlock&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1&mw=247",
      "link": "https://www.bing.com/images/search?q=Coffee+Cart&qft=&FORM=IRTRRL",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Coffee+Cart",
      "name": "Cart"
    },
    {
      "thumbnail": "https://th.bing.com/th?q=Healthy+Coffee&w=120&h=120&c=1&rs=1&qlt=90&cb=1&pid=InlineBlock&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1&mw=247",
      "link": "https://www.bing.com/images/search?q=Healthy+Coffee&qft=&FORM=IRTRRL",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Healthy+Coffee",
      "name": "Healthy"
    },
    {
      "thumbnail": "https://th.bing.com/th?q=Coffee+Packaging&w=120&h=120&c=1&rs=1&qlt=90&cb=1&pid=InlineBlock&mkt=en-US&cc=US&setlang=en&adlt=moderate&t=1&mw=247",
      "link": "https://www.bing.com/images/search?q=Coffee+Packaging&qft=&FORM=IRTRRL",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Coffee+Packaging",
      "name": "Packaging"
    },
    ... other refined searches
  ],
  "related_searches": [
    {
      "link": "https://www.bing.com/images/search?q=Teeccino+Coffee&qft=&fsm=1&FORM=SHOPSO",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Teeccino+Coffee",
      "name": "Teeccino Coffee"
    },
    {
      "link": "https://www.bing.com/images/search?q=Espresso+Coffees&qft=&fsm=1&FORM=SHOPSO",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Espresso+Coffees",
      "name": "Espresso Coffees"
    },
    {
      "link": "https://www.bing.com/images/search?q=Folgers+Coffee&qft=&fsm=1&FORM=SHOPSO",
      "serpapi_link": "https://serpapi.com/search.json?device=desktop&engine=bing_images&q=Folgers+Coffee",
      "name": "Folgers Coffee"
    }
  ],
  "shopping_results": []
}
```

📌Note: Head to the [playground](https://serpapi.com/playground?engine=bing_images) for a live and interactive demo.


<h2 id="links">Links</h2>

- [Code in the online IDE](https://replit.com/@chukhraiartur/blog-bing-images-api#main.py)
- [Bing Images API](https://serpapi.com/bing-images-api)

<p style="text-align: center;">Join us on <a href="https://twitter.com/serp_api">Twitter</a> | <a href="https://www.youtube.com/channel/UCUgIHlYBOD3yA3yDIRhg_mg">YouTube</a></p>

<p style="text-align: center;">Add a  <a href="https://github.com/serpapi/public-roadmap/issues">Feature Request</a>💫 or a <a href="https://github.com/serpapi/public-roadmap/issues">Bug</a>🐞</p>