Scrape Google Autocomplete Suggestions
Published: 2022-08-08 18:51:00


In this post, we will learn to scrape Google Autocomplete Suggestions.



## Requirements:

Before we begin, we should install everything we may need in this tutorial to move forward.

1. [Node JS](https://nodejs.org/en/)
2. [Unirest JS](https://www.npmjs.com/package/unirest)
3. [Cheerio JS](https://www.npmjs.com/package/cheerio)

So before starting, set up your Node JS project and after that,  install both the packages - Unirest JS and Cheerio JS. You can install both packages from the above stated link.


## Target:


![coffee suggestions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0zohmzgmznuc43l56ezh.png)
We will target to scrape the autocomplete suggestions of coffee.


## Process:

Now, all the things are installed that we will need to prepare our scraper. We will use an npm library Unirest JS to make a get request to our target URL so we can get our raw HTML data. Then we will use Cheerio JS for parsing the extracted HTML data.

We will target this URL:

```
https://www.google.com/complete/search?&hl=en&q=coffee&gl=us&client=chrome
```

Copy this URL in your browser and press enter. You will see a text file downloading in your browser by entering this URL in your browser. Open this file in your respective code editor.

![coffee scraped results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rm6hlhl4dh9dkn6f1kp8.png)

We will now first convert this JSON string into object and get the respective data we need.


![autocomplete scraper code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aipevxsgvahx3pgvhyq5.png)

You can copy this code from the following link: [https://github.com/Darshan972/GoogleScrapingBlogs/blob/main/GoogleAutocompleteScraper.js](https://github.com/Darshan972/GoogleScrapingBlogs/blob/main/GoogleAutocompleteScraper.js)


## Result: 


```
[
  { value: 'coffee near me', relevance: 1250, type: 'QUERY' },
  { value: 'coffee shops near me', relevance: 650, type: 'QUERY' },
  { value: 'coffee shop', relevance: 601, type: 'QUERY' },
  { value: 'coffee table', relevance: 600, type: 'QUERY' },
  { value: 'coffee maker', relevance: 553, type: 'QUERY' },
  { value: 'coffee bean', relevance: 552, type: 'QUERY' },
  { value: 'coffee grinder', relevance: 551, type: 'QUERY' },
  { value: 'coffee meets bagel', relevance: 550, type: 'QUERY' }
]
1300
```

## [With Google Autocomplete API](https://docs.serpdog.io/google-search-api/autocomplete-results)

If you want to scrape Google Autocomplete Results easily, without making a scraper, as scraping can take a lot of time sometimes, you can try this API.

[Serpdog | Google Search API](https://serpdog.io) also provides 100 free searches per month, and if you want to scale your requests quota, you can buy paid plans also.

Example request code: 

```
const axios = require('axios');

axios.get('https://api.serpdog.io/autocomplete?api_key=APIKEY&q=football&gl=us')
  .then(response => {
    console.log(response.data);
  })
  .catch(error => {
    console.log(error);
  });
```

Results:

```
{
  "meta": {
    "api_key": "APIKEY",
    "q": "football",
    "gl": "us"
  },
  "suggestions": [
    {
      "value": "football cleats",
      "relevance": 601,
      "type": "QUERY"
    },
    {
      "value": "football games",
      "relevance": 600,
      "type": "QUERY"
    },
    {
      "value": "football wordle",
      "relevance": 555,
      "type": "QUERY"
    },
    {
      "value": "football today",
      "relevance": 554,
      "type": "QUERY"
    },
    {
      "value": "football gloves",
      "relevance": 553,
      "type": "QUERY"
    },
    {
      "value": "football field",
      "relevance": 552,
      "type": "QUERY"
    },
    {
      "value": "football movies",
      "relevance": 551,
      "type": "QUERY"
    },
    {
      "value": "football positions",
      "relevance": 550,
      "type": "QUERY"
    }
  ],
  "verbatim_relevance": 1300
}
```



## Conclusion

In this tutorial we learned to scrape Google Autocomplete Suggestions Results. If you have any questions, feel free to ask me in the comments. Follow me on [Twitter](https://twitter.com/darshan0_1). Thanks for reading!


## Additional Resources

1. [Google Maps Reviews Scraper](https://serpdog.io/blog/scrape-google-maps-reviews.html)
2. [Google News Scraper](https://dev.to/darshan0_1/scrape-google-news-results-1je4)
3. [Google Images Scraper](https://hackernoon.com/scrape-google-images-with-node-js)
4. [Google Organic Search Results Scraper](https://medium.com/@darshankhandelwal12/how-to-scrape-google-organic-search-results-with-node-js-d3abe0274f40)


## Author

I am Darshan, founder of [serpdog.io](https://serpdog.io). I love to scrape websites and write blog about them.