Scrape Google Trends with Python (simple API, pytrends alternative)
Tags: webscraping, python
Published: 2024-01-31 03:29:45

We are surrounded by tons of data. But what's the point if we can't understand it all? That's where Google Trends comes in. Google Trends offers valuable insights into what people search for on the internet. Let's dive in and see how we can programmatically scrape Google Trends data using Python (Psst.. no PyTrends needed!).

![Scraping Google Trends data using Python API](https://serpapi.com/blog/content/images/2024/01/scraping-google-trends-1.webp)

Step-by-step scraping Google Trends data with Python
----------------------------------------------------

Without further ado, let's start and collect data from Google Trends.

### Step 1: Tools

We'll use the new official Python library by SerpApi: [serpapi-python](https://github.com/serpapi/serpapi-python) .  
That's the only tool that we need!

![Python library from SerpApi, the tool we need to scrape Google SERP](https://serpapi.com/blog/content/images/2023/10/tools-to-scrape-google-search-in-python.webp)

> As a side note: You can use this library to scrape search results from other search engines, not just Google.

Usually, you'll write your DIY solution using [PyTrends](https://github.com/GeneralMills/pytrends), BeautifulSoup, Selenium, Scrapy, Requests, etc., to scrape Google Trends results. You can relax now since we perform all these heavy tasks for you. **So, you don't need to worry about all the problems you might've encountered while implementing your web scraping solution.**

### Step 2: Setup and preparation

*   Sign up for free at [SerpApi](https://serpapi.com/). You can get 100 free searches per month.
*   Get your SerpApi Api Key from [this page](https://serpapi.com/manage-api-key).
*   Create a new `.env` file, and assign a new env variable with value from API\_KEY above.  
    `SERPAPI_KEY=$YOUR_SERPAPI_KEY_HERE`
*   Install python-dotenv to read the `.env` file with  
    `pip install python-dotenv`
*   Install SerpApi's Python library with  
    `pip install serpapi`
*   Create new `main.py` file for the main program.

Your folder structure will look like this:

```
|_ .env
|_ main.py
```


### Step 3: Write the code for scraping Google Trends

Let's say I want to get this result:  
\- Keyword: 'standing desk'  
\- Region: 'worldwide'  
\- Time: 'Past 12 months'

![sample Google Trends live result](https://serpapi.com/blog/content/images/2024/01/sample-past-12-months.webp)

Here is the Python code

```
import os
import serpapi
from dotenv import load_dotenv

load_dotenv()
api_key = os.getenv('SERPAPI_KEY')

client = serpapi.Client(api_key=api_key)
search =  client.search(
    engine="google_trends",
    q="standing desk",
    api_key=api_key,
  )

print(search)
```


By default, the Google Trends API will set the location for `worldwide` and duration of the `past 12 months` . We'll take a look at how to change these values later. For now, take a look at our perfect result:

![Google Trends API result on search inspector](https://serpapi.com/blog/content/images/2024/01/json-data-representation.webp)

> Please note, that we only return the response in JSON format. You'll need to build the graph yourself if you need one.

### Step 4: Adjust region and timeline

Let's change the settings. I set the region to `United States` and timeline to `Past 30 days` .

![sample Google trends for the past 30 days](https://serpapi.com/blog/content/images/2024/01/sampe-US-data-for-30-days.webp)

Let's see how it is represented in our code.

```
search =  client.search(
    engine="google_trends",
    q="standing desk",
    api_key=api_key,
    geo="US", // Two-code-letter country code
    date="today 1-m" // Timeline for last 1 month
)
```


The result is perfect as well:

![Google trends API result](https://serpapi.com/blog/content/images/2024/01/JSON-result-for-Trends-API.webp)

More information about available parameters: https://serpapi.com/google-trends-api

SerpApi Google Trends API features
----------------------------------

Using SerpApi, it's possible to grab other parts from Google Trends, like the breakdown by region result, interest by region, related queries, related topics, and what's trending now.

**Searching for multiple terms**  
It's possible to search for multiple terms. You can separate the terms by a comma. Here are the limitations:  
\- Max 5 terms per search  
\- Max 100 characters per search

```
search =  client.search(
    engine="google_trends",
    q="standing desk, ergonomic chair",
    api_key=api_key,
)

```


**Interest by Region**  
We can grab the data for interest by region. We need to change the `data_type` to `GEO_MAP_0`

```
search = client.search(
    engine="google_trends",
    q="coffee latte",
    date="today 12-m",
    tz="420",
    data_type="GEO_MAP_0", // Important part
    api_key=api_key,
)
```


Here is the list of possible values for `data_type`

![data types possible values](https://serpapi.com/blog/content/images/2024/01/CleanShot-2024-01-31-at-10.37.35.png)

For example, if you're interested in the related topics, you need to adjust your code to:

```
search = client.search(
    engine="google_trends",
    q="coffee latte",
    date="today 12-m",
    tz="420",
    data_type="RELATED_TOPICS", // Adjust this accordingly
    api_key=api_key,
)

print(search)
```


**References to our other related API:**

*  [Trends Autocomplete API](https://serpapi.com/google-trends-autocomplete)
*   [Trends Breakdown by region API](https://serpapi.com/google-trends-compared-breakdown)
*   [Trends by Region API](https://serpapi.com/google-trends-interest-by-region)
*   [Interest overtime API](https://serpapi.com/google-trends-interest-over-time)
*   [Trends related queries API](https://serpapi.com/google-trends-related-queries)
*   [Trends related topics API](https://serpapi.com/google-trends-related-topics)
*   [Trending Now API](https://serpapi.com/google-trends-trending-now)



Pytrends Tutorial to scrape Google Trends data
----------------------------------------------

There is also another unofficial API for Google Trends for Python. It's called Pytrends. We'll see a basic tutorial on how to implement this.

> Please note that the repository has not been very active these last few months. There are also some unadressed issues. Therefore, we recommend using [Google Trends API](https://serpapi.com/google-trends-api) instead.

Install Pytrends

```
pip install pytrends
```


Initialize the object. We also initialize [`pandas`](https://pandas.pydata.org/) so it's easier to read the data.

```
from pytrends.request import TrendReq
import pandas as pd

pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])
```


Inside the `build_payload` method, we can put the keywords list in an array.

**Interest Over Time**

```
# Interest over time
pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])
df = pytrend.interest_over_time()
print(df)
```


Here is the result

![Pytrends tutorial result](https://serpapi.com/blog/content/images/2024/01/CleanShot-2024-01-31-at-10.06.14@2x.png)

There are some other parameters for `build_payload` method. Unfortunately, I sometimes get error results. Here are the parameters you can play with: `year_start, month_start, day_start, hour_start, year_end, month_end, day_end, hour_end.`

**Interest by Region**  
Here is how to split the trends by country and sort the top 10 results.

```
pytrend = TrendReq()
pytrend.build_payload(kw_list=['Standing Desk'])
df = pytrend.interest_by_region(resolution='COUNTRY')
print(df.sort_values(by='Standing Desk', ascending=False).head(10))
```


**Related topics**  
Here is how to show related topics using Pytrends:

```
df = pytrend.related_topics()
print(df)
```


**Related queries**  
Here is how to show related topics using Pytrends:

```
df = pytrend.related_queries()
print(df)
```


**Trending searches**  
Here is how to display trending searches by Region from Google Trends

```
#Trending Searches
df = pytrend.trending_searches(pn='united_states') # trending searches in real time for United States
print(df.head)
```


The following API methods are available for PyTrends:

*   Interest Over Time: returns historical, indexed data for when the keyword was searched most as shown on Google Trends' Interest Over Time section.
*   Multirange Interest Over Time: returns historical, indexed data similar to interest over time but across multiple time date ranges.
*   Historical Hourly Interest: returns historical, indexed, hourly data for when the keyword was searched most, as shown on Google Trends' Interest Over Time section. It sends multiple requests to Google, each retrieving one week of hourly data. It seems like this would be the only way to get historical, hourly data.
*   Interest by Region: This returns data for where the keyword is most searched, as shown in Google Trends' Interest by Region section.
*   Related Topics: returns data for the related keywords to a provided keyword shown on Google Trends' Related Topics section.
*   Related Queries: returns data for the related keywords to a provided keyword shown on Google Trends' Related Queries section.
*   Trending Searches: returns data for the latest trending searches shown on Google Trends' Trending Searches section.
*   Top Charts: returns the data for a given topic shown in Google Trends' Top Charts section.
*   Suggestions: Return a list of additional suggested keywords that can be used to refine a trend search.

* * *

_Thank you for reading this blog post! I hope it can help you to gather data from the Google Trends site._

_We have a Google Trends playground where you can play with our API:_ [https://serpapi.com/playground?engine=google\_trends](https://serpapi.com/playground?engine=google_trends)

See you in the next post!