How to do an SEO Competitor Analysis (Presidential Elections)
Tags: python, serp, seo, analysis
Published: 2024-04-01 06:00:12


Conducting an SEO competitor analysis can provide valuable insights into your performance in search compared to your competitors. This process also uncovers the sources of traffic for your competitors. Furthermore, it helps you determine your SEO priorities and strategy if it's not too late in some cases.

## Final result of SERP Heatmap - Presidential Elections in Slovakia 2024
This will be the result of your effort while playing around with a few lines of Python code.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/44edr0lcqq9v95ofd9z0.png)

## Let's start

In this article, I'll explore how you can leverage the Python library [Advertools](https://pypi.org/project/advertools/) (from [Elias Dabbas](https://www.linkedin.com/in/eliasdabbas/)) to enhance your competitive analysis visualization capabilities. Additionally, you'll be rewarded by an interactive SERP heatmap demonstrating coverage, appearance and average ranking for competitors you would like to observe. Let's start to set the environment.

## How to install Jupyter Lab
For this tutorial, I focus on setting up an environment in `OSX`. In this case, I choose Jupyter Lab environment and working space. You can skip this if you are familiar with [Google Colab](https://colab.research.google.com/) and head right to the coding section.

Let's start with a simple installation in your `Terminal`. Now just head to `Applications > Utilities` and look for [Terminal.app](https://support.apple.com/guide/terminal/welcome/mac). Open the `Terminal` and add this simple command to start the installation.

```
brew install jupyterlab
```
If you are using Linux you can use the same command or try a different way to install Jupyter Lab on a platform you prefer as mentioned on the official [installation page](https://jupyter.org/install)

## How to start Jupyter Lab
All needed has been installed onto your system. Now enter this command into `Terminal`. 
```
jupyter lab
```
In your default browser will open a tab with `http://localhost:8888`.

## Let's start writing the code
Head to `File > New > Notebook`. A new tab will open for you. First thing first you need to install libraries to play around with the needed variables. 

**Installing libraries**
These libraries will be installed into `Jupiter Lab`. There is a tiny difference when it comes to standard `Terminal` commands. 
You have to enter before the command `!`(exclamation).

```
!pip install advertools
!pip install adviz
```
In your `Jupiter Lab` environment it will look like this. To run the command you stay in the active field where your code is and hit `Shift + Enter`. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/quxir0s3apjonl8g0c52.png)

To continue to enter the rest of the code you need to hit the `+` icon on the top to add a new empty field for new code syntax.

**Importing packages**
What has been installed needs to be imported. So we'll import `pandas`, `advertools`, `adviz` and `plotly`. Enter into the new field this code.

```
import pandas as pd
import advertools as adv
import adviz
import plotly.express as px
```
**Setting up Custom search engine and API key**
You have to set the Custom search engine ID (in code known as cx) to query your preferred search engine via API then later on with the list of your keywords you would like to see results for. Visit [Programmable search engine homepage](https://programmablesearchengine.google.com/controlpanel/create) for `What to search?` select `○ Search the entire web`, for `Search settings` select `○ Image search` and `○ SafeSearch`. Tick reCAPTCHA and press `Create`.

From the whole script, you have to copy just the part after `cx=`.

```
<script async src="https://cse.google.com/cse.js?cx=YOUR-SEARCH-ENGINE-ID">
</script>
<div class="gcse-search"></div>
```

To have Custom Search API head [here](https://developers.google.com/custom-search/v1/overview#api_key) and press `Get a Key`. The code you'll copy is referred to `key` in the script. What goes to the Jupyter lab field is in final this code.

```
cx = "YOUR-SEARCH-ENGINE-ID"
key = "YOUR-CUSTOM-SEARCH-API-KEY"
```

**Check keywords via API**
In this part, we are going to query API via `Advertools` as `adv` with the help of `serp_goog`. I created the variable `dataz` where results will be stored. You can name it whatever you want. The value `gl=["sk"]` defines what `Google` you want to query. In this case, it is `Google.sk`.

In your final code, you can add more keywords. For code purposes, I choose just a few. In my final code, I entered quite a lot of variations of keywords and presidential candidates' names. Feel free to experiment.

```
dataz = adv.serp_goog(cx=cx,
                    key=key,
                    gl=["sk"],
                    q=["prezidentské voľby preukaz",
                       "prezidetske volby preukaz",
                       "prezidentské voľby voličský preukaz",
                       "prezidentské voľby kandidáti",
                       "voľby 2024",
                       "prezidentské voľby",
                       "prezidentské voľby slovensko",
                       "prezidentské voľby 2024",
                       "prezidentské voľby slovensko 2024",     
                      ])
```
When you run this field with `Shift + Enter` the process of querying will start. You have to wait.

> To know what else you can set up for `serp_goog` enter an empty field of Jupyter Lab `adv.serp_goog?` (with ? question mark)

**Draw SERP Heatmap**
This code is straightforward. It can be more robust and you can modify via `fig` almost all values you will see in the SERP Heatmap, but for this case, I kept it as simple as it can be. With `fig.layout.title` we define `title` of the heatmap. I call variable `dataz` via `serp_heatmap`. Here we define a number of results as `num_domains` (number of domains). That's all that we have to set up to draw the SERP Heatmap.

```
adviz.serp_heatmap(dataz,
                   num_domains=10)
fig = adviz.serp_heatmap(dataz)
fig.layout.title = "SERP Heatmap - Presidential Elections 2024 in Google.sk"
fig
```

## Interactive result of SERP Heatmap - Presidential Elections
As shown at the beginning you should now see the result of the SERP Heatmap in the new field of Jupyter Lab. It's fully interactive, so you can hover over the circles to see what keywords on what position have been triggered for each domain as shown on this [Interactive SERP Heatmap](https://kukucka.work/slovak-presidential-elections-2024/).

**Unstable results of SERP Heatmap**
> Results can vary based on the date you run your analysis because most of the websites produce content and Google algorithms reflect changes and recalculate coverage and position for each domain/website (mostly frequently updated news websites) almost simultaneously. For example, before the first round of the Presidential election in Slovakia, there were lots of results from domains like `Wikipedia`, `Twitter`, `Facebook` or `Instagram` mostly for candidates names.

**How to modify SERP Heatmap results based on top-level domain**
> If you want to narrow your results in the SERP Heatmap just to domains from a specific country in other words restrict search results to documents originating in a particular country, you may use boolean operators in the `cr` parameter's value. The whole code for domains originating just from Slovakia is `cr=["countrySK"]` (place it in the code where you specify keywords). Google Search then determines the country of a document by analyzing the top-level domain (TLD) of the document's URL, and the geographic location of the Web server's IP address.

Thanks to [Steven Lelham](https://unsplash.com/@slelham?utm_source=unsplash&utm_medium=referral&utm_content=CreditCopyTextGratitude) for the cover image from Unsplash.