Extract the number of Google search results using beta GPT-3
Tags: openai, gpt3, webscraping
Published: 2021-09-03 11:37:31

---
canonical_url: https://serpapi.com/blog/gpt-3-total-results-google-serp/
description: Using OpenAI to extract numbers from HTML of Google SERP. The next step is to use Codex to generate parser code.
---

Today we received beta access to Codex and GPT-3 models and started playing with automated web scraping.

![image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gki658ioxelzvxu7k1kz.png)

OpenAI has a good [example for prompt design in the documentation for Completion](https://beta.openai.com/docs/guides/completion/improving-the-classifiers-efficiency). Based on the example, we provided samples of HTML and expected results in English, French and German languages.

```
This is an extractor of the number of search results from HTML
 
HTML: "<div id="result-stats">About 3,200,000,000 results<nobr> (0.97 seconds)&nbsp;</nobr></div>"
Number of search results: 3200000000

HTML: "<div id="result-stats">About 2,200,000,000 results<nobr> (0.297 seconds)&nbsp;</nobr></div>"
Number of search results: 2200000000

HTML: "<div id="result-stats">Environ 1 400 000 000&nbsp;résultats<nobr> (1,05&nbsp;secondes)&nbsp;</nobr></div>"
Number of search results: 1 400 000 000

HTML: "<div id="result-stats">About 1,790,000 results<nobr> (0.55 seconds)&nbsp;</nobr></div>"
Number of search results: 5000000

HTML: "<span class="nums_text">百度为您找到相关结果约100,000,000个</span>"
Number of search results: 100000000

HTML text
1. "<div id="result-stats">About 1,800,0020,000 results<nobr> (0.589 seconds)&nbsp;</nobr></div>"
2. "<div id="result-stats">About 953,626,112 results<nobr> (1.29 seconds)&nbsp;</nobr></div>"
3. "<div id="result-stats">Ungefähr 1.240.000.000 Ergebnisse<nobr> (0,72 Sekunden)&nbsp;</nobr></div>"

Extracted number of search results
1. 18000020000
2. 953626112
3. 1240000000
```

Then we provided HTML to extract data and a prompt.

```
HTML text
1. "<div id="result-stats">About 1,800,0020,000 results<nobr> (0.589 seconds)&nbsp;</nobr></div>"
2. "<div id="result-stats">約 1,510,000,000 件<nobr> （0.82 秒）&nbsp;</nobr></div>"
3. "<div id="result-stats">Aproximadamente 2.180.000.000 resultados<nobr> (0,73 segundos)&nbsp;</nobr></div>"
4. "<div id="result-stats">Sekitar 2.480.000.000 hasil<nobr> (0,72 detik)&nbsp;</nobr></div>"
5. "<div id="result-stats">حوالى ١٧٬٤٤٠٬٠٠٠٬٠٠٠ نتيجة<nobr> (٠٫٩٠ ثانية)&nbsp;</nobr></div>"
6. "<div id="result-stats">Yaklaşık 2.680.000.000 sonuç bulundu<nobr> (0,62 saniye)&nbsp;</nobr></div>"
7. "<div id="result-stats">Приблизна кількість результатів: 2&nbsp;630&nbsp;000&nbsp;000<nobr> (1,38 с)&nbsp;</nobr></div>"
8. "<div id="result-stats">Aproximadamente 19.250.000.000 resultados<nobr> (0,73 segundos)&nbsp;</nobr></div>"
9. "<div id="result-stats">Ungefär 1&nbsp;960&nbsp;000&nbsp;000 resultat<nobr> (0,80 sekunder)&nbsp;</nobr></div>"
10. "<div id="result-stats">Περίπου 2.480.000.000 αποτελέσματα<nobr> (0,76 δευτερόλεπτα)&nbsp;</nobr></div>"

Extracted number of search results
1.
```

OpenAI was able to extract data from different languages too: Japanese, Ukrainian, Greek, Turkish, Spanish. Absolutely amazing!

```
Extracted number of search results
1. 18000020000
2. 15100000
3. 21800000
4. 24800000000
5. 196000000
6. 268000000
7. 26300000000
8. 19000000
9. 19600000000
10. 2480000000
```

It has incorrectly extracted Arabic digits (#5) probably because there were no examples.

We used the [Davinci model and default parameters in the OpenAI Playground](https://beta.openai.com/playground/p/57Wk2RJShlb7rpf8PeveLN9L).

Here is a video of data extraction on OpenAI Playground:

{% youtube rONX-Zfuq0Q %}

Next time we will use the Codex model to generate a Ruby or Python program that extracts the number of search results. The end goal is to replace part of hand-crafted parsers with automated data extraction.

### Links
[OpenAI Playground](https://beta.openai.com/playground/p/57Wk2RJShlb7rpf8PeveLN9L?model=davinci) • [Request beta access for OpenAI](https://beta.openai.com/) • [Try SerpApi for free](https://serpapi.com/users/sign_up?plan=free)

### Outro
If you have any questions or an idea on how to properly automatically extract data from SERPs, feel free to drop a comment via Twitter at [@serp_api](https://twitter.com/serp_api).