Web scraping Google Jobs Listing with Nodejs
Tags: webscraping, node, serpapi
Published: 2022-11-17 08:54:26

<h2 id='intro'>Intro</h2>

In this blog post I'll show you how to use [Google Jobs Listing Results API](https://serpapi.com/google-jobs-listing-api) from SerpApi.

The main advantages of our API is that you don't need to use browser automation to scrape results, create the parser from scratch and maintain it.

There's also a chance that the request might be blocked at some point from Google, we handle it on our backend so there's no need to figure out how to do it yourself or figure out which CAPTCHA, proxy provider to use.

<h2 id='what'>What will be scraped</h2>

![what](https://user-images.githubusercontent.com/64033139/199776737-a91ffdbd-a829-47b3-a8c4-0a393a7121ab.png)

<h2 id='full_code'>Full code</h2>

If you don't need an explanation, have a look at [the full code example in the online IDE](https://replit.com/@MikhailZub/Google-Jobs-Listing-with-SerpApi#index.js)

```javascript
const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(process.env.API_KEY);

const jobId =
  "eyJqb2JfdGl0bGUiOiJCYXJpc3RhIiwiY29tcGFueV9uYW1lIjoiQmx1ZSBCb3R0bGUgQ29mZmVlIiwiY29tcGFueV9taWQiOiIvbS8wM3AxMnFkIiwiYWRkcmVzc19jaXR5IjoiU2FuIEZyYW5jaXNjbyIsImFkZHJlc3Nfc3RhdGUiOiJDYWxpZm9ybmlhIiwiaHRpZG9jaWQiOiJBNnFRZGw1VjZvU1ZxbnI0QUFBQUFBPT0iLCJ1dWxlIjoidytDQUlRSUNJa1UyRnVJRVp5WVc1amFYTmpieUJDWVhrZ1FYSmxZU3hWYm1sMFpXUWdVM1JoZEdWeiIsImZjIjoiRW80QkNtY3dZVFZSY0RkWE9VTkJSek42WVdocVVHRkZXV2hqTkdGamJGSm5lRUpmWVhOc2NVRmlhRXhKWHpaek1Hb3hWamx2UWxKSlpGVnBVRUpyV1ZkeU1sQXpXRmhVVDB3MVRHbFNjamh5T0VzelRVcGhhM2x6WjFsQ1NuZFBWMjVJTlVKQlJWTjJOREZCRWhaME5rUXdXSE5pYjB4TmRWVXRaMVEwTkc5SFNVTkJHZ3RQTFVKSGRqaFNVWEp4TkEiLCJmY2VudiI6IjEiLCJmY3YiOiIyIiwiZmNfaWQiOiJmY183In0";

const params = {
  engine: "google_jobs_listing", // search engine
  q: jobId, // job id from https://serpapi.com/playground?engine=google_jobs&q=Barista
};

const getJson = () => {
  return new Promise((resolve) => {
    search.json(params, resolve);
  });
};

const getResults = async () => {
  const json = await getJson();
  const { apply_options = "No apply options for this job", salaries = "No salaries for this job", ratings = "No ratings for this job" } = json;
  return { applyOptions: apply_options, salaries, ratings };
};

getResults().then(console.log);
```

<h2 id='preparation'>Preparation</h2>

First, we need to create a Node.js\* project and add [`npm`](https://www.npmjs.com/) package [`google-search-results-nodejs`](https://www.npmjs.com/package/google-search-results-nodejs) to scrape and parse what you need using [SerpApi](https://serpapi.com/).

To do this, in the directory with our project, open the command line and enter:

```bash
$ npm init -y
```

And then:

```bash
$ npm i google-search-results-nodejs
```

\*<span style="font-size: 15px;">If you don't have Node.js installed, you can [download it from nodejs.org](https://nodejs.org/en/) and follow the installation [documentation](https://nodejs.dev/learn/introduction-to-nodejs).</span>

<h3 id='serp_api_code_explanation'>Code explanation</h3>

First, we need to declare `SerpApi` from [`google-search-results-nodejs`](https://www.npmjs.com/package/google-search-results-nodejs) library and define new `search` instance with your API key from [SerpApi](https://serpapi.com/manage-api-key):

```javascript
const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(API_KEY);
```

Next, we write the job ID and the necessary parameters for making a request. You can get the job Id parameter using [Web scraping Google Jobs organic results with Nodejs](https://serpapi.com/blog/web-scraping-google-jobs-organic-results-with-nodejs/#serp_api) blog post:

```javascript
const jobId =
  "eyJqb2JfdGl0bGUiOiJCYXJpc3RhIiwiY29tcGFueV9uYW1lIjoiQmx1ZSBCb3R0bGUgQ29mZmVlIiwiY29tcGFueV9taWQiOiIvbS8wM3AxMnFkIiwiYWRkcmVzc19jaXR5IjoiU2FuIEZyYW5jaXNjbyIsImFkZHJlc3Nfc3RhdGUiOiJDYWxpZm9ybmlhIiwiaHRpZG9jaWQiOiJBNnFRZGw1VjZvU1ZxbnI0QUFBQUFBPT0iLCJ1dWxlIjoidytDQUlRSUNJa1UyRnVJRVp5WVc1amFYTmpieUJDWVhrZ1FYSmxZU3hWYm1sMFpXUWdVM1JoZEdWeiIsImZjIjoiRW80QkNtY3dZVFZSY0RkWE9VTkJSek42WVdocVVHRkZXV2hqTkdGamJGSm5lRUpmWVhOc2NVRmlhRXhKWHpaek1Hb3hWamx2UWxKSlpGVnBVRUpyV1ZkeU1sQXpXRmhVVDB3MVRHbFNjamh5T0VzelRVcGhhM2x6WjFsQ1NuZFBWMjVJTlVKQlJWTjJOREZCRWhaME5rUXdXSE5pYjB4TmRWVXRaMVEwTkc5SFNVTkJHZ3RQTFVKSGRqaFNVWEp4TkEiLCJmY2VudiI6IjEiLCJmY3YiOiIyIiwiZmNfaWQiOiJmY183In0";

const params = {
  engine: "google_jobs_listing", // search engine
  q: jobId, // job id from https://serpapi.com/playground?engine=google_jobs&q=Barista
};
```

Next, we wrap the search method from the SerpApi library in a promise to further work with the search results:

```javascript
const getJson = () => {
  return new Promise((resolve) => {
    search.json(params, resolve);
  });
};
```

And finally, we declare the function `getResult` that gets data from the page and return it:

```javascript
const getResults = async () => {
  ...
};
```

In this function, we get `json` with results from the page (`getJson` function). Then we [destructure](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment) received `json`, set [default values](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment#default_value) to each destructured property and return object with results:

```javascript
const json = await getJson();
const { 
    apply_options = "No apply options for this job",
    salaries = "No salaries for this job",
    ratings = "No ratings for this job"
    } = json;
return { applyOptions: apply_options, salaries, ratings };
```

After, we run the `getResults` function and print all the received information in the console:

```javascript
getResults().then(console.log);
```

<h2 id='serp_api_output'>Output</h2>

```json
{
  "applyOptions": "No apply options for this job",
  "salaries": [
    {
      "job_title": "Barista",
      "link": "https://www.ziprecruiter.com/Salaries/Barista-Salary-in-San-Francisco,CA?utm_campaign=google_jobs_salary&utm_source=google_jobs_salary&utm_medium=organic",
      "source": "ZipRecruiter",
      "salary_from": 21000,
      "salary_to": 40000,
      "salary_currency": "$",
      "salary_periodicity": "year",
      "thumbnail": "https://serpapi.com/searches/6363e399a3f4ef79bb0ff373/images/f1d87c26c4968270a8e938f7cf10b5cdc1a6c6457fd1184e.png",
      "based_on": "Based on local employers"
    },
    {
      "job_title": "Coffee Barista",
      "link": "https://www.salary.com/research/salary/alternate/coffee-barista-salary/san-francisco-ca?utm_campaign=google_jobs_salary&utm_source=google_jobs_salary&utm_medium=organic",
      "source": "Salary.com",
      "salary_from": 27000,
      "salary_to": 36000,
      "salary_currency": "$",
      "salary_periodicity": "year",
      "thumbnail": "https://serpapi.com/searches/6363e399a3f4ef79bb0ff373/images/f1d87c26c496827075cfe00b6aeb887d3533b03d6f8540fc.png",
      "based_on": "Based on local employers"
    },
    {
      "job_title": "Barista",
      "link": "https://www.payscale.com/research/US/Job=Barista/Hourly_Rate/ae670208/San-Francisco-CA?utm_campaign=google_jobs_salary&utm_source=google_jobs_salary&utm_medium=organic",
      "source": "Payscale",
      "salary_from": 14,
      "salary_to": 20,
      "salary_currency": "$",
      "salary_periodicity": "hour",
      "thumbnail": "https://serpapi.com/searches/6363e399a3f4ef79bb0ff373/images/f1d87c26c4968270a20489154adfa8fce64f52ba237d9ed1.png",
      "based_on": "Based on local employers"
    }
  ],
  "ratings": [
    {
      "link": "https://www.glassdoor.com/Reviews/Blue-Bottle-Coffee-Reviews-E803867.htm?utm_campaign=google_jobs_reviews&utm_source=google_jobs_reviews&utm_medium=organic",
      "source": "Glassdoor",
      "rating": 3.6,
      "reviews": 303
    },
    {
      "link": "https://www.indeed.com/cmp/Blue-Bottle-Coffee-Company/reviews?utm_campaign=google_jobs_reviews&utm_source=google_jobs_reviews&utm_medium=organic",
      "source": "Indeed",
      "rating": 3.3,
      "reviews": 70
    },
    {
      "link": "https://www.comparably.com/companies/blue-bottle-coffee?utm_campaign=google_jobs_reviews&utm_source=google_jobs_reviews&utm_medium=organic",
      "source": "Comparably",
      "rating": 2.5,
      "reviews": 40
    }
  ]
}
```

<h2 id='links'>Links</h2>

- [Code in the online IDE](https://replit.com/@MikhailZub/Google-Jobs-Listing-with-SerpApi#index.js)
- [Google Jobs Listing Results API](https://serpapi.com/google-jobs-listing-api)

If you want to see some projects made with SerpApi, [write me a message](mailto:miha01012019@gmail.com).

---

<p style="text-align: center;">Join us on <a href="https://twitter.com/serp_api">Twitter</a> | <a href="https://www.youtube.com/channel/UCUgIHlYBOD3yA3yDIRhg_mg">YouTube</a></p>
<p style="text-align: center;">Add a  <a href="https://github.com/serpapi/public-roadmap/issues">Feature Request</a>💫 or a <a href="https://github.com/serpapi/public-roadmap/issues">Bug</a>🐞</p>