How to scrape Indeed jobs and company profiles
Tags: jobs, api, career
Published: 2023-10-24 10:40:47

---
title: How to scrape Indeed jobs and company profiles
published: true
date: 2023-07-24 22:00:00 UTC
tags: jobs, API, careers
canonical_url: https://blog.apify.com/scrape-indeed-jobs/
---

Indeed's APIs don't cover the job listings available on the site. The good news is that because of its clear categories and customizable parameters, the Indeed website is perfect for web scraping.

[Indeed](https://gotajob.indeed.com/) helps people find their dream jobs. Or at least the ones that meet their standards. Indeed has a few of its own APIs, but they don't cover the job listings available on the site. The good news is that because of its clear categories and customizable parameters, the Indeed website is perfect for [web scraping](https://apify.com/web-scraping). So, let's give scraping Indeed a try using a specialized scraping tool.

{% embed https://youtu.be/0zhoR0AQ4ME %}

## **Can I scrape jobs using Indeed API?**

Not really. There are several Indeed APIs available. In the past, Indeed used to have an API group designed to scrape Indeed job listings, it was called [Publisher Jobs API](https://developer.indeed.com/docs/publisher-jobs) (_Get Job API_ and _Job Search API_). However, these APIs have been deprecated. The Job Search API allowed users to access data like job titles, company names, locations, posting times, and job descriptions. Due to these changes, users have been looking for alternatives to Indeed APIs, such as this [Indeed web scraper](https://apify.com/hynekhruska/indeed-scraper) 🔗

### **Can I scrape applicant info using Indeed API?**

To an extent. The available Indeed APIs are useful for tasks like integrating Indeed data into applicant tracking systems, monitoring applicant conversions, or managing interview schedules. They mainly cater to the hiring audience of the platform. Unfortunately, they have a number of limitations and are not suitable for job searching purposes.

Sadly, limitations on website APIs have become very common these days. Websites with huge user bases rarely leave their APIs available and open, even if they did in the past. This is where scraping comes in. Because of the rising trend in API limits, over time we've built hundreds of ready-to-use tools for scraping and automation to keep the public data available for automated extraction. If youre interested in keeping track of job listings on [Indeed](https://gotajob.indeed.com/), you can try using [Indeed Scraper](https://apify.com/hynekhruska/indeed-scraper) 🔗 to automate the process of collecting and extracting the data.

## **🗂 How does Indeed scraper work?**

[Indeed Scraper](https://apify.com/hynekhruska/indeed-scraper) 🔗 works in a way similar to manually searching through the Indeed website and extracting information from each page. But it's way faster!

It starts by going over to the Indeed website and navigating to the specified location. Then, it enters your search query into the search bar and proceeds to scroll down until it reaches the final page. The scraper queues up all the job listings it finds and efficiently copies and organizes all the visible data into a well-structured document. All you have to do is download it.

## **🖇 How can I use the data extracted from Indeed?**

| **For job seekers**                                                                   | **For recruiters**                                                                  |
|---------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|
| 🎖 Extract job listings by titles, benefits, skills, certifications                    | 👥 Collect candidate data and automate the applicant management process              |
| ⏱ Extract Indeed job openings with relevant positions the moment they appear          | 💰 Compare salaries in the industry to determine fair market rates for similar roles |
| 🏡 Find job positions with specific requirements (part-time, WFH, no experience, etc.) | 🔍 Research to support investment decisions for future office space needs            |
| 💶 Get salary estimates on job positions posted on Indeed                              | 📊 Analyze competitors, track hiring trends for skills and technology gaps           |
| 📇 Analyze hiring trends, forecast skills, and job role demands                        | 😊 Assess employee sentiment                                                         |



## **📇 Step-by-step guide to Indeed scraping**

### **Step 1. Find Indeed Scraper**

First, go to [Indeed Scraper's page](https://apify.com/misceres/indeed-scraper), and click the **Try for free** button.

![indeed salary web scrape.png](https://blog.apify.com/content/images/2023/07/indeed_salary_web_scrape.png)

**_Step 1. Find Indeed Scraper_**

If you're not signed in to the Apify platform, clicking the button will take you to the signup page. You can sign up using your email account, Google, or GitHub and try the Indeed scraper for free.

![indeed job openings.png](https://blog.apify.com/content/images/2023/07/indeed_job_openings.png)

**_Sign in to the Apify platform. Don't worry, we don't bite_ 😁**

### **Step 2. Find Indeed jobs to scrape**

You can now access the scrapers page in Apify Console. This is where you can set up, run, and download data from scraping. Now let's fill in the required input fields, mainly:

- **Positions:** e.g. let's look for _data analyst_ jobs on Indeed

- **Country:** for our example, were going with the US

- **Location:** were using _San Francisco_

- **Max items:** let's get _100_ job listings.

We will also use the **Scrape company details** toggle to get the hiring companies' info such as name, rating, number of reviews **.**

![indeed job scraping.png](https://blog.apify.com/content/images/2023/07/indeed_job_scraping.png)

**_Step 2. Find Indeed jobs to scrape_**

### **Step 3. Start scraping Indeed**

Once you are all set, click the **Start** button. You will notice that your task will change its status to **Running 🏃**. It will be just a minute before you see the status switch to **Succeeded**✅

![scrape indeed job postings.png](https://blog.apify.com/content/images/2023/07/scrape_indeed_job_postings.png)

**_Step 3. Start scraping Indeed_**

### **Step 4. Download Indeed job listings**

When the status changes to **Succeeded** ✅, you can check the results in the **Storage** tab.

We've managed to scrape 100 data analysis job postings together with information on the companies that offer these positions. Now we can preview and download this data as Excel, HTML table, JSON, CSV, or XML. We can also share the scraped Indeed data directly via an API.

![scrape indeed python.png](https://blog.apify.com/content/images/2023/07/scrape_indeed_python.png)

**_Step 4. Download Indeed job listings_**

### **Step 5. Bonus: set up job scraping Indeed every hour, day, week, or month**

You can set up the whole scraping job to run automatically every day. First, you have to save your scraping parameters (specific location, job position, number of listings) as a **task**. This step will make sure the scraper always has the right input which you now don't have to type in yourself. Then click **Actions ->**  **Schedule.**

Now set up the timing conditions for your scraping task: how frequently it should scrape Indeed, at what time, and in which time zone. All you have left to do is click **Create**. Last but not least, you can also **set up a notification to get an email** every time your scheduled Indeed scraping has been completed.

![downoad indeed job posting.png](https://blog.apify.com/content/images/2023/07/downoad_indeed_job_posting.png)

![](https://blog.apify.com/content/images/2023/07/jobs-web-scraping.png)

**_Step 5. Set up job scraping every hour, day, week, or month_**

## ❓**FAQ**

### **Can I scrape data from LinkedIn?**

Yes. If you dont want to scrape Indeed job postings and rather get data from another job-hunting platform, you can use some of the other available job listing scrapers. You can [scrape reviews on Glassdoor](https://apify.com/alexey/glassdoor-jobs-scraper), [scrape data from LinkedIn](https://apify.com/bebity/linkedin-jobs-scraper), or freelancers offers on [Upwork](https://apify.com/trudax/upwork-scraper) or [Fiverr](https://apify.com/newpo/fiverr-scraper). Just browse our 30+ available [job listing scrapers](https://apify.com/store/categories/jobs).

| 🔍 [Google Jobs Scraper](https://apify.com/dan.scraper/google-jobs-scraper)                        | 🚪 [Glassdoor Scraper](https://apify.com/alexey/glassdoor-jobs-scraper)    |
|----------------------------------------------|------------------------|
| 🚀 [Crunchbase.com Scraper](http://Crunchbase.com)[Scraper](https://apify.com/epctex/crunchbase-scraper)                     | 💼 [Upwork Scraper](https://apify.com/trudax/upwork-scraper)       |
| 👹 [Monster Job Search Scraper](https://apify.com/mscraper/monster-job-search-scraper)                 | 🎨 [Fiverr Scraper](https://apify.com/newpo/fiverr-scraper)       |
| 👔 [LinkedIn Company URL - Mass Profile Finder](https://apify.com/anchor/linkedin-company-url-finder) | 💼 [LinkedIn Company Ads](https://apify.com/saswave/linkedin-company-ads) |
| 📰 [LinkedIn Posts Scraper](https://apify.com/curious_coder/linkedin-post-search-scraper)                     | 🏙️ [Craigslist Scraper](https://apify.com/ivanvs/craigslist-scraper)   |
| 


> ❓[**How to scrape LinkedIn companies and profiles**](https://blog.apify.com/how-to-scrape-linkedin-company-pages/)
> 
> [Learn how to scrape LinkedIn for URLs with these scrapers.](https://blog.apify.com/how-to-scrape-linkedin-company-pages/)

### **Is it legal to scrape Indeed data?**

Web scraping is legal. However, different rules set by regulations such as GDPR or CCPA may apply based on where you are in the world. So be careful when scraping personal data (users, resumes, and other sensitive information) and avoid websites that are not publicly available or replicating copyrighted content. Read our [is web scraping legal article](https://blog.apify.com/is-web-scraping-legal/) to learn more about the subject.

### **Can I scrape Indeed with Python?**

Yes. To carry out Indeed scraping using Python, you can use Apify API. It gives you programmatic access to the Apify platform: any datasets with scraped data, performance metrics, results, versions, and more. You can access the API using Python via `apify-client` [PyPI package](https://apify.com/misceres/indeed-scraper/api).

### **Can I scrape Indeed company reviews?**

In theory, you can scrape anything available on the website. So Indeed reviews could be scraped. Indeed Scraper doesn't have that functionality yet, but you can request it using the [Issues tab](https://console.apify.com/actors/hMvNSpz3JnHgl5jkh/issues). Another way would be to send a request to one of our community devs on [Discord](https://discord.com/invite/crawlee-apify-801163717915574323); they'll make a custom scraper for Indeed reviews for a fee.

Last but not least, you can build this Indeed Reviews scraper yourself - by using [scraper templates](https://apify.com/templates) or the open-source scraping library [Crawlee](https://crawlee.dev/).