Selenium WebDriver Auto Apply
By: Rob Gonzalez-Pita
Tags: Selenium, Asyncawait, Javascript Development, JavaScript, Selenium Webdriver

So you’re looking for a job? — I am, among countless others. The days of looking for work in the weekly newspaper are long gone, especially for someone with a skillset in software development. Today, the best way to browse for available Software Engineer positions is online through job boards like Indeed, Glassdoor, ZipRecruiter, and AngelList.

Further — and frustratingly so — many of my Software Developer peers are typically hired internally or from a referral. But what does a job seeker do when they don’t have those connections, or already work for a company?

It’s time-consuming to search for a company that works on exciting projects, has a great work/life balance, and has an excellent corporate culture. Job seekers must go to company websites, navigate to their career page, check if there are any open positions, and ultimately gauge whether they are interested in any of those positions. That is if the job seeker exhibits the required qualifications.

With the advent of the online job board, it has become easier to search a single site, as opposed to visiting each potential employer’s listings separately. While this makes the application process more straightforward, it also introduces a whole new set of complexities.

When I’m looking at a job board, I often find myself asking:

Are all the available jobs that are of interest to me listed on the job board?

Do companies listing these available positions to a job board ever hire from that source?

How many hundreds of other people are applying for this one job?

If my application stands out amongst the competition, how do I know?

Who is the person on the other end looking at the applications?

And ultimately, I find myself asking…

Is it worth my time to apply to positions listed on job boards?

I should begin with a caveat — job boards are a good way to find available positions at companies, as well as to find companies to individually reach out to.

However, I don’t think searching and applying for jobs ad infinitum is worth my time. I want to spend the majority of my time building new websites and applications. I want to spend my time networking with others in the field, learning about the work of others, and speaking with individuals who work at companies that interest me.

In reality, I want to spend the least amount of time applying to positions online.

In the past, I’ve been hired for positions that I found online(with a little bit of legwork reaching out to hiring managers). However, in a world with growing tech talent and an increased desire to work remotely, it often feels as if I’m sending my applications into the void.

So, I asked myself…

Why not create a program that applies to all the jobs on a specific job board?

The aggregated position listings on job boards are a perfect place to apply for each one methodically and sequentially. The Selenium WebDriver program I created automates the search filters, field population, and cover letter process, so the user doesn’t have to spend their time doing so.

Disclaimers: This wasn’t my original idea, and second, my expectations are pretty low in gaining employment from this approach.

This project initially began as a way to increase my efficiency and volume of applications sent. However, it also served as a great exercise in learning browser automation with WebDrivers and technologies associated with them.

Companies automate the resume screening process for potential employees. Why shouldn’t job seekers automate the application process?

While it saves time, Selenium WebDriver is probably not the best approach in creating an original application and cover letter. However, I wonder if the hiring manager reads each cover letter and application? Or, do they skim resumes and choose the first ten that catches their attention?

While creating your own Selenium WebDriver Auto Application program may not be a perfect solution, it does return time spent on the application process to the job seeker.

I’ve discussed my mindset in the efficacy of applying to positions on job boards, and below I will implement a solution to some of the qualms I have with this whole process.

TL;DR — If you’re tired of applying ad nauseam for job after job, the rest of this article will give you some direction in how to go about automating some of those processes. Most developers use Selenium for automation testing (to see how a ‘virtual user’ would interact with a website and if that site is properly functioning). I’ve adapted the use of Selenium WebDriver to automate the job-hunting process.

INSTALLATION PROCESS

Before starting, users should have familiarity with the command line, IDEs, and basic file structure.

Install Selenium:

npm install selenium-webdriver

Install a webdriver (I used Chromedriver and would suggest it). Make sure you are downloading the correct version. You can find your version of Chrome by clicking the three dots in the very top right of the browser window:

Update the path of downloaded Chromedriver exe:

sudo mv Chromedriver /usr/local/bin

This link helped my understanding of including the ChromeDriver location in my PATH environment variable. The PATH variable is what allows Chrome to find the downloaded ChromeDriver exe file.

If you’re using macOS, you also may have to update your system preferences to allow the Chromedriver to be downloaded from an untrusted source.

Heads up… this step could be very simple, or it could take some time to get it all working properly.

Once the Chromedriver is installed and the PATH environment variable is pointing to the correct location, you should be able to open up a .js file in your IDE. Then, cut and paste the code below, run it, and watch ChromeDriver do its magic. If the magic isn’t happening, you’ll most likely have to ensure your PATH is correct and/or troubleshoot your specific error message.

Little code to get you started and see if you’ve done the Selenium installation correctly:

After verifying that the WebDriver and Selenium are working correctly, now is where things get interesting. From here, the sky’s the limit with leveraging Selenium WebDriver to your advantage in a variety of tasks you’d like to automate (for non testing purposes). Now you can navigate to specific websites and interact with the page (i.e., login using your username and password).

As you can see (if you’ve followed the instructions, and the above code is working) the whole process of creating a new browser window, performing tasks, and quitting the ‘driver’ is wrapped in an asynchronous function that is immediately invoked (IIFE). Some familiarity or willingness to learn how asynchronous code works with Javascript as a single-threaded language is required to continue building off of this starter code.

A great video introduction of this concept and sandbox can be found here: http://latentflip.com/loupe

If you already have a basic understanding of asynchronous javascript code, you’re ready to move on and start getting Selenium WebDriver to interact with some different websites.

Complications do arise with sites that don’t have easily identifiable HTML element selectors (IDs or unique class names). Your classic `document.getElementById()` or `document.getElementsByClassName()` (in Selenium: `.findElement(By.name()` ) methods become quite challenging. It could have just been some syntax errors, but this led me down the road of using Xpath to select elements in the DOM. Xpath identifies elements in the XML document (another can of worms, referenced here) and allows the user to implement different methods in searching for elements in Selenium.

Note: It can be a wild goose chase to find the correct elements and get the WebDriver to interact with them appropriately. This is a great resource to understand Xpath and how to use Xpath locators for identifying more difficult to access HTML elements.

Hopefully, that’s enough background information to get you started on your journey of using Selenium WebDriver to create ‘automated tests’ . Or, better yet, write a program that automatically applies to jobs for you.

For the sake of brevity, I won’t give step by step instructions on how exactly I went about writing the code that automated applying for jobs on angel.co. But if you’re interested looking at how I went about it, you can fork and clone my repo- github.com/coloradorca/SeleniumApply

Special thanks to Dylan, Myles, and Max for the assistance along the way.