Getting Started With Browser Automation Testing in Python
Tags: python, tutorial, testing
Published: 2019-09-27 05:22:35


---
title: Getting Started With Browser Automation Testing in Python 
published: true
description: Developers have unit tests to test atomic functionality and integration tests to test system interoperability. Web developers have to take it a step further and test actual browser behavior. This can be done in many ways, but most often it&#8217;s with some implementation of Selenium webdriver and an xUnit testing framework. In this article, I&#8217;m [&hellip;]
canonical_url: http://erikscode.space/index.php/2019/09/27/getting-started-with-browser-automation-testing-in-python/
cover_image: https://i0.wp.com/erikscode.space/wp-content/uploads/2019/09/david-clode-RAfIk-ZKbPk-unsplash.jpg?fit=5886%2C3818
---

Developers have unit tests to test atomic functionality and integration tests to test system interoperability. Web developers have to take it a step further and test actual browser behavior. This can be done in many ways, but most often it’s with some implementation of Selenium webdriver and an xUnit testing framework. In this article, I’m going to show you how to write a basic framework in Python to get your tests able to interact with a browser.

What Exactly are we Doing?
--------------------------

Before we get started, let’s make sure we know what we’re doing and why. Browser tests, or end to end tests are exactly what they sound like. You think about the steps a user would take, and you write code that emulates those actions. This isn’t abstract like an integration test that sets off a chain of API calls in the order a user might trigger, we are actually going to drive a browser with our code.

In our code, we are going to test out the functionality of a multi-language news broadcaster. (It’s the web page I used to write a JavaScript observer pattern tutorial, the article for that is [here](https://dev.to/erikwhiting88/observer-design-pattern-tutorial-in-javascript-fun-with-a-language-api-21o3)).

The code we’re going to write in this article can be found in [this GitHub repository](https://github.com/erik-whiting/LuluTest/tree/blog-example). We are using my personal LuluTest Python testing framework, but I will leave this branch unchanged for the purposes of this tutorial.

Overview
--------

Our test framework will have 2 functional components, and a testing component (like, for testing the framework). The two functional components are

- **Configuration Module:** This will consist of a `Config` class that we’ll use to set things like
  - *The web driver* which means what kind of browser will be doing our tests (Chrome, FireFox, etc.).
  - The URL including the http prefix, subdomain, and port if necessary.
- **Page Module:** In this module, we’ll have 2 classes
  - The `BasePage` class will have a `Config` element and methods for going to a URL, closing the browser, and selecting elements within a page.
  - The `BaseElement` class represents elements on the web page we’re testing and contains helper methods for interacting with the different kinds of elements we could encounter.

Set Up
------

To get started, you need a few things. First of all, I’m using Python 3.6 but I’ve seen it work with earlier versions. I can almost guarantee it won’t work with 2.7, but I don’t know that for sure.

You also need Selenium. Go on over to your terminal and run a `pip install selenium` or do it within the virtual environment once you’ve started the project.

Finally, we need a webdriver. I’ll be using Chrome for this tutorial. You can download Chrome webdriver [here](https://chromedriver.chromium.org/downloads) and you need to put the driver’s exe file in your system’s path variable. Alternatively, you can pass the path of the driver but you will need to do so any time you see my code say something like `webdriver.Chrome()` or `webdriver.Chrome(chrome_options=chrome_options)`. Just pass the path like another parameter.

> Full disclosure regarding passing the path of Chrome driver. This definitely used to be the case, but I cannot actually find anything supporting this claim right now. It’s possible you don’t have to pass the webdriver’s .exe path anymore. If these tests work without you passing the path or putting it in your own environment’s path variable, please let me know!

And that should be it! Let’s get coding.

Whet Your Appetite
------------------

This is actually going to be a bit of a long process, so let me first show you what selenium web driver is capable of. Once you’ve finished setting up, open up a python terminal and do this:

```
>>> from selenium import webdriver
>>> from selenium.webdriver.common.keys import Keys
>>> driver = webdriver.Chrome()
# Give it a second, a browser should pop up
>>> driver.get("http://www.google.com")
>>> search_box = driver.find_element_by_name("q")
>>> search_box.send_keys("python")
>>> search_box.send_keys(Keys.RETURN)
>>> driver.close()
```


Isn’t that cool??? Basically, today we’re going to make all of this more robust and quicker to write.

Config Class
------------

Inside your project, make a Python module called Configs and create a `Config.py` file.

> I wanted to write this article TDD style, but for the sake of brevity, we’re going to skip tests for the tutorial. I don’t want you to get bogged down in a war-and-peace size article. I will include the test files at the bottom of this article for anyone interested.

The purpose of the config class is mostly to handle the URL for the page to be tested as well as the kind of driver we’ll be using. When we make the actual automated tests, we’ll need to be able to handle any kind of URL. We could have subdomains or ports. We could also have neither, so let’s make sure we don’t make a URL function that returns something like .google.com:

Here’s what I came up with:

```python
class Config:
  def __init__(self):
    # Set your configuration items here
    self.driver = 'Chrome'
    self.headless = False
    self.base_url = ''
    self.subdomain = ''
    self.http_prefix = 'http://'
    self.port = ''

  def url(self):
    full_url = self.base_url
    if self.subdomain:
      full_url = self.subdomain + '.' + full_url

    if self.port:
      full_url = full_url + ':' + self.port

    full_url = self.http_prefix + full_url

    return full_url
```


Nothing super fancy going on here, especially nothing related to testing with a web driver, so I won’t talk about it too much. Do take note, however, that the `driver` attribute is a string. This is because we’re going to instantiate the actual driver within the `Page` class.

Also note, I’ve set most of these attributes myself. This is because the project (LuluTest) is a personal project and at the time of writing, it’s a bit basic. Feel free to add some more configurability if you’d like.

BasePage Class
--------------

Finally, we get to some selenium stuff. The `BasePage` class is responsible for instantiating the actual webdriver class, be it Chrome, Safari, or otherwise. Also, this class will be responsible for going to a URL, closing the browser, and finding elements. Let’s get started with the constructor:

```python
from selenium import webdriver
from selenium.webdriver.common.by import By
from Page.BaseElement import BaseElement
from selenium.webdriver.chrome.options import Options


class Page:

  def __init__(self, config, url_extension=''):
    self.driver = config.driver
    self.headless = config.headless
    self.page = self.web_driver()
    if not url_extension:
      self.url = config.url()
    else:
      self.url = config.url() + '/' + url_extension

```


Let’s start with the last parameter `url_extension` which we default to nothing. The rest of the URL will be configured with attributes from the `config` object we pass, and users will be able to add something like “path/to\_page/being\_tested” if needed. The `config` object also provides the `headless` boolean and `driver` string which is used to configure the `web_driver` attribute, which we call `page`. Here’s the code for that bit:

```python
def web_driver(self):
  if self.driver == 'Chrome':
    chrome_options = Options()
    if self.headless:
      chrome_options.add_argument("--headless")
    return webdriver.Chrome(chrome_options=chrome_options)
  elif self.driver == 'Safari':
    return webdriver.Safari()
```


To set the `page` attribute, we first instantiate an `Options` object, which is passed to the webdriver class constructor and contains the headless option, among other things.

> Notice here that the code only does this for Chrome driver because I have never tested this with any other browser. Like I said earlier, this is an early stage project. Feel free to add options for Safari or other browsers, but keep in mind you might have to do some of your own research to keep up with this tutorial.

The `page` attribute is the *true* webdriver in this class and lets us actually interact with the browser. As such, our `go` and `close` methods will use it.

```python
def go(self):
  self.page.get(self.url)

def close(self):
  self.page.close()
```
Nothing special here, but now you know: to make a webdriver object go to a web page, you use `webdriver.get(url_of_page)`. We’ll see this in action soon. Also, the `close` method closes the browser (really??).

The next thing our `BasePage` class needs is a way to grab elements. But before we get to that, we’re going to make the `BaseElement` class and then come back to this one.

BaseElement Class
-----------------

This is my favorite part. The `BaseElement` well represent elements on a page such as buttons, input boxes, and other such things. It will also allow us to manipulate those objects, which is obviously a necessity when testing browser features.

Once we’re on a page, we only need two things to select an element: the property we’re going to select it by, and the value of that thing. For example, if we want to enter text into an input box with an id of “username”, the `by` is “id,” and the `value` is “username”.

Usually, we do this by writing something like `web_driver.find_element(By.ID, "username")` but there’s one thing we have to worry about.

Webdrivers are ***fast*** as heck. Sometimes our tests will fail because the test is looking for an element that has not yet rendered. The element could load only a split second after the test script tries to get it, but by then it’s too late, and the test will fail because it couldn’t find the element.

To combat this, sometimes developers will put in `sleep` commands. Every time you want to do that, I want you to slap yourself in the face with a newspaper and yell “no! bad developer!”

We don’t use sleeps for two reasons. One, if you have 200 tests that interact with an average of 3 elements per test, and each time you select an element, you sleep for 2 seconds, your test can never run faster than 20 minutes. Two, you actually don’t know if your sleep is long enough. Maybe that element is having a hard day and takes 4 seconds to render but you only waited for 2. Luckily, there is a solution, so let’s just jump into the code.

We’ll start with the constructor and the method that returns an actual web element object we can manipulate:

```python
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.wait import WebDriverWait


class BaseElement:
  def __init__(self, by, value, driver, name=''):
    self.driver = driver
    self.by = by
    self.value = value
    self.name = name
    self.locator = (self.by, self.value)
    self.element = self.web_element()

  def web_element(self):
    return WebDriverWait(self.driver, 10).until(ec.visibility_of_element_located(locator=self.locator))
```


The solution to the problem of slow rendering web elements is in the `web_element` method. Notice we use a method called `WebDriverWait`. This comes from selenium and takes two parameters: a web driver object and a timeout. The timeout is the only time you’re allowed to hardcode seconds, and this attribute basically says “ok, after x amount of seconds, we’re just goonna assume the element isn’t rendering.” You can adjust this as you see fit.

We also call the `until` method which takes in an `expected_conditions` object (which we’re calling `ec` in the code above) property. In this case, we use the property of `visibility_of_element_located` which basically says “when it shows up on the page” and pass it the element’s locator (which we’ll talk about in a second).

This is all a fancy way of saying “give me this element as soon as it shows up on the page; if it takes more than 10 seconds, give up.”

Also notice in the constructor we are creating a tuple called `locator`. This is going to be populated with values that will come from the `BasePage` object, but let’s talk about it real quick. The `by` parameter will be translated in the `BasePage` object to return an actual `By` object, which could be Id, Xpath, class name, or a few other things. The “value” part of this tuple is the value of the Id, Xpath, class name or whatever of the element we’re trying to find. To reuse the example of the username input box, the locator tuple would be `("id", "username")`.

What good is having an element object if we can’t do stuff to it? Let’s finish up this class by adding some fairly self explanatory methods and one decorated method:

```python
def click(self):
  self.element.click()

def input_text(self, text):
  self.element.send_keys(text)

def clear(self):
  self.element.clear()

def clear_text(self):
  self.element.send_keys(Keys.CONTROL + 'a')
  self.element.send_keys(Keys.DELETE)

def select_drop_down(self, index):
  self.element.select_by_index(index)

@property
def text(self):
  return self.element.text
```


 These methods manipulate the object and let us click, add text, and clear the object. One note about the `clear_text` method: The pure `clear` method does not always work the way you would expect, so this method essentially gets inside the element, types control+a (the keyboard shortcut for selecting all), and presses the delete key.

The `text` method lets us evaluate the text in the element and works for input boxes, paragraph elements, and just about anything else that might have text in it. The `@property` line is called a “decorator” and to be honest with you, I have no idea what it’s for. But this code will not work without it. \*shrug\*

Now let’s head back to our `BasePage` class and finish it out. We’re getting close to being done!

Back to BasePage
----------------

The last part of the `BasePage` class will be the `element_by` method in which we will take in “indicator” and “location” as arguments. The indicator will be used to get that `By` object we talked about earlier, and the locator will be the value. Here’s the code:

```python
def element_by(self, indicator, locator):
  indicator = indicator.lower()
  indicator_converter = {
      "id": By.ID,
      "xpath": By.XPATH,
      "selector": By.CSS_SELECTOR,
      "class": By.CLASS_NAME,
      "link text": By.LINK_TEXT,
      "name": By.NAME,
      "partial link": By.PARTIAL_LINK_TEXT,
      "tag": By.TAG_NAME
  }
  return BaseElement(indicator_converter.get(indicator), locator, self.page)
```


As you can see, there are quite a few ways to identify an element outside of its Id.

### Side Note

I do want to caution you about one thing regarding selectors. Try to avoid using xpath when you can. The reason for this is because if an element changes places on a screen either by function or because of tweaks in design, your xpath patter has to change, making the test very fragile. Grab by unique IDs when you can.

One More Thing
--------------

Before we write the test, I want you to write one more thing for the sake of keeping the code readable. Make a package called `tests` and create 2 files, `helpers` and `test_feature`. We’re going to make one method in the helper class that will keep us from having a six foot wide line of code in our tests later. Here’s `helpers.py`:

```python
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait


def evaluate_element_text(element, text):
  WebDriverWait(element.driver, 10).until(ec.text_to_be_present_in_element(element.locator, text))
  return element.text == text

```


This is going to take in an element object and some text we expect to have and return true or false. Instead of writing that long line every time we need to evaluate an element’s text, we’ll just call this method.

See it in Action!
-----------------

We are officially done building our framework and it’s time to write our first test script. Open `test_features.py` and lets get started.

First, we do our imports, then we’ll do a little set up:

```python
import unittest
from Configs import Config
from Page import BasePage
from tests import helpers as helper


class TestFeature(unittest.TestCase):
  cf = Config.Config()
  cf.base_url = 'erikwhiting.com'
  cf.subdomain = ''
  cf.base_url += '/newsOutlet'
  bp = BasePage.Page(cf)
```


Then, we can write all the tests for that page that we want. For our example, we’re going to go to the news site, enter “Hello” in the input box, click the transmit button, and then make sure the div that gets transmitted to has the word “Hello” in it. Behold!

```python
def test_write_and_click(self):
  bp = self.bp
  bp.go()
  bp.element_by("id", "sourceNews").input_text("Hello")
  bp.element_by("id", "transmitter").click()
  english_div = helper.evaluate_element_text(bp.element_by("id", "en1"), "Hello")
  self.assertTrue(english_div)
  bp.close()
```


Basically, in the first part we instantiate our `Config` object, and use it to create our `BasePage` object. From there, the test script is pretty easy to write. We `go()` to the URL, get the element with the ID “sourceNews” and input “Hello” into it. We find the element with the “transmitter” ID, and `click()` it. Then we find the element with ID “en1”, send it to our helper method along with the text we expect, and then use the `unittest` method `assertTrue` to evaluate it.

In a terminal in your project root, write:

```
$> python -m unittest tests.test_features
```


and hit enter. See the browser go!

Closing Remarks
---------------

Did that seem like a lot of work? Well, it may have been, but there’s some things to keep in mind. Not only are we getting elements in an efficient non-sleep way, but we are capturing a lot of functionality in just a couple of classes. All the automated tests we write from here on out will be a breeze, and that’s the real time saver.

One more thing, a bit of a self plug. If you’re interested in this project, you are more than welcome to make a pull request on its github. The first goal of this project is to be as easy and configurable as possible. The second goal is to then be turned into a testing DSL to allow less technical users to write tests. No pressure though! Just throwing it out there.

The Tests
---------

Like I said earlier, I really wanted to do this tutorial in a TDD type of way, since I ***love*** TDD, but that would have made this already long article even more so. But, as promised, there are the tests that are included in the GitHub repo:

`test_config.py`

```python
import unittest
from Configs import Config


class TestConfigs(unittest.TestCase):

  test_url = 'eriktest.com'
  test_sub_domain = 'test'
  test_port = '5000'

  def test_config_returns_basic_url(self):
    cf = Config.Config()
    cf.base_url = self.test_url
    cf.subdomain = ''
    cf.port = ''
    self.assertEqual(cf.url(), 'http://' + self.test_url)

def test_config_returns_url_with_subdomain(self):
    cf = Config.Config()
    cf.base_url = self.test_url
    cf.subdomain = self.test_sub_domain
    cf.port = ''
    self.assertEqual(cf.url(), 'http://' + self.test_sub_domain + '.' + self.test_url)

  def test_config_returns_url_with_port_only(self):
    cf = Config.Config()
    cf.base_url = self.test_url
		cf.subdomain = ''
		cf.port = self.test_port
		self.assertEqual(cf.url(), 'http://' + self.test_url + ':' + self.test_port)

  def test_config_returns_url_with_port_and_subdomain(self):
    cf = Config.Config()
    cf.base_url = self.test_url
    cf.subdomain = self.test_sub_domain
    cf.port = self.test_port
    val_to_test = 'http://' + self.test_sub_domain + '.' + self.test_url + ':' + self.test_port
    self.assertEqual(cf.url(), val_to_test)

```


`test_base_page.py`

```python
import unittest
from Configs import Config
from Page import BasePage


class TestBasePage(unittest.TestCase):
  cf = Config.Config()
  cf.driver = 'TestDriver'
  cf.base_url = 'TestMe.com'

def test_base_page_returns_config_url(self):
  bp = BasePage.Page(self.cf)
  self.assertEqual(bp.url, self.cf.url())

def test_bast_page_returns_config_url_with_sub_dir(self):
  bp = BasePage.Page(self.cf, 'about')
  self.assertEqual(bp.url, self.cf.url() + '/about')
```


`test_feature.py`

```python
import unittest
from Configs import Config
from Page import BasePage
from tests import helpers as helper


class TestFeature(unittest.TestCase):
  cf = Config.Config()
  cf.base_url = 'erikwhiting.com'
  cf.subdomain = ''
  cf.base_url += '/newsOutlet'
  bp = BasePage.Page(cf)

def test_write_and_click(self):
  bp = self.bp
  bp.go()
  bp.element_by("id", "sourceNews").input_text("Hello")
  bp.element_by("id", "transmitter").click()
  english_div = helper.evaluate_element_text(bp.element_by("id", "en1"), "Hello")
  self.assertTrue(english_div)
  bp.close()

```