The Painful Parts of End-to-End Test Automation for your Windows Application
Tags: testautomation, windows, appium, webdriverio
Published: 2021-01-27 21:58:39

---
title: The Painful Parts of End-to-End Test Automation for your Windows Application
published: true
description: This post will walk you through the steps required to implement Windows test automation using open-source tools. Covering setting up the test environment, configuring the client, writing your first test and overcoming the biggest obstacle to running it in a CI/CD pipeline. 
tags: #testautomation #windows #appium #webdriverio
//cover_image: https://direct_url_to_image.jpg
---

# Overview
This post is targeted at all you test engineers who do not want to add C# and all the wonderful Visual Studio tools that come with it to your test automation stack. You’re thinking, since our web app and mobile app’s e2e testing are already implemented with NodeJS why should testing our windows app be any different?

Here are the weapons we will be wielding to achieve this:

- A minimum of Windows 10 or Windows Server 2016 desktop
- An automation tool to drive your application under test (AUT) - [WinAppDriver](https://github.com/microsoft/WinAppDriver) & [Appium](http://appium.io/)
- A compatible automation client - [WebdriverIO](https://webdriver.io/)
- A test runner - [Mocha](https://mochajs.org/)
- A tool to run tests without manual user logon - [paexec](https://www.poweradmin.com/paexec/)
- An Automation Server of sorts (details of this are out of scope of this post)

Other handy extras:

- [Appium Desktop](https://github.com/appium/appium-desktop)
- [Inspect.exe](https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects)


*...Note: Many parts in this post are only vaguely covered because there are already so many great resources out there. The parts that are described in detail were difficult to find good resources on, hence this post...*

---

# Pain Point 1 - Preparing Test Environment for Automation (lil' painful)

## WinAppDriver & Appium:
WinAppDriver is Microsoft's in-house developed tool to automate Universal Windows Platform (UWP), Windows Forms (WinForms), Windows Presentation Foundation (WPF), and Classic Windows (Win32) applications and is developed with Appium integration in mind. WinAppDriver is what does the heavy lifting behind the scenes. It carries out our mouse clicks and keyboard presses as we request. The image below shows the communication flow of our solution:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/be24yizt6qgq6h2wt158.png)

Appium is a RESTful server that acts as a wrapper between WinAppDriver and your automation client. Appium accepts commands from WebdriverIO and forwards them to WinAppDriver. In response to a request, it returns a status code and logs to the automation client.

To run WinAppDriver & Appium on our test environment we need to:

1. [Download] (https://github.com/Microsoft/WinAppDriver/releases) and install WinAppDriver
2. [Enable Developer mode](https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development) on Windows
3. Install [NodeJS](https://nodejs.org/)

With these prerequisites installed you are ready to automate using WebdriverIO’s test runner - [wdio] (https://webdriver.io/docs/gettingstarted.html).

---

# Pain Point 2 - Configuring WebdriverIO and running your first test (somewhat painful)

## WebdriverIO
WebdriverIO (wdio) is a Selenium-like client service that sends requests to Appium via [W3C WebDriver Protocol](https://w3c.github.io/webdriver/). This is the framework we will use to write our test cases and specify how they will run.

Before proceeding, please initialize a `nodejs` project in your users home directory. If you're unsure how to do this [Phil Nash's guide](https://philna.sh/blog/2019/01/10/how-to-start-a-node-js-project/) is excellent. I have creatively decided to call my project `my-project`.

Taking from WebdriverIO's [guide](https://webdriver.io/docs/clioptions.html) let's set up the test-runner by opening the Command Prompt and running:

```bash
mkdir my-project & cd my-project
npm install @wdio/cli
npx wdio config
```

This will launch a helper utility where you choose what services and project structure you wish to use when running wdio. To get things started please mirror these selections:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/03t84nzzj3p6tzae5l8o.png)

Your project file tree should look like:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/7bizz71v1jxkentn62cn.png)

`wdio.conf.js` is the file that holds the configuration properties of the entire communication flow diagram in the first section. It is now loaded with the properties needed to allow communications between our webdriver client and Appium server. However, we still need to define our [capabilities] (http://appium.io/docs/en/writing-running-appium/caps/). Capabilities define which environment we are running Appium  on, which protocol Appium will use, and what AUT it will be automating. Let’s uphold tradition and use the classic calculator example. Below are the capabilities required to connect Appium to the calculator application on Windows Server 2019 along with other properties relevant to this post:

```javascript
// wdio.conf.js snippet
    runner: 'local',
    path: '/wd/hub',
    port: 4723,
    baseUrl: 'http://localhost',

    capabilities: [{
        platformName: 'windows',
        automationName: 'windows',
        deviceName: 'WindowsPC',
        app: 'C:\\Windows\\System32\\win32calc.exe'
    }],
 
    services: [
        ['appium', {
            logPath: './'     // logs are friends
        }]
    ],
 
    framework: 'mocha',
 
    specs: [
        './src/test/specs/**/*.js'
    ],
```

A few things to note here - 
1. Defining appium in `services` instructs wdio to start an Appium server
2. `Runner`, `path`, `port` and `baseUrl` properties are used to connect wdio client to the host Appium server 
3. Defining `capabilities` tells Appium what application to automate and what automation tool we want it to configure; `WindowsPC` maps Appium to `WinAppDriver`
4. We define `mocha` as the test `framework` and
5. We tell said `framework` what location the `spec` file are in

More details on Appium/WinAppDriver can be found [here](https://github.com/appium/appium-windows-driver#windowsdriver-specific-capabilities).

### Side Note: 

I highly recommend using [Appium-Desktop](https://github.com/appium/appium-desktop) as a development tool to troubleshoot any Appium connectivity issues. You can  download it [here](https://github.com/appium/appium-desktop/releases/tag/v1.19.1).

Start Appium Desktop from Start Menu, open File -> New Session Window… and enter the `capabilities` above in the 'Desired Capabilities' tab:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2p6z6nfqugvhk7ip7yhb.png)

Click `Start Session` to connect Appium to calculator app:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/j4tc0yix8pfn3foxfr76.png)

If you cannot connect to your AUT with capabilities in Appium-Desktop, you will also not be able to connect through wdio, so make sure to get things working here first. Here's a contrived error on Appium-Desktop to give an example of error logs:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2dlp1hat27m11vjfktx7.png)

Now with our capabilities sorted we can start writing test cases for the wdio client.

---

# Pain Point 3 - Controlling app elements (pretty painful)

Shifting attention to our test spec file `./test/spec/example.e2e.js`, we can define code to send requests to the Appium server. One of the beauties of webdriver is that the client object is accessible anywhere in wdio’s scope so we can directly call its properties. Let’s do some quick math to verify our calculators `sum()` functionality => `2 + 2 = 4`

```javascript
// ./test/spec/example.e2e.js
describe('Quick math', () => {
    it('Two plus two is four', () => {
        $('~132').click();              // click 2
        $('~93').click();               // click +
        $('~132').click();              // click 2
        $('~121').click();              // click equals
        const result = $('~150')        // get result
        expect(result).toHaveText('4 ') // verify it equals 4 
    });
});
```
A few things to note here:
1. Test spec files are using [mochajs](https://mochajs.org/)
2. Webdriver uses [expect](https://webdriver.io/docs/api/expect.html) for assertions
3. The [selector](https://webdriver.io/docs/api/browser/$.html) method uses `~` to identify the `AutomationId`. There’s not much clear cut documentation on this but you can work back from [here](http://appium.io/docs/en/drivers/windows/index.html) to figure it
4. Yes, I did append a space on the `4`

We are finally ready to run our tests! In the Command Prompt run

```
npx wdio
```

Here’s the result from `npx wdio` without appending a space:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/kbvzsfd3d0i3ul5frkd3.png)

And here it is with appending a space:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xnipfxdjteo6u9to44xc.png)

OK, I cheated. I should have parsed the data before asserting. But I refuse to apologize.
 
### Side Note:
Use [inspect.exe](https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects) to find the AutomationId of your elements. Open inspect.exe and hover mouse over element you wish to inspect. Here’s an example of the ‘2’ button on the calculator app:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/98fy67hjh29e9uvrw9vv.png)

Thus concludes setting up automation of a windows desktop application. 

On a side note, [here's](https://github.com/microsoft/WinAppDriver/tree/master/Samples/JavaScript/packages/webdriverio) a great example of a scalable, maintainable wdio project using the Page Object Model pattern.

---

# Pain point 4 - Running WDIO in a CI/CD pipeline without manual access to the GUI (painful)

What we have achieved so far is great for development and presentation purposes. But to provide real value and execute this in a CI/CD pipeline we will need some extra setup in our test environment. Keeping cost saving in mind, we will not want our test environment running when tests are not being executed. It also becomes clear that Remote Desktop (RDP) is not very useful when it comes to automated pipelines. This now presents an interesting challenge - how do we run our tests with no user interaction or manual login step? One way is to send the test command over ssh. This is what we will explore.

***Aside:*** configuring communication to the test environment is out of scope of this post; there are lots of great articles on setting up ssh servers out there.

This part assumes we are running our test environment in a cloud based CI/CD pipeline, therefore the test environment is not being accessed through the GUI via RDP nor is it sitting beside you on your desk. With all that in mind, let’s remotely run our test cases.

```bash
ssh ${USER}@${IP_ADDRESS} 'cd my-project & npx wdio'
```

Hmmmm, we are met with a pleasant 500 server error.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/mnxibnnni00gkizhopch.png)

Let’s see what’s going on by checking our processes in parallel while wdio is attempting to run: 

```bash
ssh ${USER}@${IP_ADDRESS} 'tasklist /fi "imagename eq node.exe" & tasklist /fi "imagename eq win32calc.exe"'
```

This gives us:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1th6t4cszlk0kzeafpdr.png)

Hmmmm, `node.exe` and `win32calc.exe` are running as services on `Session# 0`. Service processes on `Session# 0` are reserved for user agnostic processes and do not run with a GUI. Therefore Appium/WinAppDriver has no GUI output for it to find the running calculator application which also doesn't have a GUI output. This is sad times for people hoping to test the GUI.

Let’s take a step back to figure out what's going on. Let's open an RDP connection to our test environment. Here, we can directly run our tests, and in parallel check our running processes.
Using the Command Prompt (cmd) on the test environment let's run
```bash
cd my-project & npx wdio
```
And at the same time in a separate terminal run:
```bash
tasklist /fi "imagename eq node.exe" & tasklist /fi "imagename eq win32calc.exe
```
We see the tests have ran successfully:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/nma2qi76croldk399zgs.png)

And our processes are showing an RDP Session which does use the GUI. All actual logged in users get assigned `Session# > 0`.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/iba95rog5hw85c3y7z7h.png)

To solve the issue in our CI/CD pipeline we must configure our test environment to run our code as a logged in user. This will allow `node.exe` and `win32calc.exe` to run as a GUI Session.
 
To achieve this, we must do 2 things:
1. Enable Autologon  &
2. Use `paexec` to run processes as the System account user

This solution is stated by hepivax on [this thread](https://github.com/microsoft/WinAppDriver/issues/147#issuecomment-500903983).

## Enable AutoLogon:
Please use guide [here](https://docs.microsoft.com/en-us/troubleshoot/windows-server/user-profiles-and-logon/turn-on-automatic-logon).

## PAExec:
[PAExec](https://www.poweradmin.com/paexec/) is an open-source equivalent of [PsExec](https://docs.microsoft.com/en-us/sysinternals/downloads/psexec)

PAExec allows you to run interactive command-prompts on local and remote servers as the System user account (it can do a lot more too!).

Let’s install it on our test server and add it to a System Variable location so we can run it from anywhere with `cmd`:
1. [Download](https://www.poweradmin.com/paexec/paexec.exe) `paexec.exe`
2. Copy and paste to `C:\Windows\paexec.exe` (If you’re unsure of this step refer to [this](https://www.computerhope.com/issues/ch000549.htm))
3. Test `cmd` has access to `paexec.exe` by running
```bash
where paexec
```
it should return `C:\Windows\paexec.exe`

You can run `paexec` in command prompt for more details on its use cases.

Now that PAExec is installed and accessible, we can open a Command Prompt with PAExec so that `cmd` is running under System user. Let's do this and verify where we are in the file system by running:
```
paexec -s cmd /C "cd"
```
Breaking this command down:
`paexec` -> Executes paexec.exe
`-s` -> Flags PAExec to run requested process under System user
`cmd` -> Open Command Prompt
`/C` -> Tells PAExec to pass a command to Command Prompt
`“cd”` -> Once paexec runs Command Prompt as System user it executes `cd` which returns the current directory path

This outputs our System account user path, verifying we are running `cmd` as System user:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/an0y9kg3g9yciz9obaxr.png)

Now any command we choose to execute in the Command Prompt with PAExec will run under the System user

***...piecing it all together***
Over in our test automation server (or whatever you are using to remotely trigger your test suite), run the following:

```bash
ssh ${USER}@${IP_ADDRESS} 'paexec -s cmd /C "cd C:\Users\${USER}\my-project & npx wdio"'
```
***Note:*** Remember the new path is required here because we opened a `cmd` window from the System user account with PAExec.

...And in parallel check that `node.exe` and `win32calc.exe` are running as a logged on user with GUI access:

```bash
ssh ${USER}@${IP_ADDRESS} 'tasklist /fi "imagename eq node.exe" & tasklist /fi "imagename eq win32calc.exe"'
```

Let's look at our processes:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/yvtsawyc94e4ijt5irdm.png)

We see they are running under a `Session# > 0` and Voila! Test executed successfully.