Automatically reduce test cases using pairwise methods
Tags: testing
Published: 2022-03-03 12:40:45

When testing an application, the number of test cases depends on which browser is used, which device is used, and which impression method is used.

Let's take a look at this case.

- devices: Mac, Windows, iPhone, iPad, Android
- browsers: Safari, Chrome, Firefox, IE
- login authentication methods: Facebook login, Twitter login, Google login, Instagram login

The number of all test cases is calculated as follows.
This is simple elementary level math.

```
devices (5) x browsers (4) x login authentication methods (4) = 80 cases
```
So there are 80 different ways.
Well, I can do it if I share it with my colleagues, but I don't want to if I can help it.

## Use pairwise methods to overwhelmingly reduce the number of test cases
A simple explanation of the pair-wise method is: Basically, an app bug is caused by one or two factors. Therefore, even if there are more than three factors, there is no need to check all the combinations.

However, it is a bit difficult to adapt the pairwise method to already existing test cases on your own, so you should use an open source tool called "PICT" published by Microsoft.

## Install PICT
You can implement it in the following steps.

```
$ git clone https://github.com/Microsoft/pict.git
$ cd pict/
$ make
$ sudo install -m 0755 pict /usr/local/bin/pict
```
Now that the pict command is available, the next step is to write the test case so that pict can read it. The way to write it is super simple and looks like this.

```
devices: mac,windows,iphone,ipad,android
browser: safari,chrome,firefox,ie
login_authentication_method: facebook,twitter,google,instagram
```
Save this with a .txt extension.

## Run PICT
Now let's see how many test cases have been reduced from the 80 simple calculations made earlier.

```
pict test_case.txt
```
result
```
device	browser	login_authentication_method
windows	firefox	facebook
ipad	safari	twitter
android	chrome	google
windows	ie	instagram
ipad	firefox	google
iphone	ie	google
windows	safari	google
iphone	safari	instagram
ipad	ie	facebook
iphone	firefox	twitter
mac	safari	facebook
android	ie	twitter
mac	ie	google
mac	chrome	twitter
iphone	chrome	facebook
android	firefox	instagram
windows	chrome	twitter
ipad	chrome	instagram
android	safari	facebook
mac	firefox	instagram
```

**Down to only 20 cases (1/4)!!!**

For wrong combinations like Mac+IE, please google for details as you can write conditions and drop them.

If you simply want to know the number of cases, you can run it with /s at the end.
```
$ pict test_case.txt /s
　　　　Combinations:   56
　　　　Generated tests:20
　　　　Generation time:0:00:00
```
If you are wondering if this is really reliable, please read the paper published by Microsoft.
[Pairwise Testing in the Real World: Practical Extensions to Test-Case Scenarios](https://docs.microsoft.com/en-us/previous-versions/software-testing/cc150619(v=msdn.10)?redirectedfrom=MSDN)

## Advertisement
I am working on a web app called [spliito.com](https://bit.ly/3H7w3Zf) that solves the tedious task of recording the money owed and borrowed that occurs when you go on a trip with friends in a spreadsheet and calculating who should pay back how much to whom at the end of the trip.

![screen](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qlh32pmf0tb3z4u8xmz6.png)
[Created a product used by 50,000 people per month](https://dev.to/miyachin/created-a-product-used-by-50000-people-per-month-3m50)