Test less and cover more with combinations using Pairwise and Pict
Tags: testing, microsoft, pairwise, cli
Published: 2024-03-11 04:35:07

    Go straight to the point ! 
    Test less and cover more, 
    sounds good ! 

**When ?** You have a lot of data or different behaviors to test.

**Who ?** For developers and testers who want to reduce their number of tests cases.

**How does it work ?** 
> "Pairwise testing is an effective test case generation technique that is based on the observation that most faults are caused by interactions of at most two factors. Pairwise-generated test suites cover all combinations of two therefore are much smaller than exhaustive ones yet still very effective in finding defects" 
([Pairwise.org](https://www.pairwise.org/))


In this post we will use [PICT](https://github.com/microsoft/pict) to generate efficient combinations of data at entry point of your test cases. Pict is an open source CLI tool developed by Microsoft. 

_Form your own opinion_ by using this great **online console** : https://pairwise.yuuniworks.com/

---

**How to use ?**
Download the Pict.exe and open a terminal. 
It's an .exe, so you need to be on the same folder or to indicate the right path to the  Pict.exe.
```javascript
Pict my_file_to_combine.txt
```

:bulb: **_Tips_** : render the results in an external file instead of in the command line interface using a return function :
```javascript
Pict my_file_to_combine.txt > results.csv
```

:pencil: Pict is also available with a container image. Go to the README to have more infos : [README on github](https://github.com/microsoft/pict)

---

Imagine a **database with CRUD authorizations combined with user rights**. You need to test if the action of the user properly depends on the right authorization. 

Some usual exemples :

- reader can only see the user-infos-table
- contributor can create into the user-infos-table
- admin can also update the localization-table
- admin is authorized to delete / localization-table
... etc etc.

Without the Pairwise method, we would need to do several iterations for this test to cover all the possibilities.

**Let's try the Pairwise method ! Such of entries...**

```yaml
#############################################################
# User's access to your database
#############################################################

userTypes: reader, admin, contributor, editor

accessTypes: view, create, read, update, delete

database name: user-infos, localization
```

... give the followings results :

```yaml
userTypes   accessTypes   database name
admin       read          localization
admin       delete        user-infos
contributor create        user-infos
admin       view          localization
editor      create        localization
reader	    delete     	  localization
editor      view          user-infos
contributor update        localization
editor      read          user-infos
editor      delete        localization
editor      update        user-infos
contributor view          localization
admin       create        localization
contributor read          localization
admin       update        user-infos
reader      view          localization
contributor delete        localization
```

Seems like a .csv file. The first row is the header with all the entries we mentioned.

Without the Pairwise method, in the following case we would need to do at least **_32_** iterations for this test to cover all the possibilities.

With the Pairwise generation method we get **_17_** results. Amazing !


Now we can add some additional _restrictions_ :


`if [userTypes] = "reader" then ([accessTypes] = "view");`


```yaml
#############################################################
# User's access to your database 
#############################################################

userTypes: reader, admin, contributor, editor

accessTypes: view, create, read, update, delete

database name: user-infos, localization

if [userTypes] = "reader" then ([accessTypes] = "view");
```

And the results are ...

```yaml
userTypes   accessTypes   database name
admin       read          localization
admin       delete        user-infos
contributor create        user-infos
admin       view          localization
editor      create        localization
editor      view          user-infos
contributor update        localization
editor      read          user-infos
editor      delete        localization
editor      update        user-infos
contributor view          localization
admin       create        localization
contributor read          localization
admin       update        user-infos
reader      view          localization
contributor delete        localization
```

---

More details here about the Pairwise combination method : [Pairwise.org](https://www.pairwise.org/) 

You can find some useful examples of restriction in the [official documentation](https://github.com/Microsoft/pict/blob/main/doc/pict.md )

---
The time you **win** with this Pairwise method is precious to any other test action ! Think about **exploratory testing** !

Hope it helps ! 
**_Daphné Hervé (API, monitoring and QA automation lover)_**