Tips for Executable Specification and Cucumber
By: Felipe Carvalho
Tags: Runnable Specification, Atdd, Acceptance Tests, Specification By Example, Cucumber

I have been working with Cucumber doing executable specifications by example for some time now, so I decided to create a collection of recommendations to make specifications more readable in general and easier to implement with cucumber in specific. I will assume the reader has some basic knowledge with BDD, Acceptance Criteria, Specification by Example or Cucumber:



1- Minimize global context manipulation by avoiding unnecessary step division



Look at the scenario below:

Scenario: Register a new user

Given a new User named José

And located at Fool Street n:0

And with 20 years old

When I register the new user

Then the registration is done successfully

See how the configuration of the new user is divided by three steps? The first one defines its name, the second one where it lives and finally the last define its age. Aside from been very verbose, it creates a difficult situation for developers since they must keep track of the user configuration between three different methods, one for each step. If different scenarios are used to configure a single entity (user, for instance) the developers are forced to control global variables change on different places, which gets out of control very fast. It is better to do like this:

Scenario: Register a new user

Given a new User with:

| name | address | age |

| José | Fool Street n:0 | 20 years |

When I register the new user

Then the registration is done sucessfully

Or, if you have a lot of attributes:

Scenario: Register a new user

Given a new User with:

| Information | Value |

| name | José |

| address | Fool Street n:0 |

| age | 20 years |

When I register the new user

Then the registration is done successfully

By doing this the number of steps is decreased, therefore reducing the scenario size and complexity, and making the scenario more readable and easier to implement, since now the whole configuration of a User is done inside one single method.



2- Reduce Scenario size and complexity



It is easy to fall into the trap of filling a scenario with a bunch of context when it is not really necessary. Suppose that a Contract has three statuses: Draft, Signed and Finished. To sign a contract, its value and validity period must be provided and it must go through an approval workflow, but its only prerequisite to be Finished is to have been Signed. Given these requirements, one could write a scenario like this:

Scenario: Finish a Contract

Given a contract with:

| Information | Value |

| Value | $1000000 |

| Validity Period | 01/01/2014–03/01/2014 |

And was approved by Ana

And that is already Signed

When I Finish this contract

Then the Contract is correctly finished

Is all provided information really necessary? Isn’t the Scenario below enough?

Scenario: Finish a Contract

Given a Signed contract

When I Finish this contract

Then the Contract is correctly finished

There is no need to provide all the information that is necessary to sign a contract, just say the contract is already signed. When writing unit tests you must provide every information so that the execution can perform, but for Acceptance Tests you only have to provide the information needed to understand and validate what is being tested, you may leave the details of how things should be created to developers. So, in the above scenario, a developer will provide a StepDef named “a Signed contract” that creates a signed contract with whatever information it needs.



As a rule of thumb, I try to restrict to seven the amount of context someone must hold to understand the scenario. So the sum of steps, elements and parameters used in a scenario cannot be higher than seven. That is so because people, in general, can only hold around seven contexts at a time (*). So, the first scenario has 5 steps and 4 informations: Value, Validity Period, Approved by Ana and Contract Signed; totalizing 9 contexts, whereas the second scenario has 3 steps and 1 information: Contract Signed; totalizing 4 contexts, well below the limit.



In Cucumber, the examples section do not count, because each line of example corresponds to one scenario and can be understood separately from each other, but if you have a list of examples higher than seven, you should separate it in different blocks each with a description of what are those examples testing.



3- Never configure context after When steps



Suppose a scenario like this:

Scenario: Change user

Given an already registered user

When I change its marital status to Married

and its address to Fool Street n:1

Then the user is correctly changed

Despite the unnecessary step division, the way it is written forces the developer to put the user registration in the “its address to Fool Street n:1” step. When reading this step it is not possible to tell whether it actually permforms any action because it talks only about contextualization. It would be better to put the action on the “I change its marital status to Married” step, since it is clear that an action is being performed, but it cannot be done that way because part of the action will be configured in the next step. If the action is put on the “its address to Fool Street n:1” step, someone might try to reuse it without knowing that an action is beeing performed and eventually not being able to reuse at all just because the step should not execute an action in their case, just like in this example:



Given a new user with its name set to Alfred

And its address to Fool Street n:1

When I register the new user

Then the new user was registered sucessfully.



In that case the “its address to Fool Street n:1” should not perform any action.



The best thing is to do every contextualization before the when step and create a single atomic when step. If two actions are required, you might be doing too many things in a single scenario and you should check if it would be better to break it into two or more. If contextualization is needed in the when step, then use atomic step. Examples:

Scenario: Update user

Given an already registered user changed as follows:

| Information | New Value |

| marital status | Married |

| address | Fool Street n:1 |

When I update the user

Then the user is correctly updated

or

Scenario: Change user

Given an already registered user

When I update the user as follows:

| Information | New Value |

| marital status | Married |

| address | Fool Street n:1 |

Then the user is correctly updated

Another option is to change the steps a little from one scenario to another and avoid step reuse, so you can use the action for one and just contextualize on the other. You should choose what is better, to reuse or to facilitate implementation. I think that what is proposed by this recommendation maintain a good balance between scenario redability reuse and easy of implemention.





4- Do not create general steps for specific situations



Consider these two scenarios:

Scenario: Register a new user

Given a new User named José

And located at Fool Street n:0

And is 20 years old

When I register the new user

Then the registration is done successfully



Scenario: Register a new building

Given a new building named The Great Marfin Tower

And located at Isengard

When I register the new building

Then the registration is done successfully

Now, does the expression “located at” refers to Users or Buildings? How do I, as a developer, control this? Should I put an “if” condition that checks whether a Building or User is being configured, then update the proper one? This is clearly not a good idea, imagine if I had tree or five entities to choose now, how complex it would get? It turns out to become VERY complex!



The same is true for the Then step, it is used for both user and building registration, and it forces the developer to create complex logic inside the stepdefs to discover for which scenario the step is being used.



Lets see another even more problematic example:

Scenario: Register a new user under 18 is forbidden

Given a new 17 years old user

When I register the user

Then the user registration fails

Now, the registration fails because the user is under 18, but how would the developer test this? How does he know that in the “the registration fail” step failed because the user is under 18? Well, he could just see that the registration failed without validating why, but that is not good because the registration may fail by other reason and the test would still pass. One option is look at the contextualization, find a problem with it and expect the error later. So by looking at the configured user in the above scenario I can see he is 17 years old, and then infer that the error expected is that users cannot be under 18.



To do that, however, I will have to use business rules validations on the stepdefs to infer the error, most likely many duplication will take place and I cannot guarantee that what was inferred is actually the error expected. Besides that, the complexity of the stepdefs sky rocket as more and more validations are made. Instead, it is simpler to be more specific in the scenario definition, which usually makes it more readable too. A better example would be:

Scenario: Register a new user under 18 is forbidden

Given a new 17 years old user

When I register the user

Then user registration fails: users younger than

18 are forbidden

Now it is easy to implement the validation. I know exactly what I am verifying (age) and that it refers to users below 18 years old.



You can have generic step reuse when you actually can use it in the exact same way at different situations. An example of a good general step is “today is “. It can be used in many scenarios the way it is, because it actually defines some very global behavior. You can use it to contextualize any functionality that uses the current date with an step like “And today is 03/05/2014” or with “Given today is 12/25/2014” or you can use it to verify that a schedule tasks supposed to run at a given day is executing correctly:

Given an open task with end date for 16/07/2014

And today is 17/07/2014

Then the task is marked as Late

5- Business rules only



Writing and maintaining good acceptance tests by example and transforming them in useful executable specification demands considerable effort, but it also provides great return. It is easier to:



1- Conduct the user through a user requirements gathering session;

2- Agree on a common language so that everybody communicates by the same terms, facilitating the whole process;

3- Spot problems in the understanding;

4- Find breaches in the specification;

5- Confirm whether a modification in one place of the system would break an unpredicted specification;

6- and probably more.



I can see those advantages while creating specifications for business rules, but when infrastructure is mixed, such as information from the web page, the database or the mechanism of communication, I think it get things more confusing and harder to get to those advantages without incurring in high costs.



Infrastructure is just a way to automate a business process, so you choose a RDBMS with a Website providing some B2B service through a RESTful API in order to provide a way for that process to work in a automated way that is both economic and useful. But before that you must understand what you are going to automate, and I think this process of understanding is what Specification by Example and tools like Cucumber are most usefull, since they have been tailored to tackle exactly this kind of problem. Lets see an example:

Scenario: Cancel a Task

Given the user select task “TSK-01” from the combo box

And today is 17/07/2014

And the requestor is João

And João email is joao1234567@company.com

When I click on the Cancel button

And I fill the justification text box

And confirm the action

Then the task “TSK-01” is cancelled with 17/07/2014

cancelation date

And and the service mail.company.com is called with

the sendmail POST HTTP method to joao1234567@company.com

And the email header and body are, respectively,

“Task Cancelled” and “Task TSK-01 was cancelled”

It is exaggerated, but I did so to make the point. See how the definition of what should be made is bloated with other minor things? What my system should do, at minimal? Before beeing automated, what the process did? Let’s rewrite the Scenario considering only the business rules:

Scenario: Cancel a Task

Given the task “TSK-01” made by João

And today is 17/07/2014

When I cancel this task with a proper justification

Then the task “TSK-01” is cancelled with 17/07/2014

cancelation date

And a notification is sent to João informing the

cancelation

Isn’t that clearer? So what if it send an email, a SMS or smoke signal? So what if there is a combo box or a list box? Why it is important to know that a POST is used instead of a PUT? What the process do is cancel something and notify someone, period. Those other things get in the way of understanding the business while imposing some considerable cost of development.





So to get the most from your Executable Specifications, stick to the business rules definition. You sure can use cucumber and specification by example to describe, for example, the behavior of a screen, the contract of a RESTful API or the description of the size limit of strings stored in RDBMS, but I find that very inefficient. Usually those things are documentation from developers to developers, and so technical documentation, even code, are better options than acceptance tests. An ER Diagram and a good commented Resfull API, for example, are much better tools to communicate decisions for data storage and Restfull services than acceptance tests. Even when the description of an infraestructure matters to the client, such as the describing of the screen behavior, I think a good prototype made with a wireframe tool does a much better job of communicating by much less expense than a written acceptance test. Maybe one exception is the specification of some non-functional requirements, but I don’t know since I didn’t experimented much on this line.





If, however, you decide to specify infraestructure with acceptance tests, at least separate concerns by doing scenarios specifically to describe business rules and other to wharever other end.



*- “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, by George A. Miller (The Psychological Review, 1956, vol. 63, pp.81–97).”