alphagov/smart-answers — pull request #1142 by camilleldn (2014-09-23 11:29:52+00:00)
Title: SPL: scenario generation, human-readable factchecking, snippet insertion, mock plugins

###Scenario generation

Tool for shared parental leave to generate all possible scenarios for use in future continuous integration tests.
The tool:
- goes through predefined list of possible answers to each possible question in the flow
- computes all possible combinations of those answers through the flow
- works out the outcome for each of those combinations

We then write down those scenarios in files named by the name of the outcome they lead to.

### Human-readable factchecking

/employee-parental-leave/factcheck

That page now displays a table of results that looks like, for both adoption and birth, in all cases as listed below (continuity of employment test, earnings and employment test): this is to facilitate factchecking with lawyers.

Due date | Mother status | M: Continuity | M: E&E | Partner status | P: Continuity | P: E&E | Outcome
-|-
05/04/2015 |self-employed | |✔ | | | |- Maternity allowance
05/04/2015 |employee |✔ |✘ |employee |✔ |✘ |- Maternity leave
- Maternity pay
- Mother shared parental leave
- Mother shared parental pay
- Paternity pay
- Paternity leave
- Partner shared parental leave
- Partner shared parental pay

### Snippet insertion

Insert all the snippets written by the content team, and rename the outcome names to match new naming scheme. We had to change the flow and add snippets in some instances:
- a number of snippets that were being used both for birth and adoption for example were using "due_date" so had to be separated
- a number of snippets for maternity pay, paternity pay etc... were relying on leave dates, which weren't being asked in the case where users are entitled to pay not leave (workers, agency workers, etc...) so the flows had to be changed so that we now require leave dates to be entered
- more small issues along those lines...

### Mock plugins

Due to the recent upgrade of Smartdown and implemenation of plugins, the flow was failing in the absence of definition of all the plugins used in the snippets so we had to had mock plugins in order to be able to navigate through the flow.

### Removing execution errors

Due to all of the above, the process of correcting those errors was through running the generate_scenarios task repeatedly until all the possible options of the flow had successfully been run without warnings.

### Performance issue

Due to the increased size of the flow now we have snippets, plugins and all that good stuff, we ran into a performance issue: the Smartdown logic was taking so long we were getting timeouts systematically. We commented out one of the preload_flow methods in the Smartdown Registry that is not actually needed given our current way of creating and registering flows. Performance issues with how responsive Smartdown is given the size of this flow still need to be looked at quite **urgently**.

### After this...

There will be still errors in the flow about the legal outcomes of those questions: fact-checking needs to be done now that the flows have had to be significantly changed to fit snippets, plugins etc...
We still need to implement all the plugins :-D
Some performance fun.