tobie/ua-parser — pull request #404 by atifaziz (2014-05-28 17:32:33+00:00)
Title: Generate C# tests from YAML using T4 templates

This pull request moves unit test from being data-driven at run-time to generating individual unit test methods from the YAML test cases using T4 text templates. The benefits are several fold:

- Each test case appears as its own
- When a test fails it can be easily identified
- Using various runners, individual cases can be run and debugged
- Test cases can be timed individually, e.g. ReSharper's runner already does this

…and the following image will speak a thousand words:

![tests](https://cloud.githubusercontent.com/assets/20511/3107836/e7d23ede-e689-11e3-89e2-d796e30c588e.png)

The organization is as follows, which was done for performance & manageability reasons:

- All generated tests appear under the `UAParser.Tests.Generated` namespace
- Each YAML becomes a namespace
- Each family (be that user agent or OS) of tests becomes a class
- Each test case becomes a method with the method name created from expectations
- The suffix of each test case method name is the line number of the source YAML document from which the method was generated.

The only exception to the above is `test_device.yaml`, which was rather small so it has a simpler organization. All its test cases appear as methods of `DeviceTests`.

The only downside of this approach is that it does push the limits on some tools. For example:

- `UserAgentStringsTests.cs` is over 112K lines of code so code analysis tools like ReSharper can have trouble keeping up.
- With over 13K individual test cases, the ReSharper unit tests runner did hang occasionally. Reorganizing test cases within namespaces and families did help tremendously.
- Xunit GUI runner takes forever to initialize and build list of test cases but console runner runs with flying colors.
- T4 template generator did start running out of memory & VS had to be restarted but this happened only in one session.

I think the above problems are an issue if you're hacking & refactoring the templates & generating the test cases code in quick bursts but shouldn't be a bother otherwise.