framework/tests/summary.py: Tests Summary.py's assignment of statues

This code tests the way Summary.py assigns statuses to tests.
Specifically it tests regressions, fixes, changes, and problems,
assuring that only the correct permutations of these tests are added to
the list.

It uses python's builtin unittest framework, and a magical decorator to
limit code duplication.

This already reveals ~20 problems in the current implementation

v2: - Move framework_tests to framework/tests
    - Spelling corrections
v3: - More spelling corrections
    - Address a number of concerns from Ken Graunke
    - Make exceptions more robust by using the ``except <error> as''
      syntax
    - Refactor framework tests to reduce code duplication
    - Use python's tempfile module to get the system dependent temporary
      file location instead of hard-coding /tmp
    - Add test for the skip list

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
piglit-framework-tests.py: Adds a utility for running piglit unit tests

This program gives the option to run the tests. These tests are meant to
test internal features of the piglit framework. Currently there are only
tests for the summary module's handling of regressions, fixes, changes,
and problems.

Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
framework/summary.py: Fix problems with status handling

Fixes status handling (fixes, regressions, changes) to be compliant with
the status handling in framework_tests/summary.py. Mainly this addresses
notrun -> * and * -> notrun, as well as a few corner cases.

v2: - Spelling corrections
v3: - More spelling corrections
    - fix skipped page generation

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
status.py: Adds a new status class and subclasses

These status classes are intended to replace the string statuses
currently used by piglit summary.

They give a couple of very nice advantages, first, they can be directly
compared to each other to determine whether a status is worse than
another status. These statuses can also be compared to any object
implementing the __int__ magic method.

This will allow us to remove a number of special functions for sorting
representations of statuses.

v2: - Use larger numbers for the statuses, this should make adding
      additional statuses easier, since there is room between the
      statuses.
    - Add a class for the "Not Run" status
    - Adds classes for dmesg-warn and dmesg-fail
    - Change statuses to match the standard set in the framework test
    - Spelling corrections
v3: - add a docstring showing status order
    - correct a couple of comments
    - Adds a function for converting a string into a status

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
status: Make use of the status objects

This adds code in framework/summary.py and framework/core.py to make use
of the status classes in status.py. This makes comparisons between
statuses much simpler and cleaner

v2: - Instead of importing all of the classes into the local namespace
      use the ``import ... as'' syntax. This means that if additional
      statuses are added they can easily be used without updating
      imports
    - Correct the sorting for fixes and regressions
    - Fix bug in TestResult that caused Fail, Warn, and Crash to be
      assigned the wrong status Objects
    - Adds support for dmesg-warn and dmesg-fail
v3: - Use the status.status_lookup() function
    - Replace use of map with list comprehension

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
summary: Remove optimization with no impact

Generating these lists is rather trivial, so just generate them all the
time.

v2: - Fix html page generation
v3: - Stop framework summary tests from trying to generate these lists

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
summary: Fold __generate_lists into __init__

Since there is nothing special about __generate_lists() anymore (it
always generates the same set of lists when called), and we need to call
it in every instance just fold it back into the constructor, so that it
no longer need to be explicitly called.

v2: - Address some spelling problems in the commit message
v3: - More spelling isues

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>