framework/summary.py: Replace Section heading with docstrings

The sections headings are not really that useful, but they might as well
be kept as class level docstrings instead of being thrown away.

V2: - Correct spelling error

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
piglit-summary-html.py: Fix listfile syntax

This patch changes the listfile syntax to be newline separated. Further
it actually hooks up the plumbing allowing the -l/--list option to
actually work

V3: - Adds this patch

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
framework/summary.py: Adds new summary class and two support classes

This new summary class (currently NewSummary), is a complete redesign of
the summary generation system. Much of the code in summary was not
commented, and also not clear as to what it was doing, making it very
difficult to maintain and even more difficult to improve. The other
problem with the code was architecturally it was clobbered together,
doing some of the work in the summary.py file, and some of the work in
the piglit-summary-html.py file.

The new summary class does all of it's work in the summary.py file, and
from an API point of view, in one class. The summary class uses it's
constructor to build lists of passes, failures, skips, problems, etc.
and then uses methods to generate output. The reference implementation
of that kind of method is for generating HTML, but in theory it could be
used to generate anything: XML, csv lists, printed lists ala
piglit-summary, etc.

Also updates the .gitignore file to ignore some generated content
created by this class.

V3: - Corrects some spelling mistakes and some comments that had old
	  method/variable names
	- renames several methods/functions
    - moves sanitizePath to a toplevel function
	- changes sanitizePath to use lambda per ken
    - Moves buildDictionary to a helper function instead of private
	  method
V4: - Allows new summary generation to work with broken json.

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
templates: adds mako templates for new summary code

Adds the actual templates used to by the new mako based summary code.

V3: - Replace tabs with spaces like python code
    - Rename test_index.mako to testrun_info.mako
    - Refactor testrun_info.mako links bar generation

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
piglit-summary-py: Use the new summary class to generate HTML

Drops almost all of the code used in this file, all of that
functionality is now handled in the summary class. Instead, the majority
of this file is just the argparse instance, and a few short lines
calling the new summary class.

V2: - Fixes the -l/--list syntax
V3: - Changes the listfile syntax to match the newline separated format

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
templates: Remove old HTML templates

These template files are no longer used, and so they should be removed

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
templates/index.css: make some css more exclusive

Change the div:nth-child to operating on everything except head to only
working on group. This is lays groundwork for later changes.

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
summary: Build subdirectories for subtest result pages

With 10,000+ tests all living in the same folder it can be hard to
identify a single file, especially when trying to work with the HTML
generation itself. This patch changes the behavior so that each test
has a directory tree for the group results with tests under it (ie
spec/GLSL 1.0/foo/bar/test.html)

V3: - Correct some spelling errors/remove excess comments

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
test_result.mako: Drop redundant status and result field

In the old summary code one was set equal to the other, and then
written. In order to produce the same output they were both brought over
to the mako code. But they're rather silly and worthless and only one
should be kept.

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
HTML summary: Provides means to exclude generating test result files

This provides a switch and method for excluding the generation of the
test result HTML files (those that live under <testrun name>/). This
allows the user to trade some verbosity in the results for a significant
increase in generation speed. A run of quick.tests with all options
enabled takes ~5.5 seconds, with just pass and skip disabled it only
takes ~2.5 seconds. This only becomes more significant as more testsruns
are added the HTML page.

V3: - Remove some bad comments
    - adds 'skip' to exclude list
V4: - Renames --exclude-generation to --exclude-details

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
summary.py: Change skip to 'Not Run' for tests not run

When comparing two test results the current behavior sets a status of
skip for tests that were not run in one set of results. This is
confusing and silly, so instead give them a status of N/A which gives a
clear understanding of what skipped, and what simply didn't run either
by exclusion or because it didn't exist yet.

V3: - Change N/A to "Not Run"
V4: - Fixes "Not Run" to not be compared for fixes, regressions or
	  problems

Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com>
Reviewed-By: Aaron Watry <awatry@gmail.com>
Reviewed-By: Kenneth Graunke <kenneth@whitecape.org>
util/x11: Don't spin when waiting for X events

The nasty spin consumed 100% cpu when tests were ran in non-auto mode.
No behavioral change when running tests in auto mode.

This patch is semantically equivalent to:
    while (true) {
-       if (!XPending(dpy))
-           continue;
        XNextEvent(dpy, &event);
        handle_event(event);
    }

Reviewed-by: Paul Berry <stereotype441@gmail.com>
Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
sparse-samplers: Increase tolerance when sampling from linear filterd texture.

Different drivers give a wide range of results. Since the test is not about the
quality of linear filtering just increase the tolerance.
Set config.requires_displayed_window in tests that use front buffers.

When running tests with a non-composited window manager, it isn't safe
to access the front buffer until after the window is shown.  This
patch sets config.requires_displayed_window in tests that access front
buffers, to make sure that the piglit framework exposes the window,
even when testing in "-auto" mode.
util/x11: Wait for expose event in manual mode or with front buffer tests.

When running piglit in manual mode (no "-auto" command-line option is
given), there's no point in calling piglit_display() before the window
is exposed.  This just leads to confusion because it will be called
again once the window appears, leading to duplicate test output in the
console window.

When running piglit in "-auto" mode, if the test accesses the front
buffer, then we need to wait until the window is exposed before
calling piglit_display(), since a front buffer is not guaranteed to be
available before then (in practice, this only happens with
non-compositing window managers).  However, if the test doesn't access
the front buffer, then it is still safe to call piglit_display()
before the expose event; doing so avoids wasting time.

Note: there are two known bugs that cause sporadic failures in certain
piglit tests with Mesa/Intel hardware (and possibly other Mesa
drivers).  See bugs #1 and #2 in
http://lists.freedesktop.org/archives/mesa-dev/2013-May/039985.html.
This change seems to exacerbate those sporadic failures.  As far
as I'm aware, the tests that are affected are:

- spec/!OpenGL 1.1/drawbuffer-modes
- spec/EXT_framebuffer_blit/fbo-sys-blit
- spec/EXT_framebuffer_blit/fbo-sys-sub-blit

I have patches on the list to fix bug #1 from that email (see
http://lists.freedesktop.org/archives/mesa-dev/2013-May/040010.html),
and Eric Anholt is working on bug #2.  I believe that once those fixes
land, the above tests should pass consistently.

Based on work by Chad Versace.

Cc: Chad Versace <chad.versace@linux.intel.com>
Cc: Eric Anholt <eric@anholt.net>
Test a front buffer corner case: invalidate causes data loss.

Cc: Eric Anholt <eric@anholt.net>

v2: Move to spec/gl-1.0.  Add to all.tests.
Basic tests of fast color clear functionality on i965/gen7+.

Fast color clears are a hardware feature in i965/gen7+ which allows
the memory writes associated with clearing a color buffer to be
deferred until the buffer is later written to or read from; in most
cases this allows the clear writes to be write-combined with later
rendering operations.

For fast color clears to work correctly, the driver must maintain a
state machine for each buffer to indicate whether there are deferred
clears pending, and if so what the clear value was.  This patch
introduces simple shader_runner tests to validate corner cases of that
state machine.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Test reading from a buffer after fast clearing it.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Test that fast color clears work properly after a blit.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Test interaction between fast color clears and glXSwapBuffers.