ValveSoftware/vogl — pull request #178 by lawlove (2014-11-20 01:33:38+00:00)
Title: UI: Fix orphaned Debug marker and Nest crashes

Description:
Tested traces containing intentionally unpaired glPopDebugGroup or
glEnd apicalls (i.e. no corresponding glPushDebugGroup or glBegin
parent) and found they would crash if the State/Render groups checkbox
was checked on.

(Note that State/Render groups allow glBegin/glEnd nests)

Cause:
Reason is because when terminating a group/nest, a check is made to
ensure the parent group/nest apicall is a valid parent ("push" or
"glBegin"). However if the terminating apicall is an orphan, then
the parent tree item may not be an apicall but a (State/Render)
group (or frame) item and has no corresponding entrypoint_id_t. The
call to retrieve the parent apicall's entrypoint_id_t in that case
returns a VOGL_ENTRYPOINT_INVALID (-1) which is then used to index
into an array for the enumerated apicall, causing the crash.

Solution:
Modify the functions that take the entrypoint_id_t index as an input
argument to ensure it's a valid entrypoint_id_t before proceeding to
use it as an index.

Testing:
The following were used to induce the crash and subsequently verify
the fix:

debug_khr-xtra-popdebuggroup.bin
glxgears-unpaired-glend.bin

Testing of traces with unpaired push/pop apicalls did not cause
problems:

debug_khr-xtra-pushdebuggroup.bin
debug_khr-unpaired-popmatrix.bin

Note
We have the option to not inform the user of an unpaired debug marker
"pop" apicall if the "Hide terminating API call" checkbox is checked
but currently it is set to always display the message regardless of
that setting.