vogleditor: add nested operations check

A check is added at the "glEnd" entrypoint to ensure it doesn't get
moved past the parent frame in the case of an unpaired glEnd(). This
can inadvertently occur in a program and will cause a crash. (Similar
defect in apitrace)

Signed-off-by: Lawrence L Love <lawrencex.l.love@intel.com>