retrace: Put a ceiling into the total size of bound blobs.

Some OpenGL applications that use vertex arrays in user memory were
exausting all memory with bound buffers.  This change enables retrace to
complete.

Merit for the idea goes to idinev, in issue #222.  This implemetation
limits total size of bound blobs, instead of total number.