Set stencil ref based on framebuffer format