Vaapivdpau: fix and optimization

this is based on #5370, only the 3 topmost commit belong to this pr.
vaapi and vdpau feel better with more than 2 render buffers.