vdpau: correctly mark invalid mixer as such on vdp_video_mixer_create() failure

Otherwise vdp_video_mixer_destroy() would later fail when called on an invalid
video mixer handle. With mesa r600 vdpau driver, this would cause a segfault.

This may just be a mesa thing idk, but e.g. vlc does basically the same thing. Now mpv doesn't segfaults anymore, but it keeps showing "Error when calling vdp_video_mixer_create: An invalid/unsupported value was supplied" with no video displayed (but the audio works). Should it just die or is this ok?