avformat/utils: dvd still frames read thru libdvdnav ended up in internal lavf buffer

This is the solution we've been using in XBMC for over 2 years for dvd still frames.
The problem is that the demuxer asks for probing of the codec in the mpeg stream.
This causes lavf to read the whole menu structure into internal buffers.
After which, it won't read from input stream anymore and no events triggers.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avformat: use AVPROBE_SCORE_STREAM_RETRY, instead of AVPROBE_SCORE_RETRY - 1

This makes the code clearer

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>