lavf/movenc: check ff_mov_init_hinting() return

Fixes a crash when the codec stream is not supported by the rtp muxer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>