renderer: Fix seeking behavior when using REL_TIME

Signed-off-by: Milan Plzik <milan.plzik@streamunlimited.com>

https://bugzilla.gnome.org/show_bug.cgi?id=686628
renderer: Use proper error-code on wrong seek

We were using 710 (Illegal seek mode) instead of the correct 711 (Illegal
seek target)

This fixes 7.3.157.9

https://bugzilla.gnome.org/show_bug.cgi?id=709077
renderer-gst: Check seek target

The seek call on playbin doesn't fail when seeking beyond the end of the file
and just switches to EOS.

https://bugzilla.gnome.org/show_bug.cgi?id=709077