Bugfix for RTSP. Segfault after unsuccessful connect

Even if connect failed we still return 0 which is wrong.
Instead we should return same negative value which we got from "avformat_open_input".