rtmp: Do not misuse memcmp

CC: libav-stable@libav.org