Fix building on linux with newer librtmp nr2

this fixes rtmp on linux as well as building with newer librtmp versions.
jenkins builds: http://jenkins.xbmc.org/view/Helpers/job/XBMC-BuildMulti-All/156/
also briefly tested the resulting android build.

fyi: the original issue was a missing XB_FIND_SONAME([RTMP]