librtmp: switch to dev version and enable shared

fixes #566