Only link with openssl stuff if it's an openssl build....

The way I do this is still rather ugly, but then again the ./configure
crap is ugly as well imo.

Signed-off-by: Toon Schoenmakers <nighteyes1993@gmail.com>