report error when preparing for non-anon TLS and this is unsupported

The error is reported when the processes own certificate is set,
which also turns off anon TLS. We now check if TLS AUTH is supported
and, if not, return an error. This enables better error tracking
in the client, as otherwise the problem is reported much later
or even not at all (in the case of a relp server).