libgit2/libgit2 — pull request #2676 by carlosmn (2014-10-31 10:28:22+00:00)
Title: Threading and crypto libraries

Branching off of https://github.com/libgit2/libgit2/pull/2637#issuecomment-60788579

Some of the more salient points:
* We initialise OpenSSL's threading unconditionally
* We may be overwriting the OpenSSL threading setup of the application
* We do not initialise libgcrypt threading
* We may be using libgcrypt (via libssh2) in a threaded way

The library *cannot know* whether we should set up the OpenSSL threading and/or libgcypt's. Libssh2 and libcurl and seemingly everything else kick the responsibility up to the application, as the human or process building it is the only one who can know what combination of crypto libraries there are.

The current situation isn't so dire in practice due to a combination of single-threaded languages and better crypto in the others, but as we e.g. cannot use stock libssh2 multi-threaded safely on Debian.

We can work around this (and others) issue partly my moving away from OpenSSL, which would be something I'd love to do, but that has its own problems. So we need to kick the responsibility for locking up the chain too. As long as we link directly to OpenSSL, we can provide a couple of functions that will set up the locking which an application author can use if they don't otherwise set up the locking (or if libgit2 is alive for the entire application run, you might as well take advantage).

But we have no connection to libgcrypt, so the user is going to have to set that up themselves (luckily libgcrypt does provide defaults for pthreads).

We need to document the differences between platforms in this regard.

And we should look into what alternatives there are. On OSX there's SecureTransport which should be able to replace OpenSSL for us, and on any unix we have GnuTLS, which can also take the place of OpenSSL and depends on nettle, which is thread-safe (but has issues with some ciphers when talking to Windows). We should eventually have a libcurl transport, which would abstract away the SecureTransport/GnuTLS/OpenSSL linking for us on the different platforms (but we'll likely still want our own HTTP code to reduce dependencies).

That would definitly kick the threading issue to the user, as we wouldn't have any idea what we're using for crypto. Then there's libssh2, which would still link against OpenSSL or libgcrypt, which has this exact same issue of the crypto being two layers away.