Two things- one, as a sibling comment's mentioned, it absolutely includes implementation of cryptographic primitives too. There are quite a few subtle bugs (mostly, but not entirely, side-channels) that end up being utterly catastrophic.

Second - even using somebody else's (high-quality, trusted) implementation of (high-quality, trusted) primitives very much isn't enough to build a secure system that uses cryptography. The obvious example here is everybody and their dog going off to implement AES or something, not using a MAC ("we just care about secrecy, not integrity") or using a MAC wrong (e.g. MAC-then-encrypt), and then dying horribly to a trivial padding oracle.

Or, trying to build a secure transport protocol (i.e. a TLS-like API, where you just get "a secure socket" after doing some dance with certificates/keypairs)... you just encrypt (and MAC, this time) all the data before you send it out, and decrypt (and verify!) all data that comes in. But... wait, our API can't really handle that easily - we have to write the data out to the network in chunks. So, hm, encrypt and MAC each chunk? Oh, then chunks can be rearranged in transit, so we'll put a counter in there. What if the counter wraps around? Do you abort, or do you just reuse old counter values? Do you get a different session key for the same (client, server) pair- if not, is that an issue? Suppose you're using an AEAD mode, like GCM- how do you manage nonces? (If you reuse a nonce once in GCM, you often leak your authentication keys(!))

Hm, all of that sounds like ... a lot. So maybe we'll just sign+encrypt requests (and include our public key inside), and have the server sign+encrypt responses (to the public key we sent). Hm, but we need to tie responses to requests, though. So we'll need to put a hash of the request in the response. Ah, hmm, length extension attacks, right. Maybe GCM will save us? Hm, not sure... What if somebody wants to replay old content to us (send us an older version of a page)? Can they do that? Right, need to either include some "challenge"/nonce in requests, or at least make sure session keys are unique per-request. What about possible reflection attacks - if we send the server its own public key as our public key, does that cause any weirdness? Oh, what is a public key, anyways? If (God forbid) you're using RSA, does that include both the exponent and the modulus, or just the modulus? If you do a key exchange (e.g. for forward secrecy), who picks the parameters- are they just fixed?

Cryptography is the sort of thing that almost actively resists abstraction, and it's really tricky in a way that's hard to appreciate. It's a world where you find a claw hammer and use it to remove a nail from your floor- maybe even looking on CarpentryOverflow first to make sure a claw hammer can be used to remove nails from floors- and it works fine, so you then go to remove a nail from your wall and discover that this makes the hammer burst into flames because, yes, most people think "claw hammers can pull nails out of stuff" and that's usually true but the unstated assumption there is that you're holding the hammer with your right hand and you're actually left-handed and it's a Tuesday in the southern hemisphere so you actually should have used a completely different tool or used a higher-level misuse-resistant nail-pulling API which does almost exactly what you want, but that's what you thought the hammer did so...