I disagree. 

  * If this invariant disappears, this means that libsodium broke public API while letting old code to link to it. This would be a major faux pas and should never ever happen; if it does, *all* downstream software will break in a dangerous way.
  * If it does not, the user is left to care about a publicly visible exception which would never happen, cannot be tested and will be always ignored (because of the above). I can't say that this is good API design.

(Tangentially, I don't think I ever seen OCaml projects that disable assertions. But of course this is just a personal anecdote.)