Actually, this is not true as the function can be discontinuous, so differentiating should give delta distributions. Which we can't represent.

Should it return NaNs? Or just plaster a warning in the documentation?