scipy/scipy — pull request #3370 by argriffing (2014-02-22 00:15:40+00:00)
Title: ENH: add two new ufuncs to scipy.special

These ufuncs `entr` and `kl_div` are useful building blocks for information theory, statistics, and convex optimization.

Here are some possible concerns or negative first impressions about these functions, with some comments.

One concern could be the weirdly abbreviated function names. In general I prefer more verbose function names, but I've chosen these for a couple of reasons. Firstly, they fit in well with the rest of the cryptically short `scipy.special` function names, and secondly they are standard atomic elementwise function names in convex optimization ([cvx](http://cvxr.com/cvx/), [cvxpy](https://github.com/cvxgrp/cvxpy), [dcp functions](http://dcp.stanford.edu/functions)). Also the weird function names could help indicate the fact that they are lower level than actual `entropy` or `kl_divergence` functions -- the higher level functions of finite distributions are reductions (sums) of the elementwise ufuncs.

Another possible concern would be redundancy with existing ufuncs like `xlogy`. I think they are different enough. The `entr` is a unary instead of a binary function, and the `kl_div` shows its usefulness in this PR in the scipy code that computes kl-divergence.

The `kl_div` may appear to be defined in an unnecessarily complicated way, with the extra `- x + y`. But this definition has the convenient property of being non-negative everywhere, and positive when `x != y`.