Repository: scipy/scipy
Pull request: MAINT: static typing of entropy
DOC: fix return type in entropy

maintenance: Add type hints for entropy function.
doc: return type of entropy is an ndarray if the argument pk isn't 1d.
This wasn't reflected in documentation, and was therefore misleading.
Additionally, the documentation now specifies pk and qk arguments as
array_like and not as sequences.
See #14680

Closes #14680

<!-- 
Thanks for contributing a pull request! Please ensure that
your PR satisfies the checklist before submitting:
http://scipy.github.io/devdocs/dev/contributor/development_workflow.html#checklist-before-submitting-a-pr

Also, please name and describe your PR as you would write a
commit message:
https://docs.scipy.org/doc/numpy/dev/development_workflow.html#writing-the-commit-message

Note that we are a team of volunteers; we appreciate your
patience during the review process.

Again, thanks for contributing!
-->

#### Reference issue
<!--Example: Closes gh-WXYZ.-->

#### What does this implement/fix?
<!--Please explain your changes.-->

#### Additional information
<!--Any additional information you think is important.-->

Problem statement:
BUG:  misleading documentation in scipy.stats.entropy 
### Reproducing Code Example

The entropy function allows calculation of entropy along an axis for multi-variate distributions. It works as expected, and returns an ndarray as a result. However in the documentation, the return value is specified as a float (which is indeed returned for univariate distributions).

Thus the problem is in the documentation, and not the implementation itself.
Additionally, as a consequence the IDE flags this as a type mismatch when type hinting is used as it assumes the function really return a float and not an array.

To reproduce:

```python
from scipy.stats import entropy
import numpy as np
pk = np.array([[2/3, 1/3],[1/3, 2/3]])
e = entropy(pk)
```
```


### SciPy/NumPy/Python version information

`1.7.1 1.21.2 sys.version_info(major=3, minor=9, micro=6, releaselevel='final', serial=0)`

Hints from issue discussion:
Thanks @YairMZ. This function is not yet typed and the IDE is inferring from the doc. Would you like to make a PR? (At least to fix the doc)
@tupui Thanks for the quick response.
I'd love to help and make a PR.
The truth is however, that my coding experience has been mostly on my my own projects, and therefore I don't really know what exactly do I need to do.
If you could direct me, I'd be happy to help.
In this case, we need to improve the documentation to have a correct return type (could probably just be `{float, array_like}`). This would render in an IDE as `Any` (because of `array_like`) and mitigate the IDE complain. To go further, we would need to type the function. In this file there is another function which is typed, we just need to take some inspiration from there.
Sounds great.
For clarity, in order to do this I first need to clone the repo and set up a development environment as explained here:
http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#contributor-toc

Am I correct?
Yes, here is the typical workflow https://scipy.github.io/devdocs/dev/contributor/development_workflow.html#development-workflow. It's a long document as it covers a lot of things. In this case, don't stress yourself with building and running tests locally as it will just be documentation/typing changes.


Some optional notes. For the commit message, it would be super if you could add at the end of the message `[skip azp]`. For example

If you just touch the doc
```
DOC: return type in entropy

[skip azp] [skip actions]
```

and if you touch the typing itself

```
MAINT: static typing of entropy

[skip azp]
```

This would skip non relevant part of the CI.
In addition to the return type, some of the parameter types would also need to be updated from "sequence" to "array_like", and descriptions may need to be generalized to make sense for ndarray input.
@mdhaber Does "array_like" suit this case? The function does handle correctly (as shown in the doc) also standard python lists? Does "array_like" refer to these too? Or should it be: {array_like, sequence}?

Currently I copied the the static typing from the differential_entropy function is was ust about to commit.

Commit message: MAINT: static typing of entropy
DOC: fix return type in entropy

maintenance: Add type hints for entropy function.
doc: return type of entropy is an ndarray if the argument pk isn't 1d.
This wasn't reflected in documentation, and was therefore misleading.
Additionally, the documentation now specifies pk and qk arguments as
array_like and not as sequences.
See #14680

Patch to review:
diff --git a/scipy/stats/_entropy.py b/scipy/stats/_entropy.py
index 1cb5a1104096..f51656e603a2 100644
--- a/scipy/stats/_entropy.py
+++ b/scipy/stats/_entropy.py
@@ -14,7 +14,11 @@
 __all__ = ['entropy', 'differential_entropy']
 
 
-def entropy(pk, qk=None, base=None, axis=0):
+def entropy(pk: np.typing.ArrayLike,
+            qk: Optional[np.typing.ArrayLike] = None,
+            base: Optional[float] = None,
+            axis: int = 0
+            ) -> Union[np.number, np.ndarray]:
     """Calculate the entropy of a distribution for given probability values.
 
     If only probabilities `pk` are given, the entropy is calculated as
@@ -27,10 +31,10 @@ def entropy(pk, qk=None, base=None, axis=0):
 
     Parameters
     ----------
-    pk : sequence
+    pk : array_like
         Defines the (discrete) distribution. ``pk[i]`` is the (possibly
         unnormalized) probability of event ``i``.
-    qk : sequence, optional
+    qk : array_like, optional
         Sequence against which the relative entropy is computed. Should be in
         the same format as `pk`.
     base : float, optional
@@ -40,7 +44,7 @@ def entropy(pk, qk=None, base=None, axis=0):
 
     Returns
     -------
-    S : float
+    S : {float, array_like}
         The calculated entropy.
 
     Examples


Code review comments:
diff_hunk: @@ -27,10 +31,10 @@ def entropy(pk, qk=None, base=None, axis=0):
 
     Parameters
     ----------
-    pk : sequence
+    pk : array_like
         Defines the (discrete) distribution. ``pk[i]`` is the (possibly
path: scipy/stats/_entropy.py
text: @user1:
This is the description that should probably be generalized. To avoid potentially unfamiliar NumPy notation for ndarrays, maybe something like 
```
Along each axis-slice of ``pk``, element ``i`` is the  (possibly unnormalized) probability of event ``i``.
```
(with line breaks as needed).

(gh-14680 mentions 
> The entropy function allows calculation of entropy along an axis for multi-variate distributions.

but this is not the only way of thinking of the use of `entropy` on an nd array. For example, each row of a 2d-array could represent a separate univariate distribution.)

@author:
Mathematically speaking, I think we must consider this (for the case of a 2d array), as a set of univariate RVs. If the distribution is really multivariate then each column (or row) is a realization and not each element on its own.

Can I modify somehow this PR t improve the the description according to your suggestion? 

@user1:
Yes. Just push another commit to your branch.