Repository: pytorch/ignite

Issue:
[CI] Doc test is failing
We have somehow doctest ci job failing right now. The failure happens with the following code snippet from our docs: 
- https://pytorch.org/ignite/generated/ignite.contrib.metrics.PrecisionRecallCurve.html

```
 **********************************************************************
File "../../ignite/contrib/metrics/precision_recall_curve.py", line ?, in default
Failed example:
    y_pred = torch.tensor([0.0474, 0.5987, 0.7109, 0.9997])
    y_true = torch.tensor([0, 0, 1, 1])
    prec_recall_curve = PrecisionRecallCurve()
    prec_recall_curve.attach(default_evaluator, 'prec_recall_curve')
    state = default_evaluator.run([[y_pred, y_true]])

    print("Precision", [round(i, 4) for i in state.metrics['prec_recall_curve'][0].tolist()])
    print("Recall", [round(i, 4) for i in state.metrics['prec_recall_curve'][1].tolist()])
    print("Thresholds", [round(i, 4) for i in state.metrics['prec_recall_curve'][2].tolist()])
Expected:
    Precision [1.0, 1.0, 1.0]
    Recall [1.0, 0.5, 0.0]
    Thresholds [0.7109, 0.9997]
Got:
    Precision [0.5, 0.6667, 1.0, 1.0, 1.0]
    Recall [1.0, 1.0, 1.0, 0.5, 0.0]
    Thresholds [0.0474, 0.5987, 0.7109, 0.9997]
```
- https://github.com/pytorch/ignite/actions/runs/4099985910/jobs/7074343114

### How to help with this issue

You need to do some detective work:
- Reproduce the issue locally
- Try to figure out which result is correct: "Expected" or "Got"
- Try to figure out why it started to happen: maybe sklearn version updated ? Previously, for example Jan 18, doctest was passing: https://github.com/pytorch/ignite/actions/runs/3894024421/jobs/6647420435
- Report here your findings and propose a way to solve the issue



Discussion:
Hi @vfdev-5,

Here are my findings :

- Expected result is correct
- It started happening when scipy version got upgraded from 1.7.3 to 1.10.0.

The proposed way to solve:

1. Since it's a dependency on an external library we can configure in the requirements-dev.txt file of this project to use scipy version 1.7.3.
@pacificdragon thanks for the report ! 

> It started happening when scipy version got upgraded from 1.7.3 to 1.10.0.

I wonder how the chain goes from sklearn to scipy ? If you may have seen that `PrecisionRecallCurve` is just calling sklearn under the hood:
https://github.com/pytorch/ignite/blob/a720dfa7dee9dc0a8ad2a7323e6c33f101aca9c7/ignite/contrib/metrics/precision_recall_curve.py#L18

So, does it mean that sklearn has a bug which depends on scipy ?


Hi @vfdev-5 

- Reproduced the issue locally
- The "Got" result is Correct
- [Scikit Learn Precision Recall Curve](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_curve.html#sklearn.metrics.precision_recall_curve)
- [Source Code of Scikit Learn Precision Recall Curve](https://github.com/scikit-learn/scikit-learn/blob/7db5b6a98/sklearn/metrics/_ranking.py#L797)
- According to scikit learn, `n_thresholds = len(np.unique(y_pred))`
- scipy version 1.7.3 too results the same results as the "Got"
- Reverted Back to the commit from before 18th Jan and Using a new Virtual Env recreated the problem, gives the same results

Proposed way to solve - 

- The "Got" result is correct so we sould probably update the DocTest
@theory-in-progress thanks for your investigation !

> Reverted Back to the commit from before 18th Jan and Using a new Virtual Env recreated the problem, gives the same results

What result did you get ? Did you get a failure or test is passing ?

> The "Got" result is Correct

How do you know that ?

I still do not quite understand the main reason why the test has started to fail since Jan 18th. Can you give more details on that ?
I guess @pacificdragon is correct. Actually on 18 Jan, Sklearn moved itself scipy from 1.7.3 to 1.10.0 as the minimum support version . Also @theory-in-progress correctly pointed out as the change date to be 18 Jan
https://github.com/scikit-learn/scikit-learn/commit/d60af3de412cbd9f7504beb14350b942ae460703
@guptaaryan16 thanks as well for your findings !

Can you guys check the following: install scipy=1.7.3 and the latest scikit-learn and run our test example without using ignite but using pure scikit-learn code to compute precision recall curve. Next, install scipy=1.10 and rerun the same code. So, do we have the same results or not ? If results are the same then we can just update our test with respect of the expected output. If results are not the same, then there is an issue inside sklearn.

Please also post here the code and the outputs. Thanks !

Resolving patch:
diff --git a/ignite/contrib/metrics/precision_recall_curve.py b/ignite/contrib/metrics/precision_recall_curve.py
--- a/ignite/contrib/metrics/precision_recall_curve.py
+++ b/ignite/contrib/metrics/precision_recall_curve.py
@@ -65,9 +65,9 @@ def sigmoid_output_transform(output):
 
         .. testoutput::
 
-            Precision [1.0, 1.0, 1.0]
-            Recall [1.0, 0.5, 0.0]
-            Thresholds [0.7109, 0.9997]
+            Precision [0.5, 0.6667, 1.0, 1.0, 1.0]
+            Recall [1.0, 1.0, 1.0, 0.5, 0.0]
+            Thresholds [0.0474, 0.5987, 0.7109, 0.9997]
 
     """
 


Test patch: