Repository: scverse/scanpy

Issue:
PAGA layout not reproducible
- [x] I have checked that this issue has not already been reported.
- [x] I have confirmed this bug exists on the latest version of scanpy.
- [ ] (optional) I have confirmed this bug exists on the master branch of scanpy.

---

I noticed that my UMAP projection was not quite the same each time I reran my notebook, so I systematically went through my pipeline to find the source(s) of this irreproducibility, one of which turned out to be `sc.pl.paga`.

Specifically, the positions (`adata.uns['paga']['pos']`) generated by this function were slightly different each time I ran the notebook, which then leads to the differences in the UMAP as I use `sc.tl.umap(adata, init_pos='paga')`.

Code:
```python
sc.tl.paga(adata)
sc.pl.paga(adata, color='degree_solid')

print(adata.uns['paga']['pos'])  # Slight differences in each run
```

I did some digging (=a lot of digging) and found that `scanpy.pl._tools.paga._compute_pos` uses igraph layouts, i.e. `pos_list = g.layout(...)`. Apparently, igraph uses python's built-in RNG, which has not been seeded.

I therefore did the following, which fixed my irreproducibility problem:

```python
import random
random.seed(0)

sc.tl.paga(adata)
sc.pl.paga(adata, color='degree_solid')

print(adata.uns['paga']['pos'])  # Exact same result each run
```

I think it would be nice if this could be included wherever igraph layouts are used to improve reproducibility.

I'm not sure, but perhaps issue #1418 occurs due to this problem.


#### Versions

<details>

-----
anndata     0.7.5
scanpy      1.7.2
sinfo       0.3.1
-----
PIL                 8.2.0
anndata             0.7.5
cairo               1.20.0
cffi                1.14.5
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.1
decorator           5.0.6
get_version         2.1
h5py                2.10.0
igraph              0.8.3
ipykernel           5.3.4
ipython_genutils    0.2.0
ipywidgets          7.6.3
joblib              1.0.1
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
matplotlib          3.3.4
mkl                 2.3.0
mpl_toolkits        NA
natsort             7.1.1
networkx            2.5
nt                  NA
ntsecuritycon       NA
numba               0.51.2
numexpr             2.7.3
numpy               1.20.1
packaging           20.9
pandas              1.2.4
pickleshare         0.7.5
pkg_resources       NA
prompt_toolkit      1.0.15
psutil              5.8.0
pycparser           2.20
pygments            2.8.1
pyparsing           2.4.7
pythoncom           NA
pytz                2021.1
pywintypes          NA
scanpy              1.7.2
scipy               1.6.2
setuptools_scm      NA
simplegeneric       NA
sinfo               0.3.1
six                 1.15.0
sklearn             0.23.2
sphinxcontrib       NA
statsmodels         0.12.0
storemagic          NA
tables              3.6.1
texttable           1.6.3
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
umap                0.4.6
wcwidth             0.2.5
win32api            NA
win32com            NA
win32security       NA
zipp                NA
zmq                 20.0.0
-----
IPython             5.8.0
jupyter_client      6.1.12
jupyter_core        4.7.1
notebook            6.3.0
-----
Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)]
Windows-10-10.0.19041-SP0
8 logical CPU cores, Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
-----

</details>



Discussion:
Thanks for the thorough issue report! I recall running into this before with some other igraph algorithm, so there might be a similar patch somewhere else in the codebase.

Would you be interested in opening a PR to fix this?
I suppose to do this properly one ought to scan the code base for uses of igraph, check which among them require the RNG and then add the seeding to those modules?

Since I'm still very new to scanpy and rather swamped at the moment, I won't have the time to do this anytime soon. Maybe add a help wanted tag? If nobody takes it up by the time my schedule lightens I'll see what I can do.
> I suppose to do this properly one ought to scan the code base for uses of igraph, check which among them require the RNG and then add the seeding to those modules?

This would be nice, but would also be a lot of work. I was thinking a PR would just make the layouts for PAGA reproducible, and add a test making sure this is the case.

Resolving patch:
diff --git a/scanpy/plotting/_tools/paga.py b/scanpy/plotting/_tools/paga.py
--- a/scanpy/plotting/_tools/paga.py
+++ b/scanpy/plotting/_tools/paga.py
@@ -14,6 +14,7 @@
 from matplotlib.axes import Axes
 from matplotlib.colors import is_color_like, Colormap
 from scipy.sparse import issparse
+from sklearn.utils import check_random_state
 
 from .. import _utils
 from .._utils import matrix, _IGraphLayout, _FontWeight, _FontSize
@@ -174,8 +175,11 @@ def _compute_pos(
     root=0,
     layout_kwds: Mapping[str, Any] = MappingProxyType({}),
 ):
+    import random
     import networkx as nx
 
+    random_state = check_random_state(random_state)
+
     nx_g_solid = nx.Graph(adjacency_solid)
     if layout is None:
         layout = 'fr'
@@ -190,9 +194,9 @@ def _compute_pos(
             )
             layout = 'fr'
     if layout == 'fa':
-        np.random.seed(random_state)
+        # np.random.seed(random_state)
         if init_pos is None:
-            init_coords = np.random.random((adjacency_solid.shape[0], 2))
+            init_coords = random_state.random_sample((adjacency_solid.shape[0], 2))
         else:
             init_coords = init_pos.copy()
         forceatlas2 = ForceAtlas2(
@@ -233,6 +237,7 @@ def _compute_pos(
             )
     else:
         # igraph layouts
+        random.seed(random_state.bytes(8))
         g = _sc_utils.get_igraph_from_adjacency(adjacency_solid)
         if 'rt' in layout:
             g_tree = _sc_utils.get_igraph_from_adjacency(adj_tree)
@@ -243,9 +248,11 @@ def _compute_pos(
             pos_list = g.layout(layout).coords
         else:
             # I don't know why this is necessary
-            np.random.seed(random_state)
+            # np.random.seed(random_state)
             if init_pos is None:
-                init_coords = np.random.random((adjacency_solid.shape[0], 2)).tolist()
+                init_coords = random_state.random_sample(
+                    (adjacency_solid.shape[0], 2)
+                ).tolist()
             else:
                 init_pos = init_pos.copy()
                 # this is a super-weird hack that is necessary as igraph’s


Test patch:
diff --git a/scanpy/tests/_images/master_paga.png b/scanpy/tests/_images/master_paga.png
Binary files a/scanpy/tests/_images/master_paga.png and b/scanpy/tests/_images/master_paga.png differ
diff --git a/scanpy/tests/_images/master_paga_continuous.png b/scanpy/tests/_images/master_paga_continuous.png
Binary files a/scanpy/tests/_images/master_paga_continuous.png and b/scanpy/tests/_images/master_paga_continuous.png differ
diff --git a/scanpy/tests/_images/master_paga_continuous_multiple.png b/scanpy/tests/_images/master_paga_continuous_multiple.png
Binary files a/scanpy/tests/_images/master_paga_continuous_multiple.png and b/scanpy/tests/_images/master_paga_continuous_multiple.png differ
diff --git a/scanpy/tests/_images/master_paga_continuous_obs.png b/scanpy/tests/_images/master_paga_continuous_obs.png
Binary files a/scanpy/tests/_images/master_paga_continuous_obs.png and b/scanpy/tests/_images/master_paga_continuous_obs.png differ
diff --git a/scanpy/tests/_images/master_paga_pie.png b/scanpy/tests/_images/master_paga_pie.png
Binary files a/scanpy/tests/_images/master_paga_pie.png and b/scanpy/tests/_images/master_paga_pie.png differ
diff --git a/scanpy/tests/test_paga.py b/scanpy/tests/test_paga.py
--- a/scanpy/tests/test_paga.py
+++ b/scanpy/tests/test_paga.py
@@ -2,6 +2,7 @@
 from pathlib import Path
 
 from matplotlib import cm
+import numpy as np
 
 import scanpy as sc
 
@@ -86,3 +87,21 @@ def test_paga_compare(image_comparer):
     sc.pl.paga_compare(pbmc, basis="umap", show=False)
 
     save_and_compare_images('master_paga_compare_pbmc3k')
+
+
+def test_paga_positions_reproducible():
+    """Check exact reproducibility and effect of random_state on paga positions"""
+    # https://github.com/theislab/scanpy/issues/1859
+    pbmc = sc.datasets.pbmc68k_reduced()
+    sc.tl.paga(pbmc, "bulk_labels")
+
+    a = pbmc.copy()
+    b = pbmc.copy()
+    c = pbmc.copy()
+
+    sc.pl.paga(a, show=False, random_state=42)
+    sc.pl.paga(b, show=False, random_state=42)
+    sc.pl.paga(c, show=False, random_state=13)
+
+    np.testing.assert_array_equal(a.uns["paga"]["pos"], b.uns["paga"]["pos"])
+    assert a.uns["paga"]["pos"].tolist() != c.uns["paga"]["pos"].tolist()