Repository: Project-MONAI/MONAI

Issue:
runtests.sh Installs Links to Current Code Directory
**Describe the bug**
When `runtests.sh` is run it tries to build with the `compile_cpp` function every time some operation is done such as autofixing. This installs links in various places to add the current directory into the Python environment such as in `$HOME/.local/lib/python3.8/site-packages` (for Python 3.8). This changes files such as the `easy-install.pth` file in that directory, and adds `monai.edd-link`, so that the current directory appears in `sys.path` mysteriously if the user isn't aware of this behaviour. If this is being run from a MONAI checkout then the directory will appear in virtual environments or conda environments even if these have MONAI already installed in them which will confuse the user about which version of the library is being used.

When the script is run there shouldn't be anything installed just by doing checks or autofixing code, so something like `-n` needs to be added to the `setup.py` commands in `compile_cpp` or the uninstall process run at the end of that function as well as the beginning. The invocation of `setup.py` includes the `--user` argument which is why the files in `$HOME/.local` are changed, without it files  in the current environment will be changed instead to install links there in the same way. Removing this argument therefore won't solve this problem.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to a MONAI checkout directory
2. Run commands './runtests.sh --autofix'

**Expected behavior**
The state of the environment should be unchanged after the script is run. 

**Environment**

Ensuring you use the relevant python executable, please paste the output of:

```
================================
Printing MONAI config...
================================
MONAI version: 0.7.0+236.g5efba23f
Numpy version: 1.20.3
Pytorch version: 1.10.0
MONAI flags: HAS_EXT = False, USE_COMPILED = False
MONAI rev id: 5efba23f47c706703897546d6fa2c2d5b1947abf

Optional dependencies:
Pytorch Ignite version: 0.4.7
Nibabel version: 3.2.1
scikit-image version: 0.18.3
Pillow version: 8.4.0
Tensorboard version: 2.7.0
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.11.1
tqdm version: 4.62.3
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.8.0
pandas version: 1.3.4
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: 1.22.0

For details about installing the optional dependencies, please visit:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies


================================
Printing system config...
================================
System: Linux
Linux version: Ubuntu 20.04.3 LTS
Platform: Linux-5.4.0-88-generic-x86_64-with-glibc2.17
Processor: x86_64
Machine: x86_64
Python version: 3.8.12
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 6
Num logical CPUs: 12
Num usable CPUs: 12
CPU usage (%): [15.8, 7.5, 53.7, 0.0, 0.0, 41.4, 0.0, 0.0, 26.9, 0.0, 80.4, 0.9]
CPU freq. (MHz): 1772
Load avg. in last 1, 5, 15 mins (%): [19.6, 15.2, 7.2]
Disk usage (%): 76.2
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 31.3
Available memory (GB): 17.1
Used memory (GB): 13.6

================================
Printing GPU config...
================================
Num GPUs: 2
Has CUDA: True
CUDA version: 10.2
cuDNN enabled: True
cuDNN version: 7605
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'compute_37']
GPU 0 Name: NVIDIA TITAN X (Pascal)
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 28
GPU 0 Total memory (GB): 11.9
GPU 0 CUDA capability (maj.min): 6.1
GPU 1 Name: NVIDIA GeForce GTX 980
GPU 1 Is integrated: False
GPU 1 Is multi GPU board: False
GPU 1 Multi processor count: 16
GPU 1 Total memory (GB): 3.9
GPU 1 CUDA capability (maj.min): 5.2
```



Discussion:
Hi @wyli ,

May I know whether PR https://github.com/Project-MONAI/MONAI/pull/3528 covers this issue?
If not, could you please help also cover it?

Thanks in advance.

Yes #3528 would address this, manually running `runtests.sh --clean` will make the environment unchanged.
I think the expected behaviour would be to not modify one's environment just by running tests or other operations. Another alternative is to run the compile part only when BUILD_MONAI is 1. My specific use case is that I have a Anaconda environment for MONAI with the current release installed, I used PYTHONPATH to point to a development checkout to change which MONAI gets loaded at import time when running tests or autofixing. If we include code to uninstall after running runtests.sh this might remove this version from the environment as well. 
Sure, I can try to make the installation optional. Since it's out of scope of #3528 I'll create a separate PR for it.

Resolving patch:
diff --git a/monai/config/deviceconfig.py b/monai/config/deviceconfig.py
--- a/monai/config/deviceconfig.py
+++ b/monai/config/deviceconfig.py
@@ -90,6 +90,7 @@ def print_config(file=sys.stdout):
         print(f"{k} version: {v}", file=file, flush=True)
     print(f"MONAI flags: HAS_EXT = {HAS_EXT}, USE_COMPILED = {USE_COMPILED}")
     print(f"MONAI rev id: {monai.__revision_id__}")
+    print(f"MONAI __file__: {monai.__file__}")
 
     print("\nOptional dependencies:", file=file, flush=True)
     for k, v in get_optional_config_values().items():


Test patch:
diff --git a/tests/min_tests.py b/tests/min_tests.py
--- a/tests/min_tests.py
+++ b/tests/min_tests.py
@@ -101,6 +101,7 @@ def run_testsuit():
         "test_label_filter",
         "test_lltm",
         "test_lmdbdataset",
+        "test_lmdbdataset_dist",
         "test_load_image",
         "test_load_imaged",
         "test_load_spacing_orientation",
diff --git a/tests/test_distcall.py b/tests/test_call_dist.py
similarity index 100%
rename from tests/test_distcall.py
rename to tests/test_call_dist.py
diff --git a/tests/test_lmdbdataset_dist.py b/tests/test_lmdbdataset_dist.py
new file mode 100644
--- /dev/null
+++ b/tests/test_lmdbdataset_dist.py
@@ -0,0 +1,72 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import shutil
+import tempfile
+import unittest
+
+import numpy as np
+
+from monai.data import LMDBDataset, json_hashing
+from monai.transforms import Transform
+from tests.utils import DistCall, DistTestCase, skip_if_windows
+
+
+class _InplaceXform(Transform):
+    def __call__(self, data):
+        if data:
+            data[0] = data[0] + np.pi
+        else:
+            data.append(1)
+        return data
+
+
+@skip_if_windows
+class TestMPLMDBDataset(DistTestCase):
+    def setUp(self):
+        self.tempdir = tempfile.mkdtemp()
+
+    def tearDown(self):
+        shutil.rmtree(self.tempdir)
+
+    @DistCall(nnodes=1, nproc_per_node=1)
+    def test_mp_cache(self):
+        items = [[list(range(i))] for i in range(5)]
+
+        ds = LMDBDataset(items, transform=_InplaceXform(), cache_dir=self.tempdir, lmdb_kwargs={"map_size": 10 * 1024})
+        self.assertEqual(items, [[[]], [[0]], [[0, 1]], [[0, 1, 2]], [[0, 1, 2, 3]]])
+        ds1 = LMDBDataset(items, transform=_InplaceXform(), cache_dir=self.tempdir, lmdb_kwargs={"map_size": 10 * 1024})
+        self.assertEqual(list(ds1), list(ds))
+        self.assertEqual(items, [[[]], [[0]], [[0, 1]], [[0, 1, 2]], [[0, 1, 2, 3]]])
+
+        ds = LMDBDataset(
+            items,
+            transform=_InplaceXform(),
+            cache_dir=self.tempdir,
+            lmdb_kwargs={"map_size": 10 * 1024},
+            hash_func=json_hashing,
+        )
+        self.assertEqual(items, [[[]], [[0]], [[0, 1]], [[0, 1, 2]], [[0, 1, 2, 3]]])
+        ds1 = LMDBDataset(
+            items,
+            transform=_InplaceXform(),
+            cache_dir=self.tempdir,
+            lmdb_kwargs={"map_size": 10 * 1024},
+            hash_func=json_hashing,
+        )
+        self.assertEqual(list(ds1), list(ds))
+        self.assertEqual(items, [[[]], [[0]], [[0, 1]], [[0, 1, 2]], [[0, 1, 2, 3]]])
+
+        self.assertTrue(isinstance(ds1.info(), dict))
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/tests/test_distributed_sampler.py b/tests/test_sampler_dist.py
similarity index 100%
rename from tests/test_distributed_sampler.py
rename to tests/test_sampler_dist.py
diff --git a/tests/test_timedcall.py b/tests/test_timedcall_dist.py
similarity index 100%
rename from tests/test_timedcall.py
rename to tests/test_timedcall_dist.py
diff --git a/tests/test_distributed_weighted_random_sampler.py b/tests/test_weighted_random_sampler_dist.py
similarity index 100%
rename from tests/test_distributed_weighted_random_sampler.py
rename to tests/test_weighted_random_sampler_dist.py