Repository: graspologic-org/graspologic

Issue:
Fix bug in `is_unweighted` for sparse
- [ ] Does this PR add any new dependencies?
- [ ] Does this PR modify any existing APIs?
   - [ ] Is the change to the API backwards compatible?
- [ ] Have you built the documentation (reference and/or tutorial) and verified the generated documentation is appropriate?

#### Reference Issues/PRs

#### What does this implement/fix? Briefly explain your changes.
`is_unweighted` doesn't work properly for a sparse array input

#### Any other comments?
I think we could instead just do `graph[graph != 0].max() == 1 and graph[graph != 0].min() == 1`
for that entire section of the code.
[BUG] Bug in joblib transitive dependency causes exception when multi-threading
## Expected Behavior
Multi-threading LatentDistributionTest using a "workers" value != 1 should return without error on all platforms.

## Actual Behavior
When using any "workers" value > 1 or equal to -1 on a Windows computer, the code throws an exception.

## Example Code

```python
test = LatentDistributionTest(input_graph=False, workers=10)
result = test.fit_predict(graph1, graph2)
```

## Full Traceback
```pytb
C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\msrwinadm4\AppData\Local\Temp\5\joblib_memmapping_folder_11132_7308949288 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
Traceback (most recent call last):
  File "GraphsByOrg.py", line 79, in <module>
    logger.info(f'Calculating nonpar for {org1} and {org2}')
  File "C:\ProgramData\Anaconda3\lib\site-packages\graspologic\inference\latent_distribution_test.py", line 487, in fit_predict
    self.fit(A1, A2)
  File "C:\ProgramData\Anaconda3\lib\site-packages\graspologic\inference\latent_distribution_test.py", line 449, in fit
    X1_hat, X2_hat, reps=self.n_bootstraps, workers=self.workers, auto=False
  File "C:\ProgramData\Anaconda3\lib\site-packages\hyppo\ksample\ksamp.py", line 166, in test
    return self.indep_test.test(u, v, reps, workers, auto=auto)
  File "C:\ProgramData\Anaconda3\lib\site-packages\hyppo\independence\dcorr.py", line 215, in test
    stat, pvalue = super(Dcorr, self).test(x, y, reps, workers)
  File "C:\ProgramData\Anaconda3\lib\site-packages\hyppo\independence\base.py", line 67, in test
    self._statistic, x, y, reps=reps, workers=workers, is_distsim=is_distsim
  File "C:\ProgramData\Anaconda3\lib\site-packages\hyppo\_utils.py", line 140, in perm_test
    [delayed(_perm_stat)(calc_stat, x, y, is_distsim) for rep in range(reps)]
  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 1027, in __call__
    self._terminate_backend()
  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 734, in _terminate_backend
    self._backend.terminate()
  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 571, in terminate
    delete_folder(self._workers._temp_folder)
  File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py", line 115, in delete_folder
    shutil.rmtree(folder_path, False, None)
  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 516, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 400, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\ProgramData\Anaconda3\lib\shutil.py", line 398, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\msrwinadm4\\AppData\\Local\\Temp\\5\\joblib_memmapping_folder_11132_7308949288\\11132-1819792920136-683b9c4b033b449dbac251acbe3decfb.pkl'
C:\ProgramData\Anaconda3\lib\site-packages\joblib\disk.py:122: UserWarning: Unable to delete folder C:\Users\msrwinadm4\AppData\Local\Temp\5\joblib_memmapping_folder_11132_7308949288 after 5 tentatives.
  .format(folder_path, RM_SUBDIRS_N_RETRY))
C:\ProgramData\Anaconda3\lib\site-packages\joblib\_memmapping_reducer.py:409: UserWarning: Failed to clean temporary folder: C:\Users\msrwinadm4\AppData\Local\Temp\5\joblib_memmapping_folder_11132_7308949288
  .format(pool_folder))

```

## Your Environment
* Python version: 3.7.6 (Anaconda)
* graspologic version: 0.1.0.dev331219603
* Windows 2016 Datacenter (448 GB RAM) x64

## Additional Details
graspologic==0.1.0.dev331219603
joblib==0.14.1
hyppo==0.1.3
scikit-image==0.16.2
scikit-learn==0.22.1
scipy==1.4.1
numpy==1.18.1

## Underlying problem:
Older versions of joblib have a known issue running on Windows.  See https://github.com/joblib/joblib/issues/806.  This appears to be fixed on May 3rd, 2020 by https://github.com/joblib/joblib/pull/966.

Hyppo uses joblib as a transitive dependency of scikit-learn but does not declare it as a dependency.  Scikit-learn only requires joblib 0.11 which does not include this fix.  See https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/_min_dependencies.py



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -51,6 +51,7 @@
         "anytree>=2.8.0",
         "gensim",
         "hyppo>=0.1.3",
+        "joblib>=0.17.0", # Older versions of joblib cause issue #806.  Transitive dependency of hyppo.
         "matplotlib>=3.0.0,<=3.3.0",
         "networkx>=2.1",
         "numpy>=1.8.1",


Test patch: