Repository: pyswmm/swmm-python (language: python)

Issue:
Trouble with swmm-toolkit install 
I am getting various errors after installing on windows and linux. I also tried building and installing from source and still get the same errors. Not sure if I am missing something.

## Windows

after `pip install swmm-toolkit` executing ` from swmm.toolkit import output, output_enum` yields:

```
>>> from swmm.toolkit import output, output_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~\Miniconda\envs\swmm\lib\site-packages\swmm\toolkit\output.py", line 18, in <module>
    from . import _output
ImportError: DLL load failed while importing _output: The specified module could not be found.
```

## Linux

after `pip install swmm-toolkit` executing ` from swmm.toolkit import output, output_enum` yields:

```
>>> from swmm.toolkit import output, output_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'output_enum' from 'swmm.toolkit' (~/miniconda3/envs/swmm/lib/python3.9/site-packages/swmm/toolkit/__init__.py)
```


Discussion / hints:
Did you activate your virtual environment prior to using it? If you don’t activate the search path for dlls won’t be set properly. 
Does it only work with venv or virtualenv? I am using a conda env that is activated.
It should work. I use both conda and venv in development. 
Are you installing from the wheels on pypi or from source?
I've tried both and am getting the same error. I'm not sure how to check by search path for dlls, is it just my $PATH env variable? 
In the latest version  output_enum.py has been renamed shared_enum.py. So that would solve the error you are getting on Linux
Can you confirm aenum version?
Linux : aenum==2.2.6
Windows : aenum==2.2.6
What version of swmm-toolkit do you have installed? 

swmm-toolkit = 0.8.2

When I install from pypi into a conda env everything works properly. Based on the info you have provided I can't isolate the problem. Can you take us through step by step what you are doing so we can recreate the problem on our side? 
ok, so changing output_enum to shared_enum fixed the issue on linux. 

On windows I made a fresh conda env, ran `pip install swmm-toolkit` am still getting:

```
>>> from swmm.toolkit import output, shared_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\KAROSC\Miniconda\envs\swmmtk\lib\site-packages\swmm\toolkit\output.py", line 18, in <module>
    from . import _output
ImportError: DLL load failed while importing _output: The specified module could not be found.

```

popped open my site-packages directory and see  swmm and swmm_toolkit folders, which matches what I find on linux. Below are the dlls in the swmm_toolkit folder:


```
Directory: C:\Users\KAROSC\Miniconda\envs\swmmtk\Lib\site-packages\swmm_toolkit

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----         7/20/2021  11:28 AM         332568 concrt140.dll
-a----         7/20/2021  11:28 AM         627992 msvcp140.dll
-a----         7/20/2021  11:28 AM          31512 msvcp140_1.dll
-a----         7/20/2021  11:28 AM         206104 msvcp140_2.dll
-a----         7/20/2021  11:28 AM          19456 swmm-output.dll
-a----         7/20/2021  11:28 AM         470016 swmm5.dll
-a----         7/20/2021  11:28 AM         154392 vcomp140.dll
-a----         7/20/2021  11:28 AM          85784 vcruntime140.dll
```
That's good info. Whats in the swmm.toolkit folder? Cause its not able to load the swmm/toolkit/_output*.dll for some reason

```
    Directory: C:\Users\KAROSC\Miniconda\envs\swmmtk\Lib\site-packages\swmm\toolkit


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         7/20/2021  11:28 AM                __pycache__
-a----         7/20/2021  11:28 AM          14300 output.py
-a----         7/20/2021  11:28 AM          12803 output_metadata.py
-a----         7/20/2021  11:28 AM          16097 shared_enum.py
-a----         7/20/2021  11:28 AM          43481 solver.py
-a----         7/20/2021  11:28 AM          52736 _output.cp39-win_amd64.pyd
-a----         7/20/2021  11:28 AM         161792 _solver.cp39-win_amd64.pyd
-a----         7/20/2021  11:28 AM            893 __init__.py

```
The library is in there. Why can't it find it? 

What shell are you using? Powershell?

Could it be a permissions issue? You don't appear to have execution rights for the library. 

We have seen this error before with conda installation. 


```
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated.  Libraries may fail to load.  To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>> from swmm.toolkit import solver
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\Anaconda3\lib\site-packages\swmm\toolkit\solver.py", line 18, in <module>
    from . import _solver
ImportError: DLL load failed while importing _solver: The specified module could not be found.
```

It happens when I don't activate my conda environment using:
`conda activate `


I am activating for sure:

```

(jupyter) PS C:\CKAROS_PROJECTS> conda activate
(base) PS C:\CKAROS_PROJECTS> conda activate swmmtk
(swmmtk) PS C:\CKAROS_PROJECTS> python
Python 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:37:25) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from swmm.toolkit import output, shared_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\KAROSC\Miniconda\envs\swmmtk\lib\site-packages\swmm\toolkit\output.py", line 18, in <module>
    from . import _output
ImportError: DLL load failed while importing _output: The specified module could not be found.
>>>
```
@karosc Can you try adding your miniconda path (and miniconda\Library\bin\) to your system environment path? 

Edit: Maybe C:\Users\KAROSC\Miniconda\Library\bin\ ? I'm not familiar with miniconda
I ran the following on both my working linux and not working windows systems:

```
import sys
[print(i) for i in sys.path
```
### Linux output 

~/miniconda3/envs/swmm/lib/python39.zip
~/miniconda3/envs/swmm/lib/python3.9
~/miniconda3/envs/swmm/lib/python3.9/lib-dynload
~/miniconda3/envs/swmm/lib/python3.9/site-packages
 
### Windows output
 C:\\Users\\KAROSC\\Miniconda\\envs\\swmmtk\\python39.zip
 C:\\Users\\KAROSC\\Miniconda\\envs\\swmmtk\\DLLs
 C:\\Users\\KAROSC\\Miniconda\\envs\\swmmtk\\lib
 C:\\Users\\KAROSC\\Miniconda\\envs\\swmmtk
 C:\\Users\\KAROSC\\Miniconda\\envs\\swmmtk\\lib\\site-packages




Another note, I can fix the windows issue by copying all dlls from `site-packages\swmm_toolkit` to `site-packages\swmm\toolkit`. So it definitely feels like a path issue....
I did the Windows dev work using cmd prompt. So I suppose it's possible this is a PowerShell related issue. Could you humor me by trying a windows command prompt? 
Still same error. 
```
(base) C:\Users\KAROSC>conda create -n swmmtk_cmd python
Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.10.1
  latest version: 4.10.3

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: C:\Users\KAROSC\Miniconda\envs\swmmtk_cmd

  added / updated specs:
    - python


The following NEW packages will be INSTALLED:

  ca-certificates    conda-forge/win-64::ca-certificates-2021.5.30-h5b45459_0
  certifi            conda-forge/win-64::certifi-2021.5.30-py39hcbf5309_0
  openssl            conda-forge/win-64::openssl-1.1.1k-h8ffe710_0
  pip                conda-forge/noarch::pip-21.1.3-pyhd8ed1ab_0
  python             conda-forge/win-64::python-3.9.6-h7840368_1_cpython
  python_abi         conda-forge/win-64::python_abi-3.9-2_cp39
  setuptools         conda-forge/win-64::setuptools-49.6.0-py39hcbf5309_3
  sqlite             conda-forge/win-64::sqlite-3.36.0-h8ffe710_0
  tzdata             conda-forge/noarch::tzdata-2021a-he74cb21_1
  ucrt               conda-forge/win-64::ucrt-10.0.20348.0-h57928b3_0
  vc                 conda-forge/win-64::vc-14.2-hb210afc_5
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.29.30037-h902a5da_5
  wheel              conda-forge/noarch::wheel-0.36.2-pyhd3deb0d_0
  wincertstore       conda-forge/win-64::wincertstore-0.2-py39hcbf5309_1006


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate swmmtk_cmd
#
# To deactivate an active environment, use
#
#     $ conda deactivate


(base) C:\Users\KAROSC>conda activate swmmtk_cmd

(swmmtk_cmd) C:\Users\KAROSC>python -m pip install swmm-toolkit
Collecting swmm-toolkit
  Using cached swmm_toolkit-0.8.2-cp39-cp39-win_amd64.whl (905 kB)
Collecting aenum==2.2.6
  Using cached aenum-2.2.6-py3-none-any.whl (43 kB)
Installing collected packages: aenum, swmm-toolkit
Successfully installed aenum-2.2.6 swmm-toolkit-0.8.2

(swmmtk_cmd) C:\Users\KAROSC>python
Python 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:37:25) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from swmm.toolkit import output, shared_enum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\KAROSC\Miniconda\envs\swmmtk_cmd\lib\site-packages\swmm\toolkit\output.py", line 18, in <module>
    from . import _output
ImportError: DLL load failed while importing _output: The specified module could not be found.
>>>
```
Well the swmm\toolkit\__init__.py file should add swmm_toolkit to the path automatically.This doesn't appear to be happening on your machine for some reason. We have experienced __init__.py not getting called when the conda environment is not activated properly. I haven't used conda in powershell. What command do you use to activate an conda env in powershell? 
 
Ok. I am able to recreate the problem. It's related to python3.9. I will investigate. 
Ok I think I may have found the cause. I changed the end of the toolkit __init__.py to include a print call:

```
# Adds directory containing swmm libraries to path
if platform.system() == "Windows":
    libdir = os.path.join(os.path.dirname(__file__), "../../swmm_toolkit")
    print(libdir)
    
    if hasattr(os, 'add_dll_directory'):
        os.add_dll_directory(libdir)
    else:
        os.environ["PATH"] = libdir + ";" + os.environ["PATH"]

```

printing libdir yields: 

`C:\Users\KAROSC\Miniconda\envs\swmmtk\lib\site-packages\swmm\toolkit\../../swmm_toolkit`

that doesn't look quite right...
looks like it might be caused by os path separator
I did some testing and found that commenting out the following in `__init__.py` fixes the issue:
```
# Adds directory containing swmm libraries to path
if platform.system() == "Windows":
    libdir = os.path.join(os.path.dirname(__file__), "../../swmm_toolkit")
    print(libdir)
    
    #if hasattr(os, 'add_dll_directory'):
    #   os.add_dll_directory(libdir)
    #else:
    os.environ["PATH"] = libdir + ";" + os.environ["PATH"]

```

This is very weird, because [python docs suggest os.add_dll_directory](https://docs.python.org/3/library/os.html#os.add_dll_directory) is the correct way to do this.

Nice catch!

A quick google search reveals this is a known issue.
Ok, even further, this is [conda issue](https://github.com/ContinuumIO/anaconda-issues/issues/12475). We need to set another environment variable if using conda. If I set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1, then the whole thing works with the original source code. 

Not sure if we want to check if using conda before setting the variable, but the following is a fix
```
# Adds directory containing swmm libraries to path
if platform.system() == "Windows":
    libdir = os.path.join(os.path.dirname(__file__), "../../swmm_toolkit")
    print(libdir)
    
    if hasattr(os, 'add_dll_directory'):
       os.environ['CONDA_DLL_SEARCH_MODIFICATION_ENABLE']="1"
       os.add_dll_directory(libdir)
       
    else:
        os.environ["PATH"] = libdir + ";" + os.environ["PATH"]

```

if we want to check for conda, we could do something like this:

```
import sys

# Adds directory containing swmm libraries to path
if platform.system() == "Windows":
    libdir = os.path.join(os.path.dirname(__file__), "../../swmm_toolkit")
    print(libdir)
    
    if hasattr(os, 'add_dll_directory'):
       if 'conda' in sys.version:
           os.environ['CONDA_DLL_SEARCH_MODIFICATION_ENABLE']="1"
       os.add_dll_directory(libdir)
       
    else:
        os.environ["PATH"] = libdir + ";" + os.environ["PATH"]
```
Awesome! Thanks so much!
This looks great. Do you want to create a PR to close this issue? 
on it!

Fix patch:
diff --git a/swmm-toolkit/src/swmm/toolkit/__init__.py b/swmm-toolkit/src/swmm/toolkit/__init__.py
index 19312b52..b9efd486 100644
--- a/swmm-toolkit/src/swmm/toolkit/__init__.py
+++ b/swmm-toolkit/src/swmm/toolkit/__init__.py
@@ -29,6 +29,7 @@
 
 import os
 import platform
+import sys
 
 
 # Adds directory containing swmm libraries to path
@@ -36,6 +37,8 @@
     libdir = os.path.join(os.path.dirname(__file__), "../../swmm_toolkit")
 
     if hasattr(os, 'add_dll_directory'):
-        os.add_dll_directory(libdir)
+       if 'conda' in sys.version:
+           os.environ['CONDA_DLL_SEARCH_MODIFICATION_ENABLE']="1"
+       os.add_dll_directory(libdir)
     else:
         os.environ["PATH"] = libdir + ";" + os.environ["PATH"]


Test patch:
diff --git a/nrtest-swmm/nrtest_swmm/output_reader.py b/nrtest-swmm/nrtest_swmm/output_reader.py
index 7b5f9666..a845be5c 100644
--- a/nrtest-swmm/nrtest_swmm/output_reader.py
+++ b/nrtest-swmm/nrtest_swmm/output_reader.py
@@ -18,7 +18,7 @@
 from itertools import islice
 
 # project import
-from swmm.toolkit import output, output_enum
+from swmm.toolkit import output, shared_enum
 
 
 def output_generator(path_ref):
@@ -40,7 +40,7 @@ def output_generator(path_ref):
     with OutputReader(path_ref) as reader:
 
         for period_index in range(0, reader.report_periods()):
-            for element_type in islice(output_enum.ElementType, 4):
+            for element_type in islice(shared_enum.ElementType, 4):
                 for element_index in range(0, reader.element_count(element_type)):
 
                     yield (reader.element_result(element_type, period_index, element_index),
@@ -57,10 +57,10 @@ def __init__(self, filename):
         self.handle = None
         self.count = None
         self.get_element_result = {
-            output_enum.ElementType.SUBCATCH: output.get_subcatch_result,
-            output_enum.ElementType.NODE: output.get_node_result,
-            output_enum.ElementType.LINK: output.get_link_result,
-            output_enum.ElementType.SYSTEM: output.get_system_result
+            shared_enum.ElementType.SUBCATCH: output.get_subcatch_result,
+            shared_enum.ElementType.NODE: output.get_node_result,
+            shared_enum.ElementType.LINK: output.get_link_result,
+            shared_enum.ElementType.SYSTEM: output.get_system_result
         }
 
     def __enter__(self):
@@ -73,7 +73,7 @@ def __exit__(self, type, value, traceback):
         output.close(self.handle)
 
     def report_periods(self):
-        return output.get_times(self.handle, output_enum.Time.NUM_PERIODS)
+        return output.get_times(self.handle, shared_enum.Time.NUM_PERIODS)
 
     def element_count(self, element_type):
         return self.count[element_type]