Repository: DataBiosphere/toil

Issue:
Realease 3.15.0 shows a bug with azure
To replicate the bug I get:

    mkvirtualenv test-toil
    pip install toil==3.15.0
    toil --help

Output: 
```
Traceback (most recent call last):
  File "/Users/arangooj/.virtualenvs/test-toil/bin/toil", line 11, in <module>
    sys.exit(main())
  File "/Users/arangooj/.virtualenvs/test-toil/lib/python2.7/site-packages/toil/utils/toilMain.py", line 12, in main
    modules = loadModules()
  File "/Users/arangooj/.virtualenvs/test-toil/lib/python2.7/site-packages/toil/utils/toilMain.py", line 38, in loadModules
    from toil.utils import (toilKill,
  File "/Users/arangooj/.virtualenvs/test-toil/lib/python2.7/site-packages/toil/utils/toilLaunchCluster.py", line 20, in <module>
    from toil.jobStores.azureJobStore import credential_file_path
  File "/Users/arangooj/.virtualenvs/test-toil/lib/python2.7/site-packages/toil/jobStores/azureJobStore.py", line 42, in <module>
    from azure.common import AzureMissingResourceHttpError, AzureException
ImportError: No module named azure.common
```

My environment is running `Python 2.7.14`

Installing `pip install azure` fixes this error, after installing a lot of dependencies. But I'm not using `azure` and it looks like it's an extra. So I think one expects that it shouldn't break `toil` if I don't really need it.

Thanks!
 


Discussion:
Thanks for reporting this!
I've run into this too when installing toil[aws,mesos].  Even using [aws,mesos,azure] gives a bunch of version warnings:

docker 2.5.1 requires backports.ssl-match-hostname>=3.5; python_version < "3.5", which is not installed.
azure 2.0.0 has requirement azure-storage~=0.34.2, but you'll have azure-storage 0.35.1 which is incompatible.
azure-cli-core 2.0.31 has requirement pip, but you'll have pip 10.0.0b2 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-batch~=4.0.0, but you'll have azure-mgmt-batch 4.1.0 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-compute~=1.0.0, but you'll have azure-mgmt-compute 2.1.0 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-containerregistry~=0.2.1, but you'll have azure-mgmt-containerregistry 1.0.1 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-dns~=1.0.1, but you'll have azure-mgmt-dns 1.2.0 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-keyvault~=0.31.0, but you'll have azure-mgmt-keyvault 0.40.0 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-network~=1.0.0, but you'll have azure-mgmt-network 1.7.1 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-rdbms~=0.1.0, but you'll have azure-mgmt-rdbms 0.2.0rc1 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-resource~=1.1.0, but you'll have azure-mgmt-resource 1.2.2 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-sql~=0.5.1, but you'll have azure-mgmt-sql 0.7.1 which is incompatible.
azure-mgmt 1.0.0 has requirement azure-mgmt-storage~=1.0.0, but you'll have azure-mgmt-storage 1.5.0 which is incompatible.


Resolving patch:
diff --git a/src/toil/jobStores/__init__.py b/src/toil/jobStores/__init__.py
--- a/src/toil/jobStores/__init__.py
+++ b/src/toil/jobStores/__init__.py
@@ -12,3 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 from __future__ import absolute_import
+azure_credential_file_path = '~/.toilAzureCredentials'
diff --git a/src/toil/jobStores/azureJobStore.py b/src/toil/jobStores/azureJobStore.py
--- a/src/toil/jobStores/azureJobStore.py
+++ b/src/toil/jobStores/azureJobStore.py
@@ -44,6 +44,8 @@
 from azure.storage.blob.models import BlobPermissions, BlobBlock
 from azure.cosmosdb.table import TableService, EntityProperty, Entity
 
+from toil.jobStores import azure_credential_file_path as credential_file_path
+
 # noinspection PyPackageRequirements
 # (pulled in transitively)
 import requests
@@ -67,7 +69,7 @@
 logging.getLogger("azure.cosmosdb.common._auth").setLevel(logging.WARNING)
 logging.getLogger("urllib3.connectionpool").setLevel(logging.WARNING)
 
-credential_file_path = '~/.toilAzureCredentials'
+
 
 
 def _fetchAzureAccountKey(accountName):
diff --git a/src/toil/utils/toilLaunchCluster.py b/src/toil/utils/toilLaunchCluster.py
--- a/src/toil/utils/toilLaunchCluster.py
+++ b/src/toil/utils/toilLaunchCluster.py
@@ -18,7 +18,8 @@
 from toil.lib.bioio import parseBasicOptions, getBasicOptionParser
 from toil.utils import addBasicProvisionerOptions
 from toil import applianceSelf
-from toil.jobStores.azureJobStore import credential_file_path
+
+from toil.jobStores import azure_credential_file_path as credential_file_path
 
 logger = logging.getLogger(__name__)
 


Test patch: