Repository: Pyomo/pyomo

Issue:
Pyomo deferred import system causes nose to fail for any package that imports pyomo
## Summary

Importing pyomo causes several instances of `DeferredImportModule` to be put into `sys.modules`. If any of these modules are not available, then calling `hasattr` on them results in an exception. When running `nosetests` with coverage, nose iterates through `sys.modules` and calls `hasattr` on them. As a result, running `nosetests` with coverage on any package that imports pyomo causes an exception if any of these modules are not available.

<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `pyomo solve` to optimize a model and ..." -->

### Steps to reproduce the issue

<!-- Please include the command run and/or provide an example script that demonstrates the problem. -->

```
In [1]: import pyomo.environ

In [2]: import sys

In [3]: from pyomo.common.dependencies import DeferredImportModule

In [4]: for k, v in sys.modules.items():
   ...:     if isinstance(v, DeferredImportModule):
   ...:         print(k)
   ...:
pyomo.common.dependencies.yaml
pyomo.common.dependencies.pympler
pyomo.common.dependencies.numpy
pyomo.common.dependencies.scipy
pyomo.common.dependencies.networkx
pyomo.common.dependencies.pandas
pyomo.common.dependencies.dill
pyomo.common.dependencies.matplotlib
pyomo.common.dependencies.scipy.stats
pyomo.common.dependencies.scipy.sparse
pyomo.common.dependencies.scipy.spatial
pyomo.common.dependencies.scipy.integrate
pyomo.common.dependencies.matplotlib.pyplot
pyomo.common.dependencies.matplotlib.pylab

In [5]: hasattr(sys.modules['pyomo.common.dependencies.yaml'], '__file__')
---------------------------------------------------------------------------
DeferredImportError                       Traceback (most recent call last)
<ipython-input-5-31ea100e11d2> in <module>
----> 1 hasattr(sys.modules['pyomo.common.dependencies.yaml'], '__file__')

~/repos/pyomo/pyomo/common/dependencies.py in __getattr__(self, attr)
    131             for _sub in self._submodule_name[1:].split('.'):
    132                 _mod = getattr(_mod, _sub)
--> 133         return getattr(_mod, attr)
    134
    135

~/repos/pyomo/pyomo/common/dependencies.py in __getattr__(self, attr)
     59             raise AttributeError("'%s' object has no attribute '%s'"
     60                                  % (type(self).__name__, attr))
---> 61         raise DeferredImportError(self._moduleunavailable_message())
     62
     63     def _moduleunavailable_message(self, msg=None):

DeferredImportError: The yaml module (an optional Pyomo dependency) failed to import: ModuleNotFoundError: No module named 'yaml'
```

### Error Message

Here is the stack trace when running nose:

```
Traceback (most recent call last):
  File "/usr/share/miniconda/bin/nosetests", line 33, in <module>
    sys.exit(load_entry_point('nose', 'console_scripts', 'nosetests')())
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/core.py", line 121, in __init__
    **extra_args)
  File "/usr/share/miniconda/lib/python3.7/unittest/main.py", line 101, in __init__
    self.runTests()
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/core.py", line 207, in runTests
    result = self.testRunner.run(self.test)
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/core.py", line 66, in run
    result.printErrors()
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/result.py", line 110, in printErrors
    self.config.plugins.report(self.stream)
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/plugins/manager.py", line 99, in __call__
    return self.call(*arg, **kw)
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/plugins/manager.py", line 167, in simple
    result = meth(*arg, **kw)
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/plugins/cover.py", line 183, in report
    self.coverInstance.stop()
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/plugins/cover.py", line 188, in <listcomp>
    if self.wantModuleCoverage(name, module)]
  File "/usr/share/miniconda/lib/python3.7/site-packages/nose/plugins/cover.py", line 233, in wantModuleCoverage
    if not hasattr(module, '__file__'):
  File "/home/runner/work/parapint/parapint/pyomo/pyomo/common/dependencies.py", line 133, in __getattr__
    return getattr(_mod, attr)
  File "/home/runner/work/parapint/parapint/pyomo/pyomo/common/dependencies.py", line 61, in __getattr__
    raise DeferredImportError(self._moduleunavailable_message())
pyomo.common.dependencies.DeferredImportError: The yaml module (an optional Pyomo dependency) failed to import: ModuleNotFoundError: No module named 'yaml'
```

<!-- If Pyomo reported an error, provide the error message. PLEASE INCLUDE THE ENTIRE STACK TRACE. -->

<!-- If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->

### Information on your system

Pyomo version: main branch, up to date
Python version: 3.8
Operating system: osx
How Pyomo was installed: source
version of nose: 1.3.7


Discussion:
Interesting.  I wonder why the slim builds do not catch this?  I think a quick patch is the following:
```diff
diff --git a/pyomo/common/dependencies.py b/pyomo/common/dependencies.py
index 8b65f32..8cede6b 100644
--- a/pyomo/common/dependencies.py
+++ b/pyomo/common/dependencies.py
@@ -109,6 +109,8 @@ class DeferredImportModule(object):
     def __init__(self, indicator, deferred_submodules, submodule_name):
         self._indicator_flag = indicator
         self._submodule_name = submodule_name
+        self.__file__ = None # Disable nose's coverage of this module
+        self.__spec__ = None # Indicate that this is not a "real" module
 
         if not deferred_submodules:
             return
```
That is a great question.
Ahh - the problem is with the `nose` coverage plugin (and not with `coverage`).  We don't use nose's coverage plugin: instead, we directly initialize coverage through the system setup (so that we can reliably get coverage of things that are run during import and get coverage of multithreaded/multiprocess parts of the codebase) and then manually process the coverage information after the fact.

PR coming shortly.

Resolving patch:
diff --git a/pyomo/common/dependencies.py b/pyomo/common/dependencies.py
--- a/pyomo/common/dependencies.py
+++ b/pyomo/common/dependencies.py
@@ -109,6 +109,8 @@ class DeferredImportModule(object):
     def __init__(self, indicator, deferred_submodules, submodule_name):
         self._indicator_flag = indicator
         self._submodule_name = submodule_name
+        self.__file__ = None # Disable nose's coverage of this module
+        self.__spec__ = None # Indicate that this is not a "real" module
 
         if not deferred_submodules:
             return


Test patch: