Repository: pvlib/pvlib-python
Pull request #816 (2019-11-11 22:53:47)

fix for *_TMY3.epw reading issue #782
This PR is for solving epw file reading problem.
A note is added in docstring of epw.py and a validity check is added in atmosphere.first_solar_spectral_correction. This validity check is done according to kwargs min_pw and max_pw. If the value is higher, np.nan is returned.

pvlib python pull request guidelines
====================================

Thank you for your contribution to pvlib python! You may delete all of these instructions except for the list below.

You may submit a pull request with your code at any stage of completion.

The following items must be addressed before the code can be merged. Please don't hesitate to ask for help if you're unsure of how to accomplish any of the items below:

 - [x] Closes  #782
 - [x] I am familiar with the [contributing guidelines](http://pvlib-python.readthedocs.io/en/latest/contributing.html).
 - [x] Fully tested. Added and/or modified tests to ensure correct behavior for all reasonable inputs. Tests (usually) must pass on the TravisCI and Appveyor testing services.
 - [x] Adds description and name entries in the appropriate `docs/sphinx/source/whatsnew` file for all changes.
 - [x] Code quality and style is sufficient. Passes LGTM and SticklerCI checks.
 - [x] New code is fully documented. Includes sphinx/numpydoc compliant docstrings and comments in the code where necessary.
 - [x] Pull request is nearly complete and ready for detailed review.

Brief description of the problem and proposed solution (if not already fully described in the issue linked to above):


tylunel: Hi, 
I need some help to understand why stickler-ci finds lints only for the lines I have edited in epw.read_epw docstring and not for the others previously existing lines (but which have the same lint however: E501 line too long). 
And for the rest, any other feedback is welcomed as well.
Thanks in advance!
CameronTStark: > I need some help to understand why stickler-ci finds lints only for the lines I have edited in epw.read_epw docstring and not for the others previously existing lines (but which have the same lint however: E501 line too long).

I believe this is to assign lint responsibility to the most recent editor. It would  interfere with usage of git-blame to troubleshoot code since it would insert a format change in that line that might obscure a bug. The alternative would be the CI making you responsible for more lines than you edited. You can imagine this being dangerous if done on running code as opposed to the documentation in your case. 

I'm sure it'll stop barking at you if you shorten those lines.
tylunel: Thanks for your feedbacks!
New version is pushed. I have made the suggested change, except for editing the table in docstring of read_epw() (sorry for that, I took the easy way to get around the problem ^^')

Patch:
diff --git a/docs/sphinx/source/whatsnew/v0.7.0.rst b/docs/sphinx/source/whatsnew/v0.7.0.rst
index 8e32278..613b458 100644
--- a/docs/sphinx/source/whatsnew/v0.7.0.rst
+++ b/docs/sphinx/source/whatsnew/v0.7.0.rst
@@ -82,6 +82,12 @@ API Changes
       - Methods `PVSystem.ashraeiam`, `PVSystem.physicaliam` and
        `PVSystem.sapm_aoi_loss` are deprecated and will be removed in v0.8.
 
+* Changes related to spectral modifier (:issue:`782`):
+   * Changes to functions
+      - Added the argument `pw_min` and `pw_max`, default values 0.1 and 8 resp., 
+	to `atmosphere.first_solar_spectral_correction`. This function now returns NaN
+	if pw value higher than `pw_max`.
+
 * Calling :py:func:`pvlib.pvsystem.retrieve_sam` with no parameters will raise
   an exception instead of displaying a dialog.
 * The `times` keyword argument has been deprecated in the
@@ -146,6 +152,8 @@ Documentation
 * Edited docstring for `pvsystem.sapm` to remove DataFrame option for input
   `module`. The DataFrame option was never tested and would cause an error if
   used. (:issue:`785`)
+* Note warning about _TMY3.epw files retrieved from energyplus.net in docstring 
+  of `epw.read_epw`
 
 Removal of prior version deprecations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/pvlib/atmosphere.py b/pvlib/atmosphere.py
index 6be7a33..aca7449 100644
--- a/pvlib/atmosphere.py
+++ b/pvlib/atmosphere.py
@@ -320,8 +320,9 @@ def gueymard94_pw(temp_air, relative_humidity):
     return pw
 
 
-def first_solar_spectral_correction(pw, airmass_absolute, module_type=None,
-                                    coefficients=None):
+def first_solar_spectral_correction(pw, airmass_absolute,
+                                    module_type=None, coefficients=None,
+                                    min_pw=0.1, max_pw=8):
     r"""
     Spectral mismatch modifier based on precipitable water and absolute
     (pressure corrected) airmass.
@@ -364,6 +365,14 @@ def first_solar_spectral_correction(pw, airmass_absolute, module_type=None,
     airmass_absolute : array-like
         absolute (pressure corrected) airmass.
 
+    min_pw : float, default 0.1
+        minimum atmospheric precipitable water (cm). A lower pw value will be
+        automatically set to this minimum value to avoid model divergence.
+
+    max_pw : float, default 8
+        maximum atmospheric precipitable water (cm). If a higher value is
+        encountered it will be set to np.nan to avoid model divergence.
+
     module_type : None or string, default None
         a string specifying a cell type. Can be lower or upper case
         letters. Admits values of 'cdte', 'monosi', 'xsi', 'multisi',
@@ -422,16 +431,18 @@ def first_solar_spectral_correction(pw, airmass_absolute, module_type=None,
     # *** Pwat ***
     # Replace Pwat Values below 0.1 cm with 0.1 cm to prevent model from
     # diverging"
-
-    if np.min(pw) < 0.1:
-        pw = np.maximum(pw, 0.1)
-        warn('Exceptionally low Pwat values replaced with 0.1 cm to prevent' +
-             ' model divergence')
+    pw = np.atleast_1d(pw)
+    pw = pw.astype('float64')
+    if np.min(pw) < min_pw:
+        pw = np.maximum(pw, min_pw)
+        warn('Exceptionally low pw values replaced with {0} cm to prevent '
+             'model divergence'.format(min_pw))
 
     # Warn user about Pwat data that is exceptionally high
-    if np.max(pw) > 8:
-        warn('Exceptionally high Pwat values. Check input data:' +
-             ' model may diverge in this range')
+    if np.max(pw) > max_pw:
+        pw[pw > max_pw] = np.nan
+        warn('Exceptionally high pw values replaced by np.nan: '
+             'check input data.')
 
     # *** AMa ***
     # Replace Extremely High AM with AM 10 to prevent model divergence
diff --git a/pvlib/iotools/epw.py b/pvlib/iotools/epw.py
index 4a714d0..3fe7e8a 100644
--- a/pvlib/iotools/epw.py
+++ b/pvlib/iotools/epw.py
@@ -67,9 +67,9 @@ def read_epw(filename, coerce_year=None):
     ===============   ======  =========================================
 
 
-    =============================       ==============================================================================================================================================================
+    =============================       ==================================================================================================================================================================================================================  # noqa: E501
     EPWData field                       description
-    =============================       ==============================================================================================================================================================
+    =============================       ==================================================================================================================================================================================================================
     index                               A pandas datetime index. NOTE, times are set to local standard time (daylight savings is not included). Days run from 0-23h to comply with PVLIB's convention
     year                                Year, from original EPW file. Can be overwritten using coerce function.
     month                               Month, from original EPW file
@@ -99,14 +99,14 @@ def read_epw(filename, coerce_year=None):
     ceiling_height                      Height of cloud base above local terrain (7777=unlimited), meter
     present_weather_observation         Indicator for remaining fields: If 0, then the observed weather codes are taken from the following field. If 9, then missing weather is assumed.
     present_weather_codes               Present weather code, see [1], chapter 2.9.1.28
-    precipitable_water                  Total precipitable water contained in a column of unit cross section from earth to top of atmosphere, cm
+    precipitable_water                  Total precipitable water contained in a column of unit cross section from earth to top of atmosphere, cm. Note that some old *_TMY3.epw files may have incorrect unit if it was retrieved from www.energyplus.net.
     aerosol_optical_depth               The broadband aerosol optical depth per unit of air mass due to extinction by aerosol component of atmosphere, unitless
     snow_depth                          Snow depth in centimeters on the day indicated, (999 = missing data)
     days_since_last_snowfall            Number of days since last snowfall (maximum value of 88, where 88 = 88 or greater days; 99 = missing data)
     albedo                              The ratio of reflected solar irradiance to global horizontal irradiance, unitless
     liquid_precipitation_depth          The amount of liquid precipitation observed at indicated time for the period indicated in the liquid precipitation quantity field, millimeter
     liquid_precipitation_quantity       The period of accumulation for the liquid precipitation depth field, hour
-    =============================       ==============================================================================================================================================================
+    =============================       ==================================================================================================================================================================================================================
 
     References
     ----------


Test patch:
diff --git a/pvlib/test/test_atmosphere.py b/pvlib/test/test_atmosphere.py
index ee4b17a..0bf7434 100644
--- a/pvlib/test/test_atmosphere.py
+++ b/pvlib/test/test_atmosphere.py
@@ -128,6 +128,27 @@ def test_first_solar_spectral_correction_ambiguous():
         atmosphere.first_solar_spectral_correction(1, 1)
 
 
+def test_first_solar_spectral_correction_range():
+    with pytest.warns(UserWarning, match='Exceptionally high pw values'):
+        out = atmosphere.first_solar_spectral_correction(np.array([.1, 3, 10]),
+                                                         np.array([1, 3, 5]),
+                                                         module_type='monosi')
+    expected = np.array([0.96080878, 1.03055092,        nan])
+    assert_allclose(out, expected, atol=1e-3)
+    with pytest.warns(UserWarning, match='Exceptionally high pw values'):
+        out = atmosphere.first_solar_spectral_correction(6, 1.5, max_pw=5,
+                                                         module_type='monosi')
+    with pytest.warns(UserWarning, match='Exceptionally low pw values'):
+        out = atmosphere.first_solar_spectral_correction(np.array([0, 3, 8]),
+                                                         np.array([1, 3, 5]),
+                                                         module_type='monosi')
+    expected = np.array([0.96080878, 1.03055092, 1.04932727])
+    assert_allclose(out, expected, atol=1e-3)
+    with pytest.warns(UserWarning, match='Exceptionally low pw values'):
+        out = atmosphere.first_solar_spectral_correction(0.2, 1.5, min_pw=1,
+                                                         module_type='monosi')
+
+
 def test_kasten96_lt():
     """Test Linke turbidity factor calculated from AOD, Pwat and AM"""
     amp = np.array([1, 3, 5])