Repository: pvlib/pvlib-python

Issue:
spectral_factor_firstsolar changes user input data
**Is your feature request related to a problem? Please describe.**
[ `pvlib.spectrum.spectral_factor_firstsolar`](https://pvlib-python.readthedocs.io/en/stable/_modules/pvlib/spectrum/mismatch.html#spectral_factor_firstsolar) contains data screens for `precipitable_water` and `airmass_absolute` that change input data. Max/min values can be set for `precipitable_water` by the user but not `airmass_absolute`.

I see two issues:

1. Changing data in any circumstance is a serious question, but in this case specifically I do not think it is pvlib's job to change user data.
2. User control and the function's flexibility is impeded by the absence of module parameters to control min/max `airmass_absolute` .

**Describe the solution you'd like**
I can think of several potential solutions but would like **community feedback on what should be implemented**. Based on the feedback, I can create a PR with proposed changes. Current ideas:

1. Do nothing.
2. **Remove all data screens** for `airmass_absolute` and `precipitable_water`
3. **Retain data screens** but replace actions to change user data with **only warnings** to the user that the input data exceeds the thresholds, and these thresholds can either be:
- Defined by the user, or
- Defined by the developer
4. Add **new arguments** (`max_airmass_absolute` and `min_airmass_absolute`) to  `pvlib.spectrum.spectral_factor_firstsolar` so that the user can define min/max values for `airmass_absolute` in the same way  they can define `max_precipitable_water` and  `min_precipitable_water`, and their input **data will be changed** accordingly.
5. Something else?

**Additional context**
I understand this issue touches on a question about the role of pvlib as a toolbox and to what extent it should hold the user's hand through the application of its functions. I touched on the subject in [this informal blog post](https://rdaxini.github.io/posts/pvlib_purpose/) where I used the same spectral factor function as an example. At the very least, I don't think we should be changing user data without letting the user control this change. Going a step further, I don't think this function should include within it an action to change user data. It's job is to calculate the spectral mismatch factor given a set of `airmass_absolute` and `precipitable_water` values. I see room for including warnings to advise the user.


Discussion:
> that change input data.

Can you clarify what you mean by this? To my knowledge, pvlib is not changing any user data. The test below demonstrates that pvlib does not change the input `pw` data. Rather, it applies a minimum value before passing the data on to the polynomial (aside: I think the choice of "replaced" in the warning message is not ideal).

```python
In [8]: s = pd.Series([0, 1])

In [9]: pvlib.spectrum.mismatch.spectral_factor_firstsolar(s, 1, module_type='cdte')
/Users/holmgren/git_repos/pvlib-python/pvlib/spectrum/mismatch.py:359: UserWarning: Exceptionally low pw values replaced with 0.1 cm to prevent model divergence
  warn('Exceptionally low pw values replaced with '
Out[9]: array([0.93459226, 0.9905102 ])

In [10]: s
Out[10]:
0    0
1    1
dtype: int64
```

Can you please provide an example of pvlib changing user data? Or are we only talking about pvlib restricting the valid input data to the polynomial? Note that First Solar contributed the pvlib implementation and considers this restriction to be part of the model.

I'm not opposed to adding similar limits parameters for airmass.
@RDaxini  correlations like firstsolar spectral function frequently have narrow bounds in which they are valid. The documentation states over what range the correlations are valid. I think a faithful implementation should enforce these limits and not allow extrapolation
Thanks both for the comments. I'm lumping my responses together there's some overlap.

@mikofski For sure there are published limits, but the limits in the [publication](https://doi.org/10.1109/PVSC.2016.7749836) and [docs](https://pvlib-python.readthedocs.io/en/v0.10.5/reference/generated/pvlib.spectrum.spectral_factor_firstsolar.html) do not match the pvlib implementation. For example, airmass between 0 and 10 in pvlib but 0 and 5 in the paper/docs. Is there another document that justifies the implemented range? Then if any limits/restrictions are to be enforced, I'm not convinced calculating a mismatch value based on a value the user did not enter is the best enforcement. @wholmgren mentioned First Solar considers the limits part of the model, which I understand, but do they advise on how inputs outside of the prescribed range should be handled? At least with pw inputs exceeding `max_precipitable_water`, these default to `np.nan` --- this seems like a better approach if the consensus is that the computation should not take place with anything other than values in the specified range.

@wholmgren  Ah, I did not mean the variable is redefined. I meant the input is changed for the calculation of mismatch, ie mismatch is calculated for an input that differs from the user input. In the case of `pw`, the user is warned and has the option to change the limits. In the case of airmass, the user is not warned and is unable to change the limits.

Eg1: Warning, `M` calculated from default pw `0.1` rather than user input `0`:
```
In [28]: s = pd.Series([0, 0.1])

In [29]: M = pv.spectrum.mismatch.spectral_factor_firstsolar(s, 1, module_type='cdte')
C:\Users\ezxrd3\Documents\GitHub\pvlib-python\pvlib\spectrum\mismatch.py:369: UserWarning: Exceptionally low pw values replaced with 0.1 cm to prevent model divergence
  warn('Exceptionally low pw values replaced with '

In [30]: M
Out[30]: array([0.93459226, 0.93459226])
```

Eg2: No warning, `M` calculated from default ama `10` rather than user input `11`, default cannot be changed:

```
In [34]: a = pd.Series([11, 10])

In [35]: M = pv.spectrum.mismatch.spectral_factor_firstsolar(0.5, a, module_type='cdte')

In [36]: M
Out[36]: 
0    0.778779
1    0.778779
```

So, just my view, for sure the docs should explain the data ranges over which the model was developed, but currently I still lean more towards letting the user execute their own data filtering. I am not entirely opposed to retaining data screens in the function though, but then in this case I think the user should have control over the air mass limits (as with pw already) and be warned if they are triggered. What do you think?

Given the discrepancy between using a max airmass of 5 vs 10, I am in support of adding a ``max_airmass_absolute`` parameter.

First, please go through the discussion on GH of the initial addition of the function to see whether there is arguments for why the max airmass is 10 and not 5 as per the paper.

If you cannot find anything on this, then I suggest asking the authors. If it turns out we have not good reason for using 10 and not 5, then I suggest a deprecating cycle where we will change the default value ``max_airmass_absolute`` from 10 to 5. For an example on this, then see lies 351-357 of the changes in #2094.
> please go through the discussion on GH of the initial addition of the function

#208 

Looks to me that AM<10 is a choice that the author (M. Lee) made when implementing the model in code. Same limit is present in the Matlab function.
@RDaxini Could you paste the quote from the paper where a limit of 5 is described?
> @RDaxini Could you paste the quote from the paper where a limit of 5 is described?

![fs_am_pw](https://github.com/pvlib/pvlib-python/assets/143435106/206acedc-608c-4503-9599-df3d9def5f95)
Image 1 (above): pw and am filter requested @AdamRJensen (extracted from Section II of the manuscript)

-----------

Image 2 (below): just some other filters/assumptions in the paper, not sure if relevant but when talking about ensuring the implementation in pvlib is true to the original model, it got me thinking about what other limits there are. How do we decide which limits are implemented in pvlib? This is just a general question from me as a new contributor, I want to understand the thinking behind this a bit more
![fs_other](https://github.com/pvlib/pvlib-python/assets/143435106/120066dc-4c70-4cb0-b15e-b4b45c66dfb8)
@RDaxini I read it as simulations were done for 1 to 5 AM, and NOT as an explicit statement that the model is valid for 1 to 5 AM. Therefore, I think it's probably ok to keep the current upper limit of 10. However, I still think there's benefit to adding the parameter to the function.
The regression was performed on the simulations with AM<5. In M. Lee's judgement, its extrapolation to 5<AM<10 was considered reasonable. I don't know if the extrapolation is specifically described in the paper.

The second excerpt reads (to me) like a description of how they cleaned up measurements to either fit or verify the model. 
@AdamRJensen agreed, it does not state that it is a specific range of model validity, nowhere in the paper are there any specific statements pertaining to model validity within any particular range (as far as I see...).
@cwhanse unless I missed it, the extrapolation is not described in the paper (neither is the upper pw limit extrapolation)
So, I was not sure how these upper limits were decided or linked to the paper to be justified as part of the implementation.

Based on this discussion, I think for now I will create a simple PR to add the parameter into the function and revise the docs accordingly.

What are your thoughts on the handling of inputs exceeding the limits --- return `np.nan` as is currently the case for exceeding `max_precipitable_water`, or stick with setting the user input to equal the specified parameter value, as is currently the case with `min_precipitable_water`?
I vote 

> stick with setting the user input to equal the specified parameter value, as is currently the case with min_precipitable_water?
Created PR #2100 
Good and important observations and discussion. Recognizing and managing the limits of model validity and validation are high on my priority/wish list.  

The clipping of pwat at 0.1 (not 8.0) and airmass at 10 was also done in a spreadsheet FS once distributed, so it was presumably also done during model validation.  We don't know whether any data points were affected by pwat clipping in the validation data sets, so we don't know whether this behaviour is truly validated.  I would consider the clipping/extrapolation to be part of the model.





Patch:
diff --git a/docs/sphinx/source/whatsnew/v0.11.1.rst b/docs/sphinx/source/whatsnew/v0.11.1.rst
index 5ffb0e564a..4ccc9687c3 100644
--- a/docs/sphinx/source/whatsnew/v0.11.1.rst
+++ b/docs/sphinx/source/whatsnew/v0.11.1.rst
@@ -13,6 +13,9 @@ Enhancements
 * Add new losses function that accounts for non-uniform irradiance on bifacial
   modules, :py:func:`pvlib.bifacial.power_mismatch_deline`.
   (:issue:`2045`, :pull:`2046`)
+* Add new parameters for min/max absolute air mass to
+  :py:func:`pvlib.spectrum.spectral_factor_firstsolar`.
+  (:issue:`2086`, :pull:`2100`)
 
 
 Bug fixes
diff --git a/pvlib/spectrum/mismatch.py b/pvlib/spectrum/mismatch.py
index 197c226231..cab6084cac 100644
--- a/pvlib/spectrum/mismatch.py
+++ b/pvlib/spectrum/mismatch.py
@@ -358,21 +358,17 @@ def integrate(e):
 def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
                                module_type=None, coefficients=None,
                                min_precipitable_water=0.1,
-                               max_precipitable_water=8):
+                               max_precipitable_water=8,
+                               min_airmass_absolute=0.58,
+                               max_airmass_absolute=10):
     r"""
     Spectral mismatch modifier based on precipitable water and absolute
     (pressure-adjusted) air mass.
 
-    Estimates a spectral mismatch modifier :math:`M` representing the effect on
-    module short circuit current of variation in the spectral
-    irradiance. :math:`M`  is estimated from absolute (pressure currected) air
-    mass, :math:`AM_a`, and precipitable water, :math:`Pw`, using the following
-    function:
-
-    .. math::
-
-        M = c_1 + c_2 AM_a  + c_3 Pw  + c_4 AM_a^{0.5}
-            + c_5 Pw^{0.5} + c_6 \frac{AM_a} {Pw^{0.5}}
+    Estimates the spectral mismatch modifier, :math:`M`, representing the
+    effect of variation in the spectral irradiance on the module short circuit
+    current :math:`M`  is estimated from absolute (pressure-corrected) air
+    mass, :math:`AM_a`, and precipitable water, :math:`Pw`.
 
     Default coefficients are determined for several cell types with
     known quantum efficiency curves, by using the Simple Model of the
@@ -383,15 +379,13 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
     * :math:`0.5 \textrm{cm} <= Pw <= 5 \textrm{cm}`
     * :math:`1.0 <= AM_a <= 5.0`
     * Spectral range is limited to that of CMP11 (280 nm to 2800 nm)
-    * spectrum simulated on a plane normal to the sun
+    * Spectrum simulated on an equatorial facing surface with 37° tilt
     * All other parameters fixed at G173 standard
 
-    From these simulated spectra, M is calculated using the known
+    From these simulated spectra, :math:`M` is calculated using the known
     quantum efficiency curves. Multiple linear regression is then
-    applied to fit Eq. 1 to determine the coefficients for each module.
-
-    Based on the PVLIB Matlab function ``pvl_FSspeccorr`` by Mitchell
-    Lee and Alex Panchula of First Solar, 2016 [2]_.
+    applied to fit Eq. 1 to determine the coefficients for each module. More
+    details on the model can be found in [2]_.
 
     Parameters
     ----------
@@ -406,11 +400,12 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
         'multisi', and 'polysi' (can be lower or upper case). If provided,
         module_type selects default coefficients for the following modules:
 
-        * 'cdte' - First Solar Series 4-2 CdTe module.
-        * 'monosi', 'xsi' - First Solar TetraSun module.
-        * 'multisi', 'polysi' - anonymous multi-crystalline silicon module.
-        * 'cigs' - anonymous copper indium gallium selenide module.
-        * 'asi' - anonymous amorphous silicon module.
+        * ``'cdte'`` - First Solar Series 4-2 CdTe module.
+        * ``'monosi'``, ``'xsi'`` - First Solar TetraSun module.
+        * ``'multisi'``, ``'polysi'`` - anonymous multi-crystalline silicon
+        module.
+        * ``'cigs'`` - anonymous copper indium gallium selenide module.
+        * ``'asi'`` - anonymous amorphous silicon module.
 
         The module used to calculate the spectral correction
         coefficients corresponds to the Multi-crystalline silicon
@@ -430,12 +425,20 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
     min_precipitable_water : float, default 0.1
         minimum atmospheric precipitable water. Any ``precipitable_water``
         value lower than ``min_precipitable_water``
-        is set to ``min_precipitable_water`` to avoid model divergence. [cm]
+        is set to ``min_precipitable_water``. [cm]
 
     max_precipitable_water : float, default 8
         maximum atmospheric precipitable water. Any ``precipitable_water``
         value greater than ``max_precipitable_water``
-        is set to ``np.nan`` to avoid model divergence. [cm]
+        is set to ``np.nan``. [cm]
+
+    min_airmass_absolute : float, default 0.58
+        minimum absolute airmass. Any ``airmass_absolute`` value lower than
+        ``min_airmass_absolute`` is set to ``min_airmass_absolute``. [unitless]
+
+    max_airmass_absolute : float, default 10
+        minimum absolute airmass. Any ``airmass_absolute`` value greater than
+        ``max_airmass_absolute`` is set to ``max_airmass_absolute``. [unitless]
 
     Returns
     -------
@@ -445,6 +448,22 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
         effective irradiance, i.e., the irradiance that is converted to
         electrical current.
 
+    Notes
+    ----
+    The ``spectral_factor_firstsolar`` model takes the following form:
+
+    .. math::
+
+        M = c_1 + c_2 AM_a  + c_3 Pw  + c_4 AM_a^{0.5}
+            + c_5 Pw^{0.5} + c_6 \frac{AM_a} {Pw^{0.5}}.
+
+    The default values for the limits applied to :math:`AM_a` and :math:`Pw`
+    via the ``min_precipitable_water``, ``max_precipitable_water``,
+    ``min_airmass_absolute``, and ``max_airmass_absolute`` are set to prevent
+    divergence of the model presented above. These default values were
+    determined by the publication authors in the original pvlib-python
+    implementation (:pull:`208`).
+
     References
     ----------
     .. [1] Gueymard, Christian. SMARTS2: a simple model of the atmospheric
@@ -461,36 +480,27 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
        MMF Approach, TUV Rheinland Energy GmbH report 21237296.003,
        January 2017
     """
-
-    # --- Screen Input Data ---
-
-    # *** Pw ***
-    # Replace Pw Values below 0.1 cm with 0.1 cm to prevent model from
-    # diverging"
     pw = np.atleast_1d(precipitable_water)
     pw = pw.astype('float64')
     if np.min(pw) < min_precipitable_water:
         pw = np.maximum(pw, min_precipitable_water)
-        warn('Exceptionally low pw values replaced with '
-             f'{min_precipitable_water} cm to prevent model divergence')
+        warn('Low precipitable water values replaced with '
+             f'{min_precipitable_water} cm in the calculation of spectral '
+             'mismatch.')
 
-    # Warn user about Pw data that is exceptionally high
     if np.max(pw) > max_precipitable_water:
         pw[pw > max_precipitable_water] = np.nan
-        warn('Exceptionally high pw values replaced by np.nan: '
-             'check input data.')
-
-    # *** AMa ***
-    # Replace Extremely High AM with AM 10 to prevent model divergence
-    # AM > 10 will only occur very close to sunset
-    if np.max(airmass_absolute) > 10:
-        airmass_absolute = np.minimum(airmass_absolute, 10)
-
-    # Warn user about AMa data that is exceptionally low
-    if np.min(airmass_absolute) < 0.58:
-        warn('Exceptionally low air mass: ' +
-             'model not intended for extra-terrestrial use')
-        # pvl_absoluteairmass(1,pvl_alt2pres(4340)) = 0.58 Elevation of
+        warn('High precipitable water values replaced with np.nan in '
+             'the calculation of spectral mismatch.')
+
+    airmass_absolute = np.minimum(airmass_absolute, max_airmass_absolute)
+
+    if np.min(airmass_absolute) < min_airmass_absolute:
+        airmass_absolute = np.maximum(airmass_absolute, min_airmass_absolute)
+        warn('Low airmass values replaced with 'f'{min_airmass_absolute} in '
+             'the calculation of spectral mismatch.')
+        # pvlib.atmosphere.get_absolute_airmass(1,
+        # pvlib.atmosphere.alt2pres(4340)) = 0.58 Elevation of
         # Mina Pirquita, Argentian = 4340 m. Highest elevation city with
         # population over 50,000.
 
@@ -519,7 +529,6 @@ def spectral_factor_firstsolar(precipitable_water, airmass_absolute,
         raise TypeError('Cannot resolve input, must supply only one of ' +
                         'module_type and coefficients')
 
-    # Evaluate Spectral Shift
     coeff = coefficients
     ama = airmass_absolute
     modifier = (
@@ -640,8 +649,8 @@ def spectral_factor_caballero(precipitable_water, airmass_absolute, aod500,
         One of the following PV technology strings from [1]_:
 
         * ``'cdte'`` - anonymous CdTe module.
-        * ``'monosi'``, - anonymous sc-si module.
-        * ``'multisi'``, - anonymous mc-si- module.
+        * ``'monosi'`` - anonymous sc-si module.
+        * ``'multisi'`` - anonymous mc-si- module.
         * ``'cigs'`` - anonymous copper indium gallium selenide module.
         * ``'asi'`` - anonymous amorphous silicon module.
         * ``'perovskite'`` - anonymous pervoskite module.
@@ -755,8 +764,8 @@ def spectral_factor_pvspec(airmass_absolute, clearsky_index,
 
         * ``'fs4-1'`` - First Solar series 4-1 and earlier CdTe module.
         * ``'fs4-2'`` - First Solar 4-2 and later CdTe module.
-        * ``'monosi'``, - anonymous monocrystalline Si module.
-        * ``'multisi'``, - anonymous multicrystalline Si module.
+        * ``'monosi'`` - anonymous monocrystalline Si module.
+        * ``'multisi'`` - anonymous multicrystalline Si module.
         * ``'cigs'`` - anonymous copper indium gallium selenide module.
         * ``'asi'`` - anonymous amorphous silicon module.
 


Test patch:
diff --git a/pvlib/tests/test_spectrum.py b/pvlib/tests/test_spectrum.py
index 969fb819b8..09b24866ea 100644
--- a/pvlib/tests/test_spectrum.py
+++ b/pvlib/tests/test_spectrum.py
@@ -264,6 +264,22 @@ def test_spectral_factor_firstsolar_supplied():
     assert_allclose(out, expected, atol=1e-3)
 
 
+def test_spectral_factor_firstsolar_large_airmass_supplied_max():
+    # test airmass > user-defined maximum is treated same as airmass=maximum
+    m_eq11 = spectrum.spectral_factor_firstsolar(1, 11, 'monosi',
+                                                 max_airmass_absolute=11)
+    m_gt11 = spectrum.spectral_factor_firstsolar(1, 15, 'monosi',
+                                                 max_airmass_absolute=11)
+    assert_allclose(m_eq11, m_gt11)
+
+
+def test_spectral_factor_firstsolar_large_airmass():
+    # test that airmass > 10 is treated same as airmass=10
+    m_eq10 = spectrum.spectral_factor_firstsolar(1, 10, 'monosi')
+    m_gt10 = spectrum.spectral_factor_firstsolar(1, 15, 'monosi')
+    assert_allclose(m_eq10, m_gt10)
+
+
 def test_spectral_factor_firstsolar_ambiguous():
     with pytest.raises(TypeError):
         spectrum.spectral_factor_firstsolar(1, 1)
@@ -276,36 +292,34 @@ def test_spectral_factor_firstsolar_ambiguous_both():
         spectrum.spectral_factor_firstsolar(1, 1, 'cdte', coefficients=coeffs)
 
 
-def test_spectral_factor_firstsolar_large_airmass():
-    # test that airmass > 10 is treated same as airmass==10
-    m_eq10 = spectrum.spectral_factor_firstsolar(1, 10, 'monosi')
-    m_gt10 = spectrum.spectral_factor_firstsolar(1, 15, 'monosi')
-    assert_allclose(m_eq10, m_gt10)
-
-
 def test_spectral_factor_firstsolar_low_airmass():
-    with pytest.warns(UserWarning, match='Exceptionally low air mass'):
+    m_eq58 = spectrum.spectral_factor_firstsolar(1, 0.58, 'monosi')
+    m_lt58 = spectrum.spectral_factor_firstsolar(1, 0.1, 'monosi')
+    assert_allclose(m_eq58, m_lt58)
+    with pytest.warns(UserWarning, match='Low airmass values replaced'):
         _ = spectrum.spectral_factor_firstsolar(1, 0.1, 'monosi')
 
 
 def test_spectral_factor_firstsolar_range():
-    with pytest.warns(UserWarning, match='Exceptionally high pw values'):
-        out = spectrum.spectral_factor_firstsolar(np.array([.1, 3, 10]),
-                                                  np.array([1, 3, 5]),
-                                                  module_type='monosi')
+    out = spectrum.spectral_factor_firstsolar(np.array([.1, 3, 10]),
+                                              np.array([1, 3, 5]),
+                                              module_type='monosi')
     expected = np.array([0.96080878, 1.03055092, np.nan])
     assert_allclose(out, expected, atol=1e-3)
-    with pytest.warns(UserWarning, match='Exceptionally high pw values'):
+    with pytest.warns(UserWarning, match='High precipitable water values '
+                      'replaced'):
         out = spectrum.spectral_factor_firstsolar(6, 1.5,
                                                   max_precipitable_water=5,
                                                   module_type='monosi')
-    with pytest.warns(UserWarning, match='Exceptionally low pw values'):
+    with pytest.warns(UserWarning, match='Low precipitable water values '
+                      'replaced'):
         out = spectrum.spectral_factor_firstsolar(np.array([0, 3, 8]),
                                                   np.array([1, 3, 5]),
                                                   module_type='monosi')
     expected = np.array([0.96080878, 1.03055092, 1.04932727])
     assert_allclose(out, expected, atol=1e-3)
-    with pytest.warns(UserWarning, match='Exceptionally low pw values'):
+    with pytest.warns(UserWarning, match='Low precipitable water values '
+                      'replaced'):
         out = spectrum.spectral_factor_firstsolar(0.2, 1.5,
                                                   min_precipitable_water=1,
                                                   module_type='monosi')