Repository: pvlib/pvlib-python

Issue:
Simple closed-form solution for alpha in detect_clearsky - clarified
### Discussed in https://github.com/pvlib/pvlib-python/discussions/2171

<div type='discussions-op-text'>

<sup>Originally posted by **agodbehere** August 20, 2024</sup>
The algorithm in clearsky.detect_clearsky optimizes for a scaling parameter alpha by finding a value that minimizes the root mean-squared error (RMSE) between measurements and a scaled clear-sky model. Currently, the implementation computes the RMSE and uses scipy.optimize to iteratively minimize the function. However, a simple closed-form solution exists, bypassing the need to call scipy.optimize.

The objective function is $f(\alpha) = \sqrt{\frac{\sum_i (G_i - \alpha C_i)^2}{n}}$, where $G$ represents measurements of GHI and $C$ represents the clear-sky model.

Since $f(\alpha)$ is nonnegative, $\arg\min_\alpha f(\alpha) = \arg\min_\alpha f^2(\alpha)$. The value $n$ is a constant and similarly does not affect the solution. 

Thus, we end up with a modified objective function $\hat{f}(\alpha) = \sum_i (G_i-\alpha C_i)^2$. Solving for $\frac{d\hat{f}}{d\alpha} = 0$ yields the optimal value: $\hat{\alpha} = \frac{\sum_i G_iC_i}{\sum_i C_i^2}$. The result is nonnegative, so the constraint $\alpha > 0$ is satisfied implicitly. 
</div>


Discussion:

Patch:
diff --git a/docs/sphinx/source/whatsnew/v0.11.1.rst b/docs/sphinx/source/whatsnew/v0.11.1.rst
index 7c8f01743e..afd1cc7215 100644
--- a/docs/sphinx/source/whatsnew/v0.11.1.rst
+++ b/docs/sphinx/source/whatsnew/v0.11.1.rst
@@ -30,6 +30,10 @@ Enhancements
 * Added function for calculating wind speed at different heights,
   :py:func:`pvlib.atmosphere.windspeed_powerlaw`.
   (:issue:`2118`, :pull:`2124`)
+* Implemented closed-form solution for alpha in :py:func:`pvlib.clearsky.detect_clearsky`, 
+  obviating the call to scipy.optimize that was prone to runtime errors and minimizing
+  computation. (:issue:`2171`, :issue:`2216`, :pull:`2217`).
+
 
 Bug fixes
 ~~~~~~~~~
@@ -74,3 +78,4 @@ Contributors
 * Marcos R. Escudero (:ghuser:`marc-resc`)
 * Bernat Nicolau (:ghuser:`BernatNicolau`)
 * Eduardo Sarquis (:ghuser:`EduardoSarquis`)
+* Andrew B Godbehere (:ghuser:`agodbehere`)
\ No newline at end of file
diff --git a/pvlib/clearsky.py b/pvlib/clearsky.py
index 62d83bc1f4..1a32b664ab 100644
--- a/pvlib/clearsky.py
+++ b/pvlib/clearsky.py
@@ -9,7 +9,6 @@
 
 import numpy as np
 import pandas as pd
-from scipy.optimize import minimize_scalar
 from scipy.linalg import hankel
 import h5py
 
@@ -874,25 +873,11 @@ def detect_clearsky(measured, clearsky, times=None, infer_limits=False,
         clear_meas = meas[clear_samples]
         clear_clear = clear[clear_samples]
 
-        def rmse(alpha):
-            return np.sqrt(np.mean((clear_meas - alpha*clear_clear)**2))
-
-        optimize_result = minimize_scalar(rmse)
-        if not optimize_result.success:
-            try:
-                message = "Optimizer exited unsuccessfully: " \
-                           + optimize_result.message
-            except AttributeError:
-                message = "Optimizer exited unsuccessfully: \
-                           No message explaining the failure was returned. \
-                           If you would like to see this message, please \
-                           update your scipy version (try version 1.8.0 \
-                           or beyond)."
-            raise RuntimeError(message)
-
-        else:
-            alpha = optimize_result.x
-
+        # Compute arg min of MSE between model and observations
+        C = (clear_clear**2).sum()
+        if not (pd.isna(C) or C == 0):  # safety check
+            # only update alpha if C is strictly positive
+            alpha = (clear_meas * clear_clear).sum() / C
         if round(alpha*10000) == round(previous_alpha*10000):
             break
     else:


Test patch:
diff --git a/pvlib/tests/test_clearsky.py b/pvlib/tests/test_clearsky.py
index 4e353e997c..3a014d9f9a 100644
--- a/pvlib/tests/test_clearsky.py
+++ b/pvlib/tests/test_clearsky.py
@@ -675,13 +675,18 @@ def test_detect_clearsky_missing_index(detect_clearsky_data):
 def test_detect_clearsky_not_enough_data(detect_clearsky_data):
     expected, cs = detect_clearsky_data
     with pytest.raises(ValueError, match='have at least'):
-       clearsky.detect_clearsky(expected['GHI'], cs['ghi'], window_length=60)
+        clearsky.detect_clearsky(expected['GHI'], cs['ghi'], window_length=60)
 
 
-def test_detect_clearsky_optimizer_failed(detect_clearsky_data):
+@pytest.mark.parametrize("window_length", [5, 10, 15, 20, 25])
+def test_detect_clearsky_optimizer_not_failed(
+    detect_clearsky_data, window_length
+):
     expected, cs = detect_clearsky_data
-    with pytest.raises(RuntimeError, match='Optimizer exited unsuccessfully'):
-        clearsky.detect_clearsky(expected['GHI'], cs['ghi'], window_length=15)
+    clear_samples = clearsky.detect_clearsky(
+        expected["GHI"], cs["ghi"], window_length=window_length
+    )
+    assert isinstance(clear_samples, pd.Series)
 
 
 @pytest.fixture