Repository: obspy/obspy

Issue:
polarization.vidale_adapt: possible mis-ordered complex covariance matrix elements
Obspy version:1.1.0, python version: 3.7

Two elements of the complex covariance matrix in obspy.signal.polarization.vidale_adapt are possibly mis-ordered.

Lines 270 and 271 of obspy.signal.polarization.py are currently
```
covmat[1][2] = np.dot(zx, nx.conjugate())
covmat[2][1] = covmat[1][2].conjugate()
```

Suggest it should be:
```
covmat[1][2] = np.dot(nx, zx.conjugate())
covmat[2][1] = covmat[1][2].conjugate()
```

Tests suggest that current implementation produces significantly noisier smallest eigenvalues than suggested fix.

[Apologies if I've not formatted this issue correctly, or mis-understood the code; first attempt at raising an issue - please let me know if I should be doing something else].



Discussion:
@jwassermann ?
Hm, needs a closer look

Von meinem iPhone gesendet

> Am 21.02.2020 um 18:22 schrieb Thomas Lecocq <notifications@github.com>:
> 
> ﻿
> @jwassermann ?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.

A little more context if it helps.

Vidale (1986) Eqn 2 is:

![Cvidale](https://user-images.githubusercontent.com/20835956/75095690-7764cb00-558f-11ea-9699-15a9cf98d5d4.png)

I think the current obspy implementation is this:

![Cobspy](https://user-images.githubusercontent.com/20835956/75095696-83e92380-558f-11ea-97d3-4f2b9110b314.png)

The consequence for the output eigenvalues is demonstrated in the example below (an air-to-ground coupled wave recorded on a 3-component seismometer arrives at approx. sample 6000):

Current implementation:

![eigenval_test_current](https://user-images.githubusercontent.com/20835956/75095709-ab3ff080-558f-11ea-85a4-ad2c3b082fa0.png)

Result when suggested implementation made:

![eigenval_test_proposed](https://user-images.githubusercontent.com/20835956/75095716-bf83ed80-558f-11ea-9ac3-94f1e16f8271.png)

The largest effect (for the example above) is for the smallest eigenvalue (but there is a small effect on the intermediate eigenvalue as well). Therefore this will have consequences for the calculation of both rectilinearity (L.287) and planarity (L.289 of obspy/polarization.py).

I haven't investigated effect on eigenvectors.

Hope this is helpful.



Just checked and yes you are perfectly right - thanks for pointing this out. I changed the code already to use the same notation as the Flinn method (np.cov). In the light of  this I would also change the adaptive windowing to be selectable - sometimes the output of the Vidale with adaptive windowing looks very weird.In order to backward compatible the switch vor adaptive will be set to True
@megies makes that sense?
> In order to backward compatible the switch for adaptive will be set to True

Sounds good to me. So what are the exact changes to be done?
 - change `np.dot(zx, nx.conjugate())` as described in first post
 - make adaptive selectable (what part of the code exactly should be optional, wasn't immediately apparent from a quick glance at the code)

Anything else @jwassermann, @dngreen?
I have the code ready and will send it asap

Von meinem iPhone gesendet

> Am 25.02.2020 um 12:04 schrieb Tobias Megies <notifications@github.com>:
> 
> ﻿
> In order to backward compatible the switch for adaptive will be set to True
> 
> Sounds good to me. So what are the exact changes to be done?
> 
> change np.dot(zx, nx.conjugate()) as described in first post
> make adaptive selectable (what part of the code exactly should be optional, wasn't immediately apparent from a quick glance at the code)
> Anything else @jwassermann, @dngreen?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.


Resolving patch:
diff --git a/obspy/signal/polarization.py b/obspy/signal/polarization.py
--- a/obspy/signal/polarization.py
+++ b/obspy/signal/polarization.py
@@ -192,7 +192,8 @@ def instantaneous_frequency(data, sampling_rate):
     return instf
 
 
-def vidale_adapt(stream, noise_thres, fs, flow, fhigh, spoint, stime, etime):
+def vidale_adapt(stream, noise_thres, fs, flow, fhigh, spoint, stime, etime,
+                 adaptive=True):
     """
     Adaptive window polarization analysis after [Vidale1986]_ with the
     modification of adapted analysis window estimated by estimating the
@@ -217,6 +218,10 @@ def vidale_adapt(stream, noise_thres, fs, flow, fhigh, spoint, stime, etime):
     :type stime: :class:`~obspy.core.utcdatetime.UTCDateTime`
     :param etime: end time for the analysis
     :type etime: :class:`~obspy.core.utcdatetime.UTCDateTime`
+    :param adaptive: switch for adaptive window estimation (defaults to
+        ``True``). If set to ``False``, the window will be estimated as
+        ``3 * max(1/(fhigh-flow), 1/flow)``.
+    :type adaptive: bool
     :returns: list of tuples containing azimuth, incidence, rectilinearity,
         planarity, and ellipticity
     """
@@ -238,15 +243,18 @@ def vidale_adapt(stream, noise_thres, fs, flow, fhigh, spoint, stime, etime):
     offset = int(3 * fs / flow)
     covmat = np.zeros([3, 3], dtype=np.complex128)
     while True:
-        adapt = int(3. * w * fs / (zi[offset] + ni[offset] + ei[offset]))
-        # in order to account for errors in the inst freq estimation
-        if adapt > int(3 * fs / flow):
-            adapt = int(3 * fs / flow)
-        if adapt < int(3 * fs / fhigh):
-            adapt = int(3 * fs / fhigh)
-        # XXX: was adapt /= 2
-        adapt //= 2
-        adapt = (2 * adapt) + 1
+        if adaptive:
+            adapt = int(3.0 * w * fs / (zi[offset] + ni[offset] + ei[offset]))
+            # in order to account for errors in the inst freq estimation
+            if adapt > int(3.0 * fs / flow):
+                adapt = int(3.0 * fs / flow)
+            elif adapt < int(3.0 * fs / fhigh):
+                adapt = int(3.0 * fs / fhigh)
+            # XXX: was adapt /= 2
+            adapt //= 2
+            adapt = (2 * adapt) + 1
+        else:
+            adapt = max(int(3. * fs / (fhigh - flow)), int(3. * fs / flow))
         newstart = stime + offset / fs
         if (newstart + (adapt / 2) / fs) > etime:
             break
@@ -261,16 +269,17 @@ def vidale_adapt(stream, noise_thres, fs, flow, fhigh, spoint, stime, etime):
         nx -= nx.mean()
         ex -= ex.mean()
 
-        covmat[0][0] = np.dot(ex, ex.conjugate())
-        covmat[0][1] = np.dot(ex, nx.conjugate())
-        covmat[1][0] = covmat[0][1].conjugate()
-        covmat[0][2] = np.dot(ex, zx.conjugate())
-        covmat[2][0] = covmat[0][2].conjugate()
-        covmat[1][1] = np.dot(nx, nx.conjugate())
-        covmat[1][2] = np.dot(zx, nx.conjugate())
-        covmat[2][1] = covmat[1][2].conjugate()
-        covmat[2][2] = np.dot(zx, zx.conjugate())
-
+        mask = (stream[0][:] ** 2 + stream[1][:] ** 2 + stream[2][:] ** 2) > \
+            noise_thres
+        xx = np.zeros((3, mask.sum()), dtype=np.complex128)
+        # East
+        xx[0, :] = ea
+        # North
+        xx[1, :] = na
+        # Z
+        xx[2, :] = za
+
+        covmat = np.cov(xx)
         eigvec, eigenval, v = np.linalg.svd(covmat)
 
         # very similar to function flinn, possible could be unified
@@ -305,7 +314,6 @@ def fun(x):
                 azimuth += 180.0
         if azimuth > 180.0:
             azimuth -= 180.0
-
         res.append((newstart.timestamp, azimuth, incidence, rect, plan, ellip))
         offset += 1
 
@@ -433,7 +441,8 @@ def _get_s_point(stream, stime, etime):
 
 
 def polarization_analysis(stream, win_len, win_frac, frqlow, frqhigh, stime,
-                          etime, verbose=False, method="pm", var_noise=0.0):
+                          etime, verbose=False, method="pm", var_noise=0.0,
+                          adaptive=True):
     """
     Method carrying out polarization analysis with the [Flinn1965b]_,
     [Jurkevics1988]_, ParticleMotion, or [Vidale1986]_ algorithm.
@@ -458,6 +467,10 @@ def polarization_analysis(stream, win_len, win_frac, frqlow, frqhigh, stime,
     :param method: the method to use. one of ``"pm"``, ``"flinn"`` or
         ``"vidale"``.
     :type method: str
+    :param adaptive: switch for adaptive window estimation (defaults to
+        ``True``). If set to ``False``, the window will be estimated as
+        ``3 * max(1/(fhigh-flow), 1/flow)``.
+    :type adaptive: bool
     :rtype: dict
     :returns: Dictionary with keys ``"timestamp"`` (POSIX timestamp, can be
         used to initialize :class:`~obspy.core.utcdatetime.UTCDateTime`


Test patch: