Repository: obspy/obspy

Issue:
Water level suppressing large parts of the spectrum in instrumental correction when derivation or integration is requested
### Avoid duplicates

- [X] I searched existing issues

### Bug Summary

We found a error with the instrumental correction tool of Obspy (any versions). The bug occur only with low waterlevel values (<80 dB) and when a change from velocity to acceleration or inversely is requested.

The waterlevel problems is observed at high frequency when the instrumental correction is applied to a velocimeter which has been changed to acceleration. The same problem is observed when an accelerogram is moved to velocity, but then at low frequency. It is due to the waterlevel value which is not change in the new domain (acceleration or velocity). You can find attached the detailed description of the problem.

Possible Obspy command line to generate the bug on the recording of a velocimeter:
waveform.remove_response(pre_filt =[0.001, 1/120, 95, 100], water_level=40, output=‘ACC’)
if you applied the previous instrumental correction to a velocimeter record, then compute its spectrum and divide it by the spectrum of the same recording instrumentally corrected with a waterlevel of 100 dB, you will see that spectral ratio is diverging from one at high frequency.
If you do the same without the output=‘ACC’ option, then the problem disappear.

The derivation or integration of the signal from velocity to acceleration (or integration from acceleration to velocity) should be performed after the instrumental correction and the application of the waterlevel !
[ProblemObspy_Waterlevel.pdf](https://github.com/obspy/obspy/files/9491855/ProblemObspy_Waterlevel.pdf)


### Code to Reproduce

```python
waveform.remove_response(pre_filt=None, water_level=40, output='ACC')
```


### Error Traceback

```Python traceback
No error occur unfortunately. Only the results are wrong, the instrumental correction provide bad amplitude...
```


### ObsPy Version?

Any version (here 1.3.0)

### Operating System?

Windows

### Python Version?

Python 3.9.12

### Installation Method?

conda


Discussion:
This issue has been mentioned on **ObsPy Forum**. There might be relevant details there:

https://discourse.obspy.org/t/issue-with-the-instrumental-correction/1487/3

Hmm.. after a first quick look at this, I don't think it's a bug, but rather water level mechanism probably not very well suited for the task when taking an instrument with flat response in velocity to acceleration (and comparable tasks). But maybe I'm not seeing through all aspects yet.

I think water level does exactly what it says it does, which is clipping the instrument response at a certain value relative to it's maximum.

In case of taking the seismometer to velocity:

![VEL](https://user-images.githubusercontent.com/1842780/189087944-a5b19a42-13cc-49e5-bd0c-9c4f438c1133.png)

In case of taking the seismometer to acceleration:

![ACC](https://user-images.githubusercontent.com/1842780/189087992-179b6968-e52d-4531-935c-c4e4714a6e43.png)

For operations that work on a response with a flat plateau, good choices of water level values  are more intuitive, while the setting described as problematic here is with the response curve in acceleration peaking and falling off rapidly on both sides, so that water level mechanism cuts in relentlessly in frequency ranges where it's not expected to hit that hard.

I think you should just use `water_level=None` and rely on `pre_filt` option, where you can directly specify the frequency range you want left alone from the stabilization/suppression.

It might be worth a thought to try and show a warning when the water level choice seems non ideal, but that might not be trivial to do, so not sure how or if to try that.

CCing @aringler-usgs who might have valuable comments on this issue
I think attempting to use `water_level` on instruments that are not flat in some region will be confusing at best.  Looking at @megies example suggests the code is doing what you would expect, but perhaps not what everyone wants.

Please do let me know if I am missing something and I can look into this further.
Thank for the reply to my input. I still believe that there is a signal processing issue here. The good way for providing to user the instrumental corrected and derive to acceleration signal from a BB sensor would be to:
1- first do the instrumental correction with waterlevel (on velocity signal for BB and acceleration signal for accelerometers)
2- then compute the derivation from velocity to acceleration (or reversely integrate from acceleration to velocity).

I can for sure avoid the problem by setting the waterlevel to 'none' or the output correctly. However, this issue is not documented and many people I know have had bad amplitude of their signal because of that without identifying the origin of the problem.
When using that function in Obspy, users will suppose that the instrumental correction are done correctly and that the waterlevel is applied (to avoid division by 0 as it was made for) before the derivation/integration. They cannot imagine that this waterlevel is applied after the derivation/integration with the rotated signal (which is the result of derivating/integrating a signal). Of course if the derivation/integration is performed before (as here) then the waterlevel cross the flat plateau parts of the instrumental response and the corrected amplitude of the signal is wrong (see my PDF or @megies plots). But it is just not the good way to do it...
It is a important issue because many researcher are working with Obspy and none of them are aware of this. It has probably already, and it will, lead to error in published studies...
Thanks for the additional clarification @vpdmtb.  I ran this by @dwilson-usgs to verify and we think this isn't a bug, but perhaps, as you say, an issue with people not understanding how the response removal operation works.  The `remove_response` call first grabs the evalresp response [code](https://github.com/obspy/obspy/blob/1c656830a33e72b7d335f01d28c38b97d8c4d58a/obspy/core/trace.py#L2674) in the units being converted to and then applies the `water_level`: [here](https://github.com/obspy/obspy/blob/1c656830a33e72b7d335f01d28c38b97d8c4d58a/obspy/core/trace.py#L2945).

The response of the instrument can be described in ways that do not make it flat in a particular band (WWSSN instruments are always described in terms of displacement).  So I don't think you would want to change how the `water_level` is applied or deciding which units the response is described in compared to how you remove the response.  Are you purposing some additional verbiage in the documentation that suggests researchers should be careful with the units being used/response removal/water level?   You can run into other problems that are not easily tracked by doing thing like integrating/differentiating before/after response removal.  For example, you can elevate the long-period noise in spectra by not rotating the response to acceleration and instead removing the response in velocity and then differentiating.

I am all for making this more accessible and easier for researchers to understand to avoid future errors, but I do think that the burden falls on the researcher to understand what the code is doing and to apply it correctly.  Would adding some verbiage about your example to the documentation help avoid this in the future?
@aringler-usgs  I guess you are right that performing the instrumental correction before the derivation/integration can lead to new and unexpected error. However, I do believe that additional information like a warning would be very beneficial to the user. It could be good to write somewhere in the documentation something like: "Warning: when the **output** option of the **remove_response** function differ from "DEF" (default value), then applying a **water_level** can lead to amplitude error in the corrected signals. In that case, it is strongly recommended not to define any **water_level** (**water_level = None**)."
For sure it would not hurt to add a warning message, but it might be tricky to get this right. The main thing that comes to mind is look at input units of first response stage and compare that to the requested output units and show a warning if these mismatch.
This might catch many cases but surely not all.



> when the **output** option of the **remove_response** function differ from "DEF" (default value)

Unfortunately our default isn't "DEF" but "VEL" so we can't use that to decide when to trigger a warning.
Do you think something like this could be added to the documentation instead of an actual warning?  While the warnings could be suppressed, I would prefer not to see these warnings when I am doing things like going from counts to acceleration on a broadband etc.

However, I don't know how often people do things like this without knowing the details.  I am happy to make a pull request if it would be helpful.
@aringler-usgs Yes that would be great to have it written in the documentation ! Please proceed to the pull request if you can because I am not very familiar with github...
> I don't know how often people do things like this without knowing the details

I think many people use `remove_response` without being intimately aware of what exactly is going on in the background and all the caveats in there. But that's something that can't really be helped.



> Do you think something like this could be added to the documentation instead of an actual warning? While the warnings could be suppressed, I would prefer not to see these warnings when I am doing things like going from counts to acceleration on a broadband etc.

Well, I meant showing a warning in that case only when `water_level` isn't `None`. You probably are solely using `pre_filt` when bringing a flat velocity response type instrument to acceleration.
In any case, I'm OK with just adding a note to docs too. Warnings are good in principle, but most people just ignore em anyway, is my feeling. :stuck_out_tongue_winking_eye: 
I will go ahead and update the documentation and make a pull request and you both can see what you think.  I might not get to it until next week.

Resolving patch:
diff --git a/obspy/core/trace.py b/obspy/core/trace.py
--- a/obspy/core/trace.py
+++ b/obspy/core/trace.py
@@ -2692,10 +2692,21 @@ def remove_response(self, inventory=None, output="VEL", water_level=60,
         (specifying the four corner frequencies of the frequency taper as a
         tuple in `pre_filt`).
 
+        .. warning::
+            The water level approach can lead to unexpected results that
+            strongly suppress valid/wanted parts of the spectrum if the
+            requested output unit is not the native quantity of the instrument,
+            i.e. the instrument response is not flat for that quantity (e.g.
+            requesting output ``"VEL"`` for an accelerometer). For details see
+            https://github.com/obspy/obspy/issues/3136.
+            In this case it might be better to set ``water_level=None`` and use
+            ``pre_filt`` option instead.
+
         .. note::
 
             Any additional kwargs will be passed on to
-            :meth:`obspy.core.inventory.response.Response.get_evalresp_response`,
+            :meth:`Response.get_evalresp_response()
+            <obspy.core.inventory.response.Response.get_evalresp_response>`,
             see documentation of that method for further customization (e.g.
             start/stop stage and hiding overall sensitivity mismatch warning).
 
@@ -2706,8 +2717,8 @@ def remove_response(self, inventory=None, output="VEL", water_level=60,
             :meth:`~obspy.core.trace.Trace.simulate` with the identical
             response provided as
             a (dataless) SEED or RESP file and when using the same
-            `water_level` and `pre_filt` (and options `sacsim=True` and
-            `pitsasim=False` which influence very minor details in detrending
+            ``water_level`` and ``pre_filt`` (and options ``sacsim=True`` and
+            ``pitsasim=False`` which influence very minor details in detrending
             and tapering).
 
         .. rubric:: Example


Test patch: