Repository: obspy/obspy

Issue:
Trace.copy: Using deepcopy causes derived Stats keys setting parent keys with computed values (leading to wrong values in edge cases)
Hi everybody,

First, I hope all of you are happy and healthy.

Second, I've got a puzzling issue when working with highly-sampled data:

-  ObsPy version: 1.2.1 (anaconda)
-  Python version: 3.7
-  Arch Linux

Working with sampling rates >= 100000. Hz produces unexpected behavior.

```
tr = Trace(data=np.ones(100000), header={'sampling_rate': 100000.})
print(tr.stats.sampling_rate)
100000.0
print(tr.copy().stats.sampling_rate)
99999.99999999999
```
I think this is a result of floating point ["representation error"](https://docs.python.org/3/tutorial/floatingpoint.html), whenever the Trace AttribDict needs to be re-evaluated, [specifically where `delta` is involved](https://github.com/obspy/obspy/blob/88687a146a7c3ca8f35c608db2243cf5fed6813c/obspy/core/trace.py#L177-L182).

Not sure how to go about addressing this though...



Discussion:
Changing [this](https://github.com/obspy/obspy/blob/88687a146a7c3ca8f35c608db2243cf5fed6813c/obspy/core/trace.py#L180) line so that it forces a rounded float:

`value = float(round(1.0 / float(value)))`

fixes my issue but would obviously break anything with a non-integer sampling rate.

Are there use-cases where decimal sampling rates are required? If so, I'm open to suggestions as to how to accommodate >100000 Hz sampling rates without rounding. Otherwise, I'm happy to submit a PR with the above change (including a little test if that's necessary?).


Hi @cjhopp,

Interesting issue! Unfortunately your problem involves parts of ObsPy that, if changed, could have huge implications for most users' codes so we need to be very very careful.

I don't think rounding to an integer value is a good idea. I have personally worked with some obscure digitizers that sample at non integer intervals and we don't want to (retroactively) preclude their use. 

The only potential path forward (which I can think of) is to leave current support for floats alone and somehow support integer representations as well. Off the top of my head, maybe stats could avoid changing `sampling_rate` datatypes at all? You could then get int or float values depending on what was provided.

Another approach might be to support `numpy.timedelta64` in some fashion for `time_delta` so a very precise sampling_period can be specified, even if the complementary `sampling_rate` is not a whole number when expressed in Hz.

Hi @d-chambers,

Thanks for the guidance! I figured there must have been a need for float values. I've just never encountered such a thing.

I like the idea of returning the type that the user provides. This might still have implications for codes where `sampling_rate` is provided as an integer on creation of the `Trace`, but then assumed to be a float for later operations?

The use case that brought me to this issue is a sanity check in the depths of `eqcorrscan`:
```
if tr.stats.sampling_rate * length != tr.stats.npts:
```
where `length` is some number of seconds. Here, the assumption is that the integer `npts` is a multiple of `sampling_rate`, which isn't so for the example above. Perhaps on the user end it's better not to assume `sampling_rate` is a whole number, but I expect these sorts of statements that assume it _is_ are fairly ubiquitous in users codebases.

So I think integer support for `sampling_rate` is a good idea, maybe also with a warning in cases where the user provides `sampling_rate >= 100000.` as a float, stating that it may produce unexpected results?
Would it be easier to just loosen some of the eqcorrscan checks? Maybe something like this would work?

```python
if not np.isclose(np.round(tr.stats.sampling_rate * length), tr.stats.npts)
``` 

If you want to make a fix on the ObsPy level I would personally start by adding a few tests:

1. A test to make sure the sampling rate doesn't change when copying a trace (like in your example)

2. A test for sampling rates which aren't whole numbers

3. A test that int sampling rates stay ints after assignment 

I haven't looked at the `Stats` internals in awhile so I don't know how hard it will be, but I have seen enough of your code in eqcorrscan to know you are a very capable developer. Also, I am happy to help if you need it! :+1: 

I wouldn't recommend any new warnings though, as the amount of numerical precision accumulated (at least in your example) is going to be array length dependent, so there is not a magic number we can use to decide exactly when to raise a warning or not. 
It's certainly something I'll take up with @calum-chamberlain over at eqcorrscan.

But I do think it's something worth addressing at the obspy level, although I acknowledge that it's a very edge case at this point.

I'll take your test suggestions and see where I get in the next week or so!
@d-chambers @cjhopp so I briefly looked into this.. the problem seems to be that we are lazy and use `deepcopy` on the whole `Trace` and then when it copies `Stats` it actually sets all attributes in it, even some of the derived ones like `delta`. So the reason for that is that we did not define `delta` as readonly, even though it itself is not stored but only `sampling_rate`, basically setting `sampling_rate` with the inverse of the value provided for setting `delta`.

Now, while this should even still be OK in most cases, @cjhopp you run into a very strange Python floating point accuracy edge case:

```python
In [2]: 1 / 1e-05                                                                                                                         
Out[2]: 99999.99999999999

In [3]: 1 / 1e-06                                                                                                                         
Out[3]: 1000000.0

In [4]: 1 / 1e-07                                                                                                                         
Out[4]: 10000000.0

In [5]: 1 / 1e-08                                                                                                                         
Out[5]: 100000000.0

In [6]: 1 / 1e-04                                                                                                                         
Out[6]: 10000.0

In [7]: 1 / 1e-05                                                                                                                         
Out[7]: 99999.99999999999

In [8]: 1/1e-5                                                                                                                            
Out[8]: 99999.99999999999

In [9]: 1e5 == 1 / 1e-5                                                                                                                   
Out[9]: False
```

---

### Long story short

In principle it's not our problem, but we could get around it by being less lazy in `Trace.copy` and only use `deepcopy` for some parts and explicitly skip setting derived keys in `Stats` (here: `delta`), thus avoiding to set the parent keys (here: `sampling_rate`) with derived values computed from derived ones (here: `1 / float(1e-5)` evaluating to 99999.9).

Resolving patch:
diff --git a/obspy/core/event/event.py b/obspy/core/event/event.py
--- a/obspy/core/event/event.py
+++ b/obspy/core/event/event.py
@@ -291,7 +291,7 @@ def __deepcopy__(self, memodict=None):
         return result
 
     def __setstate__(self, state_dict):
-        super(Event, self).__setstate__(state_dict)
+        self.__dict__.update(state_dict)
         self.scope_resource_ids()
 
     def write(self, filename, format, **kwargs):
diff --git a/obspy/core/trace.py b/obspy/core/trace.py
--- a/obspy/core/trace.py
+++ b/obspy/core/trace.py
@@ -241,6 +241,18 @@ def __str__(self):
     def _repr_pretty_(self, p, cycle):
         p.text(str(self))
 
+    def __getstate__(self):
+        state = self.__dict__.copy()
+        # Remove the unneeded entries
+        state.pop('delta', None)
+        state.pop('endtime', None)
+        return state
+
+    def __setstate__(self, state):
+        self.__dict__.update(state)
+        # trigger refreshing
+        self.__setitem__('sampling_rate', state['sampling_rate'])
+
 
 @decorator
 def _add_processing_info(func, *args, **kwargs):
diff --git a/obspy/core/util/attribdict.py b/obspy/core/util/attribdict.py
--- a/obspy/core/util/attribdict.py
+++ b/obspy/core/util/attribdict.py
@@ -106,15 +106,6 @@ def __setitem__(self, key, value):
     def __delitem__(self, name):
         del self.__dict__[name]
 
-    def __getstate__(self):
-        return self.__dict__
-
-    def __setstate__(self, adict):
-        # set default values
-        self.__dict__.update(self.defaults)
-        # update with pickle dictionary
-        self.update(adict)
-
     def __getattr__(self, name, default=None):
         """
         Py3k hasattr() expects an AttributeError no KeyError to be
@@ -131,11 +122,6 @@ def __getattr__(self, name, default=None):
     def copy(self):
         return copy.deepcopy(self)
 
-    def __deepcopy__(self, *args, **kwargs):  # @UnusedVariable
-        ad = self.__class__()
-        ad.update(copy.deepcopy(self.__dict__))
-        return ad
-
     def update(self, adict={}):
         for (key, value) in adict.items():
             if key in self.readonly:


Test patch:
diff --git a/obspy/core/tests/test_trace.py b/obspy/core/tests/test_trace.py
--- a/obspy/core/tests/test_trace.py
+++ b/obspy/core/tests/test_trace.py
@@ -2744,6 +2744,27 @@ def test_pickle_soh(self):
         tr_pickled = pickle.loads(pickle.dumps(tr_orig, protocol=2))
         self.assertEqual(tr_orig, tr_pickled)
 
+    def test_deepcopy_issue2600(self):
+        """
+        Tests correct deepcopy of Trace and Stats objects, issue 2600
+        """
+        tr = Trace(data=np.ones(2), header={'sampling_rate': 1e5})
+        sr1 = tr.stats.sampling_rate
+        sr2 = tr.copy().stats.sampling_rate
+        self.assertEqual(sr1, 1e5)
+        self.assertEqual(sr2, sr1)
+
+    def test_pickle_issue2600(self):
+        """
+        Tests correct pickle of Trace and Stats objects, issue 2600
+        """
+        tr = Trace(data=np.ones(2), header={'sampling_rate': 1e5})
+        sr1 = tr.stats.sampling_rate
+        tr_pickled = pickle.loads(pickle.dumps(tr, protocol=2))
+        sr2 = tr_pickled.stats.sampling_rate
+        self.assertEqual(sr1, 1e5)
+        self.assertEqual(sr2, sr1)
+
 
 def suite():
     suite = unittest.TestSuite()
diff --git a/obspy/core/tests/test_util_attribdict.py b/obspy/core/tests/test_util_attribdict.py
--- a/obspy/core/tests/test_util_attribdict.py
+++ b/obspy/core/tests/test_util_attribdict.py
@@ -1,10 +1,15 @@
 # -*- coding: utf-8 -*-
+import pickle
 import unittest
 import warnings
 
 from obspy.core import AttribDict
 
 
+class DefaultTestAttribDict(AttribDict):
+    defaults = {'test': 1}
+
+
 class AttribDictTestCase(unittest.TestCase):
     """
     Test suite for obspy.core.util.attribdict
@@ -247,18 +252,20 @@ class MyAttribDict(AttribDict):
         self.assertEqual(ad.test, 1)
         self.assertRaises(AttributeError, ad.__setitem__, 'test', 1)
 
-    def test_deepcopy(self):
+    def test_deepcopy_and_pickle(self):
         """
-        Tests __deepcopy__ method of AttribDict.
+        Tests deepcopy and pickle of AttribDict.
         """
-        class MyAttribDict(AttribDict):
-            defaults = {'test': 1}
-
-        ad = MyAttribDict()
+        ad = DefaultTestAttribDict()
         ad.muh = 2
-        ad2 = ad.__deepcopy__()
+        ad2 = ad.copy()
         self.assertEqual(ad2.test, 1)
         self.assertEqual(ad2.muh, 2)
+        self.assertEqual(ad2, ad)
+        ad3 = pickle.loads(pickle.dumps(ad, protocol=2))
+        self.assertEqual(ad3.test, 1)
+        self.assertEqual(ad3.muh, 2)
+        self.assertEqual(ad3, ad)
 
     def test_compare_with_dict(self):
         """