Repository: obspy/obspy

Issue:
Keep all SAC headers when reading
Currently, certain SAC header variables are not kept in the trace.stats.sac dictionary when read in from Obspy.  I feel that the expected behavior is to keep ''all'' SAC headers in the trace.stats.sac dictionary.  While it can be reconstructed from values in trace.stats, one has to look into the code to find the header translations.  Obspy is a very convenient SAC file reader, and it would be great to have a more direct connection to the SAC header.

Keep all SAC headers when reading
Currently, certain SAC header variables are not kept in the trace.stats.sac dictionary when read in from Obspy.  I feel that the expected behavior is to keep ''all'' SAC headers in the trace.stats.sac dictionary.  While it can be reconstructed from values in trace.stats, one has to look into the code to find the header translations.  Obspy is a very convenient SAC file reader, and it would be great to have a more direct connection to the SAC header.



Discussion:
[Jonathan MacCarthy jkmacc@lanl.gov] I think the SAC headers in question are:

nzyear
nzday
nzhour
nzmin
nzsec
nzmsec
delta
scale
npts
knetwk
kstnm
kcmpnm

[barsch] I tend to agree with you, however I would strongly suggest to limit those fields to read only access otherwise we have to introduce additional "mapping logic" for SAC trace objects where we'll certainly run into trouble ...

How important is it for the general SAC user to change those header variables directly in trace.stats.sac?

[M] Yes, I also think that we should add some documentation how and which header entries are converted. Searching the source code is not an option. This doc is probably missing in all waveform formats.

However, I do not understand what an additional read-only mapping would to tr.stats.sac would help. Most users would probably then like to access all attributes of e.g. tr.stats.sac.nzsec directly for write access instead of reading tr.stats.sac.nzsec and writing tr.stats.starttime.

One goal of obspy is actually to have a uniform read and write support for many waveforms. Therefore, conversion is necessary and for me this is a feature.

[Jonathan MacCarthy jkmacc@lanl.gov] Write access would be useful, but I don't think obspy should be responsible for complete SAC header consistency.  IMO, if you intend to alter header values, you should know what they mean.  A word of caution in the docs would suffice.  I would be OK with a read-only tr.stats.sac, but I ''do'' use `tr.write(format="SAC")` with headers built from scratch, so as long as I could replace the entire tr.stats.sac dictionary with my own values, that still works for me.

[barsch] AFAIK changing those values in the subobject tr.stats.sac (!AttribDict) can't be easily synced automatically with tr.stats (Stats) without overwriting Stats by checking for subformat-specific sync rules (currently tr.stats.sac does not know anything about its parent object tr.stats). Considering that the Stats class is already rather "heavy-weighted" I would really opt for documenting only and leave it as it is.

I someone finds a suitable solution for this I happily accept a patch ;)

[Jonathan MacCarthy jkmacc@lanl.gov] If I wrote something that simply kept the above-mentioned headers in tr.stats.sac, and nothing else, would it be accepted?  I feel that tr.stats.sac is simply a dictionary representation of the SAC header as it was readt, and if the user alters it, the responsibility is theirs to keep it consistent with tr.stats, if at all.

I've not submitted a patch before; do I just diff the most recent version of sacio.py with a modified version and past it here?

[megies] In my opinion, it would probably not be included, as this leads to the possibility of very unexpected and puzzling behavior when people try to control output headers via tr.stats.sac. For miniseed, for example, it possible to control the output encoding via tr.stats.mseed. One possibility I could think of would be to be able to declare certain fields read-only when generating an !AttribDict object, but that feels pretty ugly.

Generally, pasting or attaching a diff for a file is a good way to show your proposed change though, I'd say.

best,
Tobias

[Jonathan MacCarthy jkmacc@lanl.gov] While I agree with the goal of uniform read/write support, there may be in the meantime many people trying obspy on their SAC files and wondering where the most important header values went.  The SAC header is a language many seismologists understand, and I believe that any SAC reader should, at a minimum, return values that these seismologists recognize.  

If something were to be documented and left alone, my vote would be that tr.write(format="SAC") mentions that some of its header values come from tr.stats.  It already mentions that the reference time will be the first sample time; it doesn't seem to be a stretch to add a bit more.

Below is a patch for SacIO.get_obspy_header that returns the missing tr.stats.sac values.  Thanks for hearing me out on this request, whether the patch is accepted or not.

Best,
Jon

```
Index: sacio.py
===================================================================
--- sacio.py    (revision 4042)
+++ sacio.py    (working copy)
@@ -1268,6 +1268,12 @@
         # them according to the starttime, npts and delta.
         header['sac']['b'] = float(self.GetHvalue('b'))
         header['sac']['e'] = float(self.GetHvalue('e'))
+        # <Ticket #390>
+        for i in ['nzyear', 'nzjday', 'nzhour', 'nzmin', 'nzsec', 'nzmsec', 
+                  'delta', 'scale', 'npts', 'knetwk', 'kstnm', 'kcmpnm']:
+            header['sac'][i] = self.GetHvalue(i)
+        # </Ticket #390>
+
         return header
```

[Jonathan MacCarthy jkmacc@lanl.gov] I think the SAC headers in question are:

nzyear
nzday
nzhour
nzmin
nzsec
nzmsec
delta
scale
npts
knetwk
kstnm
kcmpnm

[barsch] I tend to agree with you, however I would strongly suggest to limit those fields to read only access otherwise we have to introduce additional "mapping logic" for SAC trace objects where we'll certainly run into trouble ...

How important is it for the general SAC user to change those header variables directly in trace.stats.sac?

[M] Yes, I also think that we should add some documentation how and which header entries are converted. Searching the source code is not an option. This doc is probably missing in all waveform formats.

However, I do not understand what an additional read-only mapping would to tr.stats.sac would help. Most users would probably then like to access all attributes of e.g. tr.stats.sac.nzsec directly for write access instead of reading tr.stats.sac.nzsec and writing tr.stats.starttime.

One goal of obspy is actually to have a uniform read and write support for many waveforms. Therefore, conversion is necessary and for me this is a feature.

[Jonathan MacCarthy jkmacc@lanl.gov] Write access would be useful, but I don't think obspy should be responsible for complete SAC header consistency.  IMO, if you intend to alter header values, you should know what they mean.  A word of caution in the docs would suffice.  I would be OK with a read-only tr.stats.sac, but I ''do'' use `tr.write(format="SAC")` with headers built from scratch, so as long as I could replace the entire tr.stats.sac dictionary with my own values, that still works for me.

[barsch] AFAIK changing those values in the subobject tr.stats.sac (!AttribDict) can't be easily synced automatically with tr.stats (Stats) without overwriting Stats by checking for subformat-specific sync rules (currently tr.stats.sac does not know anything about its parent object tr.stats). Considering that the Stats class is already rather "heavy-weighted" I would really opt for documenting only and leave it as it is.

I someone finds a suitable solution for this I happily accept a patch ;)

[Jonathan MacCarthy jkmacc@lanl.gov] If I wrote something that simply kept the above-mentioned headers in tr.stats.sac, and nothing else, would it be accepted?  I feel that tr.stats.sac is simply a dictionary representation of the SAC header as it was readt, and if the user alters it, the responsibility is theirs to keep it consistent with tr.stats, if at all.

I've not submitted a patch before; do I just diff the most recent version of sacio.py with a modified version and past it here?

[megies] In my opinion, it would probably not be included, as this leads to the possibility of very unexpected and puzzling behavior when people try to control output headers via tr.stats.sac. For miniseed, for example, it possible to control the output encoding via tr.stats.mseed. One possibility I could think of would be to be able to declare certain fields read-only when generating an !AttribDict object, but that feels pretty ugly.

Generally, pasting or attaching a diff for a file is a good way to show your proposed change though, I'd say.

best,
Tobias

[Jonathan MacCarthy jkmacc@lanl.gov] While I agree with the goal of uniform read/write support, there may be in the meantime many people trying obspy on their SAC files and wondering where the most important header values went.  The SAC header is a language many seismologists understand, and I believe that any SAC reader should, at a minimum, return values that these seismologists recognize.  

If something were to be documented and left alone, my vote would be that tr.write(format="SAC") mentions that some of its header values come from tr.stats.  It already mentions that the reference time will be the first sample time; it doesn't seem to be a stretch to add a bit more.

Below is a patch for SacIO.get_obspy_header that returns the missing tr.stats.sac values.  Thanks for hearing me out on this request, whether the patch is accepted or not.

Best,
Jon

```
Index: sacio.py
===================================================================
--- sacio.py    (revision 4042)
+++ sacio.py    (working copy)
@@ -1268,6 +1268,12 @@
         # them according to the starttime, npts and delta.
         header['sac']['b'] = float(self.GetHvalue('b'))
         header['sac']['e'] = float(self.GetHvalue('e'))
+        # <Ticket #390>
+        for i in ['nzyear', 'nzjday', 'nzhour', 'nzmin', 'nzsec', 'nzmsec', 
+                  'delta', 'scale', 'npts', 'knetwk', 'kstnm', 'kcmpnm']:
+            header['sac'][i] = self.GetHvalue(i)
+        # </Ticket #390>
+
         return header
```


Resolving patch:
diff --git a/obspy.sac/obspy/sac/core.py b/obspy.sac/obspy/sac/core.py
--- a/obspy.sac/obspy/sac/core.py
+++ b/obspy.sac/obspy/sac/core.py
@@ -86,7 +86,8 @@ def isSACXY(filename):
     return True
 
 
-def readSACXY(filename, headonly=False, **kwargs):  # @UnusedVariable
+def readSACXY(filename, headonly=False, debug_headers=False,
+              **kwargs):  # @UnusedVariable
     """
     Reads an alphanumeric SAC file and returns an ObsPy Stream object.
 
@@ -99,6 +100,13 @@ def readSACXY(filename, headonly=False, **kwargs):  # @UnusedVariable
     :type headonly: bool, optional
     :param headonly: If set to True, read only the head. This is most useful
         for scanning available data in huge (temporary) data sets.
+    :type debug_headers: bool, optional
+    :param debug_headers: Extracts also the SAC headers ``'nzyear', 'nzjday',
+        'nzhour', 'nzmin', 'nzsec', 'nzmsec', 'delta', 'scale', 'npts',
+        'knetwk', 'kstnm', 'kcmpnm'`` which are usually directly mapped to the
+        :class:`~obspy.core.stream.Stream` object if set to ``True``. Those
+        values are not synchronized with the Stream object itself and won't
+        be used during writing of a SAC file! Defaults to ``False``.
     :rtype: :class:`~obspy.core.stream.Stream`
     :return: A ObsPy Stream object.
 
@@ -107,7 +115,7 @@ def readSACXY(filename, headonly=False, **kwargs):  # @UnusedVariable
     >>> from obspy.core import read # doctest: +SKIP
     >>> st = read("/path/to/testxy.sac") # doctest: +SKIP
     """
-    t = SacIO()
+    t = SacIO(debug_headers=debug_headers)
     if headonly:
         t.ReadSacXYHeader(filename)
     else:
@@ -152,7 +160,8 @@ def writeSACXY(stream, filename, **kwargs):  # @UnusedVariable
     return
 
 
-def readSAC(filename, headonly=False, **kwargs):  # @UnusedVariable
+def readSAC(filename, headonly=False, debug_headers=False,
+            **kwargs):  # @UnusedVariable
     """
     Reads an SAC file and returns an ObsPy Stream object.
 
@@ -165,6 +174,13 @@ def readSAC(filename, headonly=False, **kwargs):  # @UnusedVariable
     :type headonly: bool, optional
     :param headonly: If set to True, read only the head. This is most useful
         for scanning available data in huge (temporary) data sets.
+    :type debug_headers: bool, optional
+    :param debug_headers: Extracts also the SAC headers ``'nzyear', 'nzjday',
+        'nzhour', 'nzmin', 'nzsec', 'nzmsec', 'delta', 'scale', 'npts',
+        'knetwk', 'kstnm', 'kcmpnm'`` which are usually directly mapped to the
+        :class:`~obspy.core.stream.Stream` object if set to ``True``. Those
+        values are not synchronized with the Stream object itself and won't
+        be used during writing of a SAC file! Defaults to ``False``.
     :rtype: :class:`~obspy.core.stream.Stream`
     :return: A ObsPy Stream object.
 
@@ -174,7 +190,7 @@ def readSAC(filename, headonly=False, **kwargs):  # @UnusedVariable
     >>> st = read("/path/to/test.sac") # doctest: +SKIP
     """
     # read SAC file
-    t = SacIO()
+    t = SacIO(debug_headers=debug_headers)
     if headonly:
         t.ReadSacHeader(filename)
     else:
diff --git a/obspy.sac/obspy/sac/sacio.py b/obspy.sac/obspy/sac/sacio.py
--- a/obspy.sac/obspy/sac/sacio.py
+++ b/obspy.sac/obspy/sac/sacio.py
@@ -290,9 +290,11 @@ class SacIO(object):
     ============ ==== =========================================================
     """
 
-    def __init__(self, filen=False, headonly=False, alpha=False):
+    def __init__(self, filen=False, headonly=False, alpha=False,
+                 debug_headers=False):
         self.byteorder = 'little'
         self.InitArrays()
+        self.debug_headers = debug_headers
         if filen is False:
             return
         # parse Trace object if we get one
@@ -898,14 +900,14 @@ def WriteSacBinary(self, ofname):
             msg = "Cannot write SAC-buffer to file: "
             raise SacIOError(msg, ofname, e)
 
-    def PrintIValue(self, label='=', value= -12345):
+    def PrintIValue(self, label='=', value=-12345):
         """
         Convenience function for printing undefined integer header values.
         """
         if value != -12345:
             print(label, value)
 
-    def PrintFValue(self, label='=', value= -12345.0):
+    def PrintFValue(self, label='=', value=-12345.0):
         """
         Convenience function for printing undefined float header values.
         """
@@ -1268,6 +1270,11 @@ def get_obspy_header(self):
         # them according to the starttime, npts and delta.
         header['sac']['b'] = float(self.GetHvalue('b'))
         header['sac']['e'] = float(self.GetHvalue('e'))
+        # ticket #390
+        if self.debug_headers:
+            for i in ['nzyear', 'nzjday', 'nzhour', 'nzmin', 'nzsec', 'nzmsec',
+                      'delta', 'scale', 'npts', 'knetwk', 'kstnm', 'kcmpnm']:
+                header['sac'][i] = self.GetHvalue(i)
         return header
 
 


Test patch:
diff --git a/obspy.sac/obspy/sac/tests/test_core.py b/obspy.sac/obspy/sac/tests/test_core.py
--- a/obspy.sac/obspy/sac/tests/test_core.py
+++ b/obspy.sac/obspy/sac/tests/test_core.py
@@ -379,6 +379,39 @@ def test_writeSmallTrace(self):
                 self.assertEquals(len(st[0]), num)
                 os.remove(tempfile)
 
+    def test_issue390(self):
+        """
+        Read all SAC headers if debug_headers flag is enabled.
+        """
+        # 1 - binary SAC
+        tr = read(self.file, headonly=True, debug_headers=True)[0]
+        self.assertEqual(tr.stats.sac.nzyear, 1978)
+        self.assertEqual(tr.stats.sac.nzjday, 199)
+        self.assertEqual(tr.stats.sac.nzhour, 8)
+        self.assertEqual(tr.stats.sac.nzmin, 0)
+        self.assertEqual(tr.stats.sac.nzsec, 0)
+        self.assertEqual(tr.stats.sac.nzmsec, 0)
+        self.assertEqual(tr.stats.sac.delta, 1.0)
+        self.assertEqual(tr.stats.sac.scale, -12345.0)
+        self.assertEqual(tr.stats.sac.npts, 100)
+        self.assertEqual(tr.stats.sac.knetwk, '-12345  ')
+        self.assertEqual(tr.stats.sac.kstnm, 'STA     ')
+        self.assertEqual(tr.stats.sac.kcmpnm, 'Q       ')
+        # 2 - ASCII SAC
+        tr = read(self.filexy, headonly=True, debug_headers=True)[0]
+        self.assertEqual(tr.stats.sac.nzyear, -12345)
+        self.assertEqual(tr.stats.sac.nzjday, -12345)
+        self.assertEqual(tr.stats.sac.nzhour, -12345)
+        self.assertEqual(tr.stats.sac.nzmin, -12345)
+        self.assertEqual(tr.stats.sac.nzsec, -12345)
+        self.assertEqual(tr.stats.sac.nzmsec, -12345)
+        self.assertEqual(tr.stats.sac.delta, 1.0)
+        self.assertEqual(tr.stats.sac.scale, -12345.0)
+        self.assertEqual(tr.stats.sac.npts, 100)
+        self.assertEqual(tr.stats.sac.knetwk, '-12345  ')
+        self.assertEqual(tr.stats.sac.kstnm, 'sta     ')
+        self.assertEqual(tr.stats.sac.kcmpnm, 'Q       ')
+
 
 def suite():
     return unittest.makeSuite(CoreTestCase, 'test')