Repository: librosa/librosa

Issue:
Support user-supplied time format strings in display
**Describe the bug**
I am using `librosa.specshow` with `x_axis="time"` to display spectrograms. I think i've found a bug in the xticklabels.
If the signal is less than one minute long (e.g., Brahms), it all goes well.
If it is more than one minute long and i'm showing all of it, it also goes well.
But if the signal is 90 seconds long and i want to show the last 40 seconds of it, librosa does not print xticklabels in `mm:ss` format but in scientific notation. (see below)


**To Reproduce**

Example:
```python
import librosa
from matplotlib import pyplot as plt

def specshow_with_offset(y, sr, offset):
    S = librosa.power_to_db(np.abs(librosa.stft(y)))
    times = librosa.times_like(S, sr=sr)
    librosa.display.specshow(
        S, sr=sr, x_coords=times+offset, x_axis="time",
        y_axis="hz")
    
plt.figure(figsize=(6, 6))
y, sr = librosa.load(librosa.ex("brahms"))
for fig_id, offset in enumerate([0, 60, 120]):
    plt.subplot(3, 1, 1+fig_id)
    specshow_with_offset(y, sr, offset)
```

**Expected behavior**
Use `mm:ss` or `hh:mm:ss` notations depending on the values taken by `x_coords` rather than depending on the duration of the input.

**Screenshots**
<img width="422" alt="Screenshot 2021-08-17 at 17 23 56" src="https://user-images.githubusercontent.com/3943142/129754707-eaad34cd-71c8-4b65-9d7d-bc7d74d442ff.png">


**Software versions***
```
macOS-10.15.7-x86_64-i386-64bit
Python 3.8.10 (default, May 19 2021, 11:01:55) 
[Clang 10.0.0 ]
NumPy 1.21.0
SciPy 1.4.1
librosa 0.8.1
INSTALLED VERSIONS
------------------
python: 3.8.10 (default, May 19 2021, 11:01:55) 
[Clang 10.0.0 ]

librosa: 0.8.1

audioread: 2.1.9
numpy: 1.21.0
scipy: 1.4.1
sklearn: 0.22.1
joblib: 1.0.1
decorator: 5.0.9
soundfile: 0.9.0
resampy: 0.2.2
numba: 0.48.0

```

**Additional context**
Related: #737 #760



Discussion:
Getting this kind of thing right can be annoyingly subtle.  I don't actually believe there is a single, correct "expected behavior" here.

> Use `mm:ss` or `hh:mm:ss` notations depending on the values taken by `x_coords` rather than depending on the duration of the input.

That sometimes makes sense (eg in your use-case) and sometimes doesn't.  If you wanted to show, say, a 1-second fragment of audio at a large offset, then using `hh:mm:ss` would be far too coarse to be useful.  It's kind of the same problem as the `%G` format code for numbers: how much precision do you want/need?  It really depends on context, not just the precision of the number being displayed.

I think the best option here is probably to expose more fine-grained format controls on time decorations.  Right now, we expose `time` (auto-format), `s` (always seconds), `ms` (always milliseconds), and a few variations thereof.  We could just as well expand in the other direction, `m` (always `mm:ss`), `h` (always `hh:mm:ss`) to explicitly select the different modes that `time` can automatically pick.

Would that be a satisfactory resolution?
> Getting this kind of thing right can be annoyingly subtle. I don't actually believe there is a single, correct "expected behavior" here.

Right ... to be clear, i wasn't suggesting otherwise!


> We could just as well expand in the other direction, m (always mm:ss), h (always hh:mm:ss) to explicitly select the different modes that time can automatically pick.

That's a good idea. `m` would be especially important for (pop) music because music players typically show you the time in `m:ss` format, even though `m` is zero in the first minute.

`h` would also be a welcome addition for domains like urban acoustics and bioacoustics because of the need to show the precise local time of day below the spectrogram, rather than the time within the recording.


For context, this came up while writing course material about music segmentation/structuration. I wanted to show four different verses from a pop song, of which i knew the onsets and offsets. This was the result with `x_axis="time"` (watch the last one):

<img width="516" alt="Screenshot 2021-08-17 at 18 07 04" src="https://user-images.githubusercontent.com/3943142/129761481-378b0f08-429d-43d2-9488-047d6008a3f1.png">


and now `x_axis="s"`:

<img width="522" alt="Screenshot 2021-08-17 at 17 47 28" src="https://user-images.githubusercontent.com/3943142/129760230-4fc7e3d4-25e7-49ab-bcbd-d6627fd6e0fe.png">

except that i would prefer to see `2:20` instead of `140`

@bmcfee if that's OK with you i can work on implementing `m` and `h` modes for `x_axis`, as you proposed above 
> @bmcfee if that's OK with you i can work on implementing `m` and `h` modes for `x_axis`, as you proposed above

That would be great!  It should be a pretty easy modification.  We'll probably want to expand this to cover `lag_` options as well.
Looking at this issue again today. While chatting with @bmcfee i offered the following solution:

> what if our TimeFormatter’s spoke datetime.strftime ?
> https://docs.python.org/3.6/library/datetime.html#strftime-and-strptime-behavior
> 
> i imagine it would just be an `if "%" in unit:`  branch


As a more long-term goal, there is also the prospect that our `x_coords` and `y_coords` could accept numpy arrays of  `np.datetime64` and `np.timedelta64` objects.
Him and i had the idea (at the same time) of providing a utility function `librosa.datetime_like(X, sr, hop_length, offset)`

Proof of concept


```python
>>> offset = np.datetime64('2021-08-22T04:58')
>>> np.array([offset + np.timedelta64(n, "h") for n in range(7)])
array(['2021-08-22T04:58', '2021-08-22T05:58', '2021-08-22T06:58',
       '2021-08-22T07:58', '2021-08-22T08:58', '2021-08-22T09:58',
       '2021-08-22T10:58'], dtype='datetime64[m]')
```

I am copying over these thoughts to this issue because right now they're living on a Slack chat and i don't want to lose them because of the 10k message limit ...

I can volunteer to work on the short-term plan (allow strptime strings in our TimeFormatter) right now, which should suffice to close this issue.


Resolving patch:
diff --git a/librosa/display.py b/librosa/display.py
--- a/librosa/display.py
+++ b/librosa/display.py
@@ -78,12 +78,15 @@ class TimeFormatter(Formatter):
         Anything past the midpoint will be converted to negative time.
 
     unit : str or None
-        Abbreviation of the physical unit for axis labels and ticks.
-        Either equal to `s` (seconds) or `ms` (milliseconds) or None (default).
-        If set to None, the resulting TimeFormatter object adapts its string
-        representation to the duration of the underlying time range:
-        `hh:mm:ss` above 3600 seconds; `mm:ss` between 60 and 3600 seconds;
-        and `ss` below 60 seconds.
+        Abbreviation of the string representation for axis labels and ticks.
+        List of supported units:
+        * `"h"`: hour-based format (`H:MM:SS`)
+        * `"m"`: minute-based format (`M:SS`)
+        * `"s"`: second-based format (`S.sss` in scientific notation)
+        * `"ms"`: millisecond-based format (`s.µµµ` in scientific notation)
+        * `None`: adaptive to the duration of the underlying time range: similar
+        to `"h"` above 3600 seconds; to `"m"` between 60 and 3600 seconds; to
+        `"s"` between 1 and 60 seconds; and to `"ms"` below 1 second.
 
 
     See also
@@ -125,7 +128,7 @@ class TimeFormatter(Formatter):
 
     def __init__(self, lag=False, unit=None):
 
-        if unit not in ["s", "ms", None]:
+        if unit not in ["h", "m", "s", "ms", None]:
             raise ParameterError("Unknown time unit: {}".format(unit))
 
         self.unit = unit
@@ -149,27 +152,22 @@ def __call__(self, x, pos=None):
             value = x
             sign = ""
 
-        if self.unit == "s":
+        if self.unit == "h" or ((self.unit is None) and (vmax - vmin > 3600)):
+            s = "{:d}:{:02d}:{:02d}".format(
+                int(value / 3600.0),
+                int(np.mod(value / 60.0, 60)),
+                int(np.mod(value, 60)),
+            )
+        elif self.unit == "m" or ((self.unit is None) and (vmax - vmin > 60)):
+            s = "{:d}:{:02d}".format(int(value / 60.0), int(np.mod(value, 60)))
+        elif self.unit == "s":
             s = "{:.3g}".format(value)
+        elif self.unit==None and (vmax - vmin >= 1):
+            s = "{:.2g}".format(value)
         elif self.unit == "ms":
             s = "{:.3g}".format(value * 1000)
-        else:
-            if vmax - vmin > 3600:
-                # Hours viz
-                s = "{:d}:{:02d}:{:02d}".format(
-                    int(value / 3600.0),
-                    int(np.mod(value / 60.0, 60)),
-                    int(np.mod(value, 60)),
-                )
-            elif vmax - vmin > 60:
-                # Minutes viz
-                s = "{:d}:{:02d}".format(int(value / 60.0), int(np.mod(value, 60)))
-            elif vmax - vmin >= 1:
-                # Seconds viz
-                s = "{:.2g}".format(value)
-            else:
-                # Milliseconds viz
-                s = "{:.3f}".format(value)
+        elif self.unit==None and (vmax - vmin < 1):
+            s = "{:.3f}".format(value)
 
         return "{:s}{:s}".format(sign, s)
 
@@ -619,8 +617,8 @@ def __envelope(x, hop):
 _chroma_ax_types = ('chroma', 'chroma_h', 'chroma_c',)
 _cqt_ax_types = ('cqt_hz', 'cqt_note', 'cqt_svara',)
 _freq_ax_types = ('linear', 'fft', 'hz', 'fft_note', 'fft_svara',)
-_time_ax_types = ('time', 's', 'ms',)
-_lag_ax_types = ('lag', 'lag_s', 'lag_ms',)
+_time_ax_types = ('time', 'h', 'm', 's', 'ms',)
+_lag_ax_types = ('lag', 'lag_h', 'lag_m', 'lag_s', 'lag_ms',)
 _misc_ax_types = ('tempo', 'fourier_tempo', 'mel', 'log', 'tonnetz', 'frames',)
 
 _AXIS_COMPAT = set(
@@ -728,9 +726,13 @@ def specshow(
 
         - 'time' : markers are shown as milliseconds, seconds, minutes, or hours.
                 Values are plotted in units of seconds.
+        - 'h' : markers are shown as hours, minutes, and seconds.
+        - 'm' : markers are shown as minutes and seconds.
         - 's' : markers are shown as seconds.
         - 'ms' : markers are shown as milliseconds.
         - 'lag' : like time, but past the halfway point counts as negative values.
+        - 'lag_h' : same as lag, but in hours, minutes and seconds.
+        - 'lag_m' : same as lag, but in minutes and seconds.
         - 'lag_s' : same as lag, but in seconds.
         - 'lag_ms' : same as lag, but in milliseconds.
 
@@ -953,9 +955,13 @@ def __mesh_coords(ax_type, coords, n, **kwargs):
         "chroma_c": __coord_chroma,
         "chroma_h": __coord_chroma,
         "time": __coord_time,
+        "h": __coord_time,
+        "m": __coord_time,
         "s": __coord_time,
         "ms": __coord_time,
         "lag": __coord_time,
+        "lag_h": __coord_time,
+        "lag_m": __coord_time,
         "lag_s": __coord_time,
         "lag_ms": __coord_time,
         "tonnetz": __coord_n,
@@ -1044,6 +1050,8 @@ def __decorate_axis(
     axis, ax_type, key="C:maj", Sa=None, mela=None, thaat=None, unicode=True
 ):
     """Configure axis tickers, locators, and labels"""
+    time_units = {
+        "h": "hours", "m": "minutes", "s": "seconds", "ms": "milliseconds"}
 
     if ax_type == "tonnetz":
         axis.set_major_formatter(TonnetzFormatter())
@@ -1098,30 +1106,21 @@ def __decorate_axis(
         axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
         axis.set_label_text("Time")
 
-    elif ax_type == "s":
-        axis.set_major_formatter(TimeFormatter(unit="s", lag=False))
+    elif ax_type in time_units:
+        axis.set_major_formatter(TimeFormatter(unit=ax_type, lag=False))
         axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
-        axis.set_label_text("Time (s)")
-
-    elif ax_type == "ms":
-        axis.set_major_formatter(TimeFormatter(unit="ms", lag=False))
-        axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
-        axis.set_label_text("Time (ms)")
+        axis.set_label_text("Time ({:s})".format(time_units[ax_type]))
 
     elif ax_type == "lag":
         axis.set_major_formatter(TimeFormatter(unit=None, lag=True))
         axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
         axis.set_label_text("Lag")
 
-    elif ax_type == "lag_s":
-        axis.set_major_formatter(TimeFormatter(unit="s", lag=True))
+    elif isinstance(ax_type, str) and ax_type.startswith("lag_"):
+        unit = ax_type[4:]
+        axis.set_major_formatter(TimeFormatter(unit=unit, lag=True))
         axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
-        axis.set_label_text("Lag (s)")
-
-    elif ax_type == "lag_ms":
-        axis.set_major_formatter(TimeFormatter(unit="ms", lag=True))
-        axis.set_major_locator(MaxNLocator(prune=None, steps=[1, 1.5, 5, 6, 10]))
-        axis.set_label_text("Lag (ms)")
+        axis.set_label_text("Lag ({:s})".format(time_units[unit]))
 
     elif ax_type == "cqt_note":
         axis.set_major_formatter(NoteFormatter(key=key, unicode=unicode))
@@ -1358,12 +1357,20 @@ def waveshow(
         - 'time' : markers are shown as milliseconds, seconds, minutes, or hours.
                     Values are plotted in units of seconds.
 
+        - 'h' : markers are shown as hours, minutes, and seconds.
+
+        - 'm' : markers are shown as minutes and seconds.
+
         - 's' : markers are shown as seconds.
 
         - 'ms' : markers are shown as milliseconds.
 
         - 'lag' : like time, but past the halfway point counts as negative values.
 
+        - 'lag_h' : same as lag, but in hours.
+
+        - 'lag_m' : same as lag, but in minutes.
+
         - 'lag_s' : same as lag, but in seconds.
 
         - 'lag_ms' : same as lag, but in milliseconds.


Test patch:
diff --git a/tests/baseline_images/test_display/test_time_unit.png b/tests/baseline_images/test_display/test_time_unit.png
Binary files a/tests/baseline_images/test_display/test_time_unit.png and b/tests/baseline_images/test_display/test_time_unit.png differ
diff --git a/tests/baseline_images/test_display/test_time_unit_lag.png b/tests/baseline_images/test_display/test_time_unit_lag.png
Binary files a/tests/baseline_images/test_display/test_time_unit_lag.png and b/tests/baseline_images/test_display/test_time_unit_lag.png differ
diff --git a/tests/test_display.py b/tests/test_display.py
--- a/tests/test_display.py
+++ b/tests/test_display.py
@@ -106,7 +106,7 @@ def test_unknown_time_unit(y):
     ax = plt.gca()
     ax.plot(times, y)
     ax.xaxis.set_major_formatter(
-        librosa.display.TimeFormatter(unit="neither s, nor ms, nor None")
+        librosa.display.TimeFormatter(unit="an unsupported time unit")
     )
     return plt.gcf()
 
@@ -485,15 +485,23 @@ def test_time_unit(S_abs, sr):
     # sr = 22050, hop_length = 512, S.shape[1] = 198
     # 197 * 512 / 22050 ~= 4.6s
     plt.figure(figsize=(9, 10))
-    plt.subplot(3, 1, 1)
+    plt.subplot(5, 1, 1)
     # time scale auto
     librosa.display.specshow(S_abs, sr=sr, x_axis="time")
 
-    plt.subplot(3, 1, 2)
+    plt.subplot(5, 1, 2)
+    # time unit fixed to 'h'
+    librosa.display.specshow(S_abs, sr=sr, x_axis="h")
+
+    plt.subplot(5, 1, 3)
+    # time unit fixed to 'h'
+    librosa.display.specshow(S_abs, sr=sr, x_axis="m")
+
+    plt.subplot(5, 1, 4)
     # time unit fixed to 's'
     librosa.display.specshow(S_abs, sr=sr, x_axis="s")
 
-    plt.subplot(3, 1, 3)
+    plt.subplot(5, 1, 5)
     # time unit fixed to 'ms'
     librosa.display.specshow(S_abs, sr=sr, x_axis="ms")
 
@@ -508,15 +516,23 @@ def test_time_unit(S_abs, sr):
 def test_time_unit_lag(S_abs, sr):
 
     plt.figure(figsize=(9, 10))
-    plt.subplot(3, 1, 1)
+    plt.subplot(5, 1, 1)
     # time scale auto in lag mode
     librosa.display.specshow(S_abs, sr=sr, x_axis="lag")
 
-    plt.subplot(3, 1, 2)
+    plt.subplot(5, 1, 2)
+    # time unit fixed to 'h' in lag mode
+    librosa.display.specshow(S_abs, sr=sr, x_axis="lag_h")
+
+    plt.subplot(5, 1, 3)
+    # time unit fixed to 'm' in lag mode
+    librosa.display.specshow(S_abs, sr=sr, x_axis="lag_m")
+
+    plt.subplot(5, 1, 4)
     # time unit fixed to 's' in lag mode
     librosa.display.specshow(S_abs, sr=sr, x_axis="lag_s")
 
-    plt.subplot(3, 1, 3)
+    plt.subplot(5, 1, 5)
     # time unit fixed to 'ms' in lag mode
     librosa.display.specshow(S_abs, sr=sr, x_axis="lag_ms")