Repository: rwth-i6/returnn

Issue:
Providing single speaker labels via HDFDataset
This issue is somewhat related to #331, but I still want to know if what I have is a wrong approach or a bug.

I have the following dataset:

```python
{ 'class': 'MetaDataset',
  'data_map': {'classes': ('audio', 'classes'), 'data': ('audio', 'data'), 'speaker_label': ('speaker', 'data')},
  'datasets': { 'audio': {[...]},
                'speaker': { 'class': 'HDFDataset',
                             'files': [ '/u/rossenbach/experiments/librispeech_tts/work/tts/speaker_embeddings/SpeakerLabelHDFFromBliss.xmjLCiVxKiEW/output/speaker_labels.hdf']}},
```

This is my data definition:
```python3
extern_data = {[...],
  'speaker_label': {'dim': 251, 'shape': (None,), 'sparse': True}}
```
And the stored data format is `[B,T|1]`

Unfortunately there is still not a way to store the data without a time axis, so I am doing the same as I did for my speaker vectors before, but now for the single label:
```python3
'speaker_label_notime': {'axis': 'T', 'class': 'squeeze', 'from': ['data:speaker_label']},
```

This results in the following assertion fail (the rest of the trace is not relevant):
```python3
File "/work/asr4/rossenbach/sisyphus_work_folders/librispeech_tts_work/returnn/helpers/rossenbach/checkout/CheckoutRETURNN.WE8k4r4LocNz/output/crnn/b1674b45c0449dc0fef7350bd22abaa5a03036e4/returnn/tf/layers/basic.py", line 3642, in get_out_data_from_opts
    line: return ReduceLayer.get_out_data_from_opts(
    locals:
      ReduceLayer = <global> <class 'returnn.tf.layers.basic.ReduceLayer'>
      ReduceLayer.get_out_data_from_opts = <global> <bound method ReduceLayer.get_out_data_from_opts of <class 'returnn.tf.layers.basic.ReduceLayer'>>
  File "/work/asr4/rossenbach/sisyphus_work_folders/librispeech_tts_work/returnn/helpers/rossenbach/checkout/CheckoutRETURNN.WE8k4r4LocNz/output/crnn/b1674b45c0449dc0fef7350bd22abaa5a03036e4/returnn/tf/layers/basic.py", line 3485, in get_out_data_from_opts
    line: assert not x.sparse
    locals:
      x = <local> Data(name='speaker_label', shape=(None,), dtype='int32', sparse=True, dim=251, available_for_inference=False, batch_shape_meta=[B,T|'time:var:extern_data:speaker_label'])
      x.sparse = <local> True
AssertionError
```
Which is caused by the `SqueezeLayer` using the `get_out_data_from_opts` function from the `ReduceLayer which apparently does not support sparse data.








Discussion:
The title of this bug doesn't sound like it is related to your actual bug/exception?

Fixing `SqueezeLayer` should not be a problem, right? It is clear that `SqueezeLayer` should work also for sparse data (or whatever data).

Don't mix up things. This bug report (any bug report) should not be about multiple things, like MetaDataset, HDFDataset, SqueezeLayer, ReduceLayer. I'm not actually sure which of these you think is wrong here. But the exception you posted is about `SqueezeLayer`, and yes this is a bug (obviously, right?), and this should be simple to fix (right?).

This was not intended to be an issue about a bug, but rather the situation in general. You said it is a bug, which I did consider, but could not be sure of (this is why I asked for it). And this is not simple to fix, as it is not clear to me from the code why sparse is not allowed, and I have to write new test cases for the `ReduceLayer` as well.
This is about `SqueezeLayer`, not about `ReduceLayer`. As I said, there is no reason why `SqueezeLayer` should not work on sparse data -- it is clear that it should work no matter if the data is sparse or not. This is clear, right? So this is an obvious bug.

But everything else you write is totally unrelated to all of that.

Resolving patch:
diff --git a/returnn/tf/layers/basic.py b/returnn/tf/layers/basic.py
--- a/returnn/tf/layers/basic.py
+++ b/returnn/tf/layers/basic.py
@@ -3637,13 +3637,14 @@ def get_out_data_from_opts(cls, axis, enforce_batch_dim_axis=None, allow_no_op=F
     input_data = get_concat_sources_data_template(sources)
     if enforce_batch_dim_axis is not None:
       input_data = input_data.copy_with_batch_dim_axis(enforce_batch_dim_axis)
-    if axis == "auto":
-      axis = cls._get_axes(axis, input_data=input_data)
+    axis_indices = cls._get_axes(axis, input_data=input_data)
     if allow_no_op:
-      if not cls._get_axes(axis, input_data=input_data):
+      if not axis_indices:
         return input_data.copy("%s_output" % kwargs["name"])
-    return ReduceLayer.get_out_data_from_opts(
-      axis=axis, keep_dims=False, enforce_batch_dim_axis=enforce_batch_dim_axis, sources=sources, **kwargs)
+    # remove the axis in reversed order
+    for axis_idx in sorted(axis_indices, reverse=True):
+      input_data = input_data.copy_template_excluding_axis(axis_idx)
+    return input_data
 
 
 class StackLayer(LayerBase):


Test patch: