Repository: lhotse-speech/lhotse

Issue:
There was a problem in create a CutSet.
Used command:
```
lhotse cut simple -r ./data/train_manifests/recordings.jsonl.gz -f ./data/train_manifests/features.jsonl.gz -s ./data/train_manifests/supervisions.jsonl.gz ./yesno_train.jsonl.gz
```
and 
```
lhotse cut simple -r train_manifests/recordings.jsonl.gz -f train_manifests/features.jsonl.gz -s train_manifests/supervisions.jsonl.gz ./train_manifests/my_train.jsonl.gz
```

I found that the data in `supervisions.jsonl.gz` was't written to `yesno_train.jsonl.gz`.

The bug appears on line 3184 of `lhotse/cut/set.py`:
```py
supervisions=list(
    sups.find(
          recording_id=feats.recording_id,
          channel=channel,
           start_after=feats.start,
            end_before=feats.end,
            adjust_offset=True,
            tolerance = 0.02,
     )
)
```
For the 'yesno' dataset, the `tolerance` parameter needs to be set to 0.02, and for my dataset it needs to be set above 0.02.



Discussion:
Can you try adding the `--force-eager` option to the command?
> 您可以尝试将选项添加到命令中吗？`--force-eager`

I tried to adding the `--force-eager` parameter.  `supervisions` messages are not written.
```
(py38) kuangfangjun:train_manifests$ ls
features.jsonl.gz  recordings.jsonl.gz  supervisions.jsonl.gz
(py38) kuangfangjun:train_manifests$ gunzip -c supervisions.jsonl.gz | head -n 3
{"id": "0_0_0_0_1_1_1_1", "recording_id": "0_0_0_0_1_1_1_1", "start": 0.0, "duration": 6.35, "channel": 0, "text": "NO NO NO NO YES YES YES YES", "speaker": "global"}
{"id": "0_0_0_1_0_0_0_1", "recording_id": "0_0_0_1_0_0_0_1", "start": 0.0, "duration": 6.76, "channel": 0, "text": "NO NO NO YES NO NO NO YES", "speaker": "global"}
{"id": "0_0_0_1_0_1_1_0", "recording_id": "0_0_0_1_0_1_1_0", "start": 0.0, "duration": 6.11, "channel": 0, "text": "NO NO NO YES NO YES YES NO", "speaker": "global"}

(py38) kuangfangjun:train_manifests$ gunzip -c features.jsonl.gz | head -n 3
{"type": "kaldi-fbank", "num_frames": 633, "num_features": 13, "frame_shift": 0.01, "sampling_rate": 8000, "start": 0, "duration": 6.33, "storage_type": "kaldiio", "storage_path": "/root/fangjun/open-source/kaldi/egs/yesno/s5/mfcc/raw_mfcc_train_yesno.1.ark:16", "storage_key": "0_0_0_0_1_1_1_1", "recording_id": "0_0_0_0_1_1_1_1", "channels": 0}
{"type": "kaldi-fbank", "num_frames": 674, "num_features": 13, "frame_shift": 0.01, "sampling_rate": 8000, "start": 0, "duration": 6.74, "storage_type": "kaldiio", "storage_path": "/root/fangjun/open-source/kaldi/egs/yesno/s5/mfcc/raw_mfcc_train_yesno.1.ark:8386", "storage_key": "0_0_0_1_0_0_0_1", "recording_id": "0_0_0_1_0_0_0_1", "channels": 0}
{"type": "kaldi-fbank", "num_frames": 609, "num_features": 13, "frame_shift": 0.01, "sampling_rate": 8000, "start": 0, "duration": 6.09, "storage_type": "kaldiio", "storage_path": "/root/fangjun/open-source/kaldi/egs/yesno/s5/mfcc/raw_mfcc_train_yesno.1.ark:17289", "storage_key": "0_0_0_1_0_1_1_0",
"recording_id": "0_0_0_1_0_1_1_0", "channels": 0}

(py38) kuangfangjun:train_manifests$ gunzip -c recordings.jsonl.gz | head -n 3
{"id": "0_0_0_0_1_1_1_1", "sources": [{"type": "file", "channels": [0], "source": "waves_yesno/0_0_0_0_1_1_1_1.wav"}], "sampling_rate": 8000, "num_samples": 50800, "duration": 6.35, "channel_ids": [0]}
{"id": "0_0_0_1_0_0_0_1", "sources": [{"type": "file", "channels": [0], "source": "waves_yesno/0_0_0_1_0_0_0_1.wav"}], "sampling_rate": 8000, "num_samples": 54080, "duration": 6.76, "channel_ids": [0]}
{"id": "0_0_0_1_0_1_1_0", "sources": [{"type": "file", "channels": [0], "source": "waves_yesno/0_0_0_1_0_1_1_0.wav"}], "sampling_rate": 8000, "num_samples": 48880, "duration": 6.11, "channel_ids": [0]}
```

I think the main issue is that kaldi uses `snip_edges=true` while `lhotse` assumes `snip_edges=false`.

https://github.com/lhotse-speech/lhotse/blob/eb9e6b115729697c66c0a7f5f7ba08984b6a1ee5/lhotse/cut/set.py#L3185-L3189

Line 3189 requires that 
```
     end_before=feats.end, 
```

But from the logs posted at the very beginning, we can see that the duration in supervision is always larger than that in features, so `sup.find()` always returns an empty supervision. That is why the resulting supervision is empty. 
I understand that this issue would happen pretty much only with Kaldi-imported data dirs. Maybe we can expose the tolerance parameter and use it in Kaldi import utilities, and then try to auto-fix the manifests by truncating to the shortest of {recording,supervision,features} - would that make sense?
> would that make sense?

Yes, that sounds reasonable to me.

Resolving patch:
diff --git a/lhotse/bin/modes/kaldi.py b/lhotse/bin/modes/kaldi.py
--- a/lhotse/bin/modes/kaldi.py
+++ b/lhotse/bin/modes/kaldi.py
@@ -70,6 +70,7 @@ def import_(
     Convert a Kaldi data dir DATA_DIR into a directory MANIFEST_DIR of lhotse manifests. Ignores feats.scp.
     The SAMPLING_RATE has to be explicitly specified as it is not available to read from DATA_DIR.
     """
+    from lhotse import CutSet
     from lhotse.kaldi import load_kaldi_data_dir
 
     recording_set, maybe_supervision_set, maybe_feature_set = load_kaldi_data_dir(
@@ -88,6 +89,12 @@ def import_(
         maybe_supervision_set.to_file(manifest_dir / "supervisions.jsonl.gz")
     if maybe_feature_set is not None:
         maybe_feature_set.to_file(manifest_dir / "features.jsonl.gz")
+    CutSet.from_manifests(
+        recordings=recording_set,
+        supervisions=maybe_supervision_set,
+        features=maybe_feature_set,
+        tolerance=0.1,
+    ).to_file(manifest_dir / "cuts.jsonl.gz")
 
 
 @kaldi.command()
diff --git a/lhotse/cut/set.py b/lhotse/cut/set.py
--- a/lhotse/cut/set.py
+++ b/lhotse/cut/set.py
@@ -313,6 +313,7 @@ def from_manifests(
         features: Optional[FeatureSet] = None,
         output_path: Optional[Pathlike] = None,
         random_ids: bool = False,
+        tolerance: Seconds = 0.001,
         lazy: bool = False,
     ) -> "CutSet":
         """
@@ -332,6 +333,9 @@ def from_manifests(
         :param output_path: an optional path where the :class:`.CutSet` is stored.
         :param random_ids: boolean, should the cut IDs be randomized. By default, use the recording ID
             with a loop index and a channel idx, i.e. "{recording_id}-{idx}-{channel}")
+        :param tolerance: float, tolerance for supervision and feature segment boundary comparison.
+            By default, it's 1ms. Increasing this value can be helpful when importing Kaldi data
+            directories with precomputed features (typically 0.02 - 0.1 should be sufficient).
         :param lazy: boolean, when ``True``, output_path must be provided
         :return: a new :class:`.CutSet` instance.
         """
@@ -342,6 +346,7 @@ def from_manifests(
                 features=features,
                 output_path=output_path,
                 random_ids=random_ids,
+                tolerance=tolerance,
             )
         else:
             return create_cut_set_eager(
@@ -350,6 +355,7 @@ def from_manifests(
                 features=features,
                 output_path=output_path,
                 random_ids=random_ids,
+                tolerance=tolerance,
             )
 
     @staticmethod
@@ -2975,6 +2981,7 @@ def create_cut_set_eager(
     features: Optional[FeatureSet] = None,
     output_path: Optional[Pathlike] = None,
     random_ids: bool = False,
+    tolerance: Seconds = 0.001,
 ) -> CutSet:
     """
     Create a :class:`.CutSet` from any combination of supervision, feature and recording manifests.
@@ -2993,6 +3000,9 @@ def create_cut_set_eager(
     :param output_path: an optional path where the :class:`.CutSet` is stored.
     :param random_ids: boolean, should the cut IDs be randomized. By default, use the recording ID
         with a loop index and a channel idx, i.e. "{recording_id}-{idx}-{channel}")
+    :param tolerance: float, tolerance for supervision and feature segment boundary comparison.
+        By default, it's 1ms. Increasing this value can be helpful when importing Kaldi data
+        directories with precomputed features.
     :return: a new :class:`.CutSet` instance.
     """
     assert (
@@ -3039,6 +3049,7 @@ def create_cut_set_eager(
                             start_after=feats.start,
                             end_before=feats.end,
                             adjust_offset=True,
+                            tolerance=tolerance,
                         )
                     )
                     if sup_ok
@@ -3063,11 +3074,7 @@ def create_cut_set_eager(
                     duration=recording.duration,
                     channel=channel,
                     recording=recording,
-                    supervisions=list(
-                        supervisions.find(
-                            recording_id=recording.id,
-                        )
-                    )
+                    supervisions=list(supervisions.find(recording_id=recording.id))
                     if sup_ok
                     else [],
                 )
@@ -3084,6 +3091,7 @@ def create_cut_set_lazy(
     supervisions: Optional[SupervisionSet] = None,
     features: Optional[FeatureSet] = None,
     random_ids: bool = False,
+    tolerance: Seconds = 0.001,
 ) -> CutSet:
     """
     Create a :class:`.CutSet` from any combination of supervision, feature and recording manifests.
@@ -3115,6 +3123,9 @@ def create_cut_set_lazy(
     :param features: an optional :class:`~lhotse.features.base.FeatureSet` manifest.
     :param random_ids: boolean, should the cut IDs be randomized. By default, use the recording ID
         with a loop index and a channel idx, i.e. "{recording_id}-{idx}-{channel}")
+    :param tolerance: float, tolerance for supervision and feature segment boundary comparison.
+        By default, it's 1ms. Increasing this value can be helpful when importing Kaldi data
+        directories with precomputed features.
     :return: a new :class:`.CutSet` instance.
     """
     assert (
@@ -3188,6 +3199,7 @@ def create_cut_set_lazy(
                             start_after=feats.start,
                             end_before=feats.end,
                             adjust_offset=True,
+                            tolerance=tolerance,
                         )
                     )
                     if sup_ok


Test patch: