Repository: lhotse-speech/lhotse

Issue:
Size of initial minibatch
Piotr, in our snowfall eg with mini_librispeech, the 1st minibatch is 16 seconds long which seems on the long side.
Is that typical of the data, or are they arranged from longest to shortest?
.. because if we want them in a nonrandom order we probably want shortest to longest, which would be better for convergence.


Discussion:
They are randomly shuffled. There are also multiple cuts packed into a single batch example which might make the duration excessive. Do you think some curriculum learning like option would be useful (sorting by length, possibly for a number of first epochs)? 

> Wiadomość napisana przez Daniel Povey <notifications@github.com> w dniu 11/22/20, o godz. 08:33:
> 
> ﻿
> Piotr, in our snowfall eg with mini_librispeech, the 1st minibatch is 16 seconds long which seems on the long side.
> Is that typical of the data, or are they arranged from longest to shortest?
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

I assume the length of the example is the length of the longest individual
cut?



On Sun, Nov 22, 2020 at 10:36 PM Piotr Żelasko <notifications@github.com>
wrote:

> They are randomly shuffled. There are also multiple cuts packed into a
> single batch example which might make the duration excessive. Do you think
> some curriculum learning like option would be useful (sorting by length,
> possibly for a number of first epochs)?
>
> > Wiadomość napisana przez Daniel Povey <notifications@github.com> w dniu
> 11/22/20, o godz. 08:33:
> >
> > ﻿
> > Piotr, in our snowfall eg with mini_librispeech, the 1st minibatch is 16
> seconds long which seems on the long side.
> > Is that typical of the data, or are they arranged from longest to
> shortest?
> >
> > —
> > You are receiving this because you are subscribed to this thread.
> > Reply to this email directly, view it on GitHub, or unsubscribe.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/lhotse-speech/lhotse/issues/146#issuecomment-731757701>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAZFLOYOXZGR56UXZHSKNPDSREOXFANCNFSM4T6PLHFA>
> .
>

By default it's twice of that (it helps the heuristic pack cuts better), but can be adjusted; see: https://github.com/lhotse-speech/lhotse/blob/master/lhotse/dataset/speech_recognition.py#L136


I think we should make it that by default, not twice of that, because for
many model types the time taken could be much more sensitive to the
sequence length than to the total number of frames in the sequence.
For attention models it can even take time quadratic in the num-frames.



On Sun, Nov 22, 2020 at 11:11 PM Piotr Żelasko <notifications@github.com>
wrote:

> By default it's twice of that (it helps the heuristic pack cuts better),
> but can be adjusted; see:
> https://github.com/lhotse-speech/lhotse/blob/master/lhotse/dataset/speech_recognition.py#L136
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/lhotse-speech/lhotse/issues/146#issuecomment-731762618>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAZFLO7HI4M4Q5H2S4VW7CLSRES3DANCNFSM4T6PLHFA>
> .
>

Ok, will do. I wonder though, if it makes sense to concatenate the cuts for the attention models at all (unless these are subsequent utterances from the same recording/conversation). 

> Wiadomość napisana przez Daniel Povey <notifications@github.com> w dniu 11/22/20, o godz. 10:20:
> 
> ﻿
> I think we should make it that by default, not twice of that, because for
> many model types the time taken could be much more sensitive to the
> sequence length than to the total number of frames in the sequence.
> For attention models it can even take time quadratic in the num-frames.
> 
> 
> 
> On Sun, Nov 22, 2020 at 11:11 PM Piotr Żelasko <notifications@github.com>
> wrote:
> 
> > By default it's twice of that (it helps the heuristic pack cuts better),
> > but can be adjusted; see:
> > https://github.com/lhotse-speech/lhotse/blob/master/lhotse/dataset/speech_recognition.py#L136
> >
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> > <https://github.com/lhotse-speech/lhotse/issues/146#issuecomment-731762618>,
> > or unsubscribe
> > <https://github.com/notifications/unsubscribe-auth/AAZFLO7HI4M4Q5H2S4VW7CLSRES3DANCNFSM4T6PLHFA>
> > .
> >
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.


Resolving patch:
diff --git a/lhotse/dataset/speech_recognition.py b/lhotse/dataset/speech_recognition.py
--- a/lhotse/dataset/speech_recognition.py
+++ b/lhotse/dataset/speech_recognition.py
@@ -112,7 +112,7 @@ def __init__(
             shuffle: bool = False,
             concat_cuts: bool = True,
             concat_cuts_gap: Seconds = 1.0,
-            concat_cuts_duration_factor: float = 2
+            concat_cuts_duration_factor: float = 1
     ):
         """
         K2 ASR IterableDataset constructor.
@@ -134,7 +134,7 @@ def __init__(
         :param concat_cuts_gap: The duration of silence in seconds that is inserted between the cuts;
             it's goal is to let the model "know" that there are separate utterances in a single example.
         :param concat_cuts_duration_factor: Determines the maximum duration of the concatenated cuts;
-            by default it's twice the duration of the longest cut in the batch.
+            by default it's 1, setting the limit at the duration of the longest cut in the batch.
         """
         super().__init__()
         # Initialize the fields


Test patch: