Repository: rwth-i6/returnn

Issue:
LmDataset: Not a single seq was loaded
This issue when using version later than 9611cd2bdc7680daadce35134a2b0c5684b71d6f

```
EXCEPTION
Traceback (most recent call last):
  File "/home/wwang/returnn_20231019/returnn/returnn/datasets/cached2.py", line 133, in CachedDataset2.is_less_than_num_seqs
    line: return super(CachedDataset2, self).is_less_than_num_seqs(n)
    locals:
      super = <builtin> <class 'super'>
      CachedDataset2 = <global> <class 'returnn.datasets.cached2.CachedDataset2'>
      self = <local> <LmDataset 'train' epoch=None>
      is_less_than_num_seqs = <not found>
      n = <local> 0
  File "/home/wwang/returnn_20231019/returnn/returnn/datasets/basic.py", line 981, in Dataset.is_less_than_num_seqs
    line: return n < self.num_seqs
    locals:
      n = <local> 0
      self = <local> <LmDataset 'train' epoch=None>
      self.num_seqs = <local> !NotImplementedError:
  File "/home/wwang/returnn_20231019/returnn/returnn/datasets/cached2.py", line 101, in CachedDataset2.num_seqs
    line: raise NotImplementedError
    locals:
      NotImplementedError = <builtin> <class 'NotImplementedError'>
NotImplementedError

During handling of the above exception, another exception occurred:

EXCEPTION
Traceback (most recent call last):
  File "/home/wwang/returnn_20231019/returnn/rnn.py", line 11, in <module>
    line: main()
    locals:
      main = <local> <function main at 0x7ff66b4b0d30>
  File "/home/wwang/returnn_20231019/returnn/returnn/__main__.py", line 626, in main
    line: execute_main_task()
    locals:
      execute_main_task = <global> <function execute_main_task at 0x7ff66b4b0c10>
  File "/home/wwang/returnn_20231019/returnn/returnn/__main__.py", line 433, in execute_main_task
    line: train_data and train_data.have_seqs()
    locals:
      train_data = <global> <LmDataset 'train' epoch=None>
      train_data.have_seqs = <global> <bound method Dataset.have_seqs of <LmDataset 'train' epoch=None>>
  File "/home/wwang/returnn_20231019/returnn/returnn/datasets/basic.py", line 956, in Dataset.have_seqs
    line: return self.is_less_than_num_seqs(0)
    locals:
      self = <local> <LmDataset 'train' epoch=None>
      self.is_less_than_num_seqs = <local> <bound method CachedDataset2.is_less_than_num_seqs of <LmDataset 'train' epoch=None>>
  File "/home/wwang/returnn_20231019/returnn/returnn/datasets/cached2.py", line 140, in CachedDataset2.is_less_than_num_seqs
    line: assert self.added_data, "Not a single seq was loaded?"
    locals:
      self = <local> <LmDataset 'train' epoch=None>
      self.added_data = <local> []
AssertionError: Not a single seq was loaded?
```

Example config: `/nas/models/asr/lm/en/lstmlm-20231017/tmp/bgd.cfg`


Discussion:
@michelwi mentioned that this is related to a similar issue in `MultiProcDataset` which was fixed in https://github.com/rwth-i6/returnn/pull/1374.
> @michelwi mentioned that this is related to a similar issue in `MultiProcDataset` which was fixed in #1374.

To relativate that statement: I did not do any investigation and just conjectured that the problems might be related.
@michelwi do you have the exact error + stacktrace you got somewhere?
I think the commit e215b19d5dd80ad2ac118a2032e80b09530fc5f3 probably was also related to fix this for `MetaDataset`.

Resolving patch:
diff --git a/returnn/datasets/basic.py b/returnn/datasets/basic.py
--- a/returnn/datasets/basic.py
+++ b/returnn/datasets/basic.py
@@ -956,7 +956,9 @@ def have_seqs(self) -> bool:
             return total_num_seqs > 0
         except NotImplementedError:
             pass
-        return self.is_less_than_num_seqs(0)
+        if self.epoch is not None:
+            return self.is_less_than_num_seqs(0)
+        raise NotImplementedError(f"{self} have_seqs() is not implemented (and neither get_total_num_seqs())")
 
     def len_info(self):
         """
diff --git a/returnn/datasets/cached2.py b/returnn/datasets/cached2.py
--- a/returnn/datasets/cached2.py
+++ b/returnn/datasets/cached2.py
@@ -98,6 +98,8 @@ def num_seqs(self):
         """
         if self._num_seqs is not None:
             return self._num_seqs
+        if self.epoch is None:
+            return 0
         raise NotImplementedError
 
     def _load_seqs(self, start, end):
@@ -128,6 +130,8 @@ def is_less_than_num_seqs(self, n):
         """
         if n < self.expected_load_seq_start:
             return True
+        if self.epoch is None:
+            return False
         # noinspection PyBroadException
         try:
             return super(CachedDataset2, self).is_less_than_num_seqs(n)
diff --git a/returnn/datasets/lm.py b/returnn/datasets/lm.py
--- a/returnn/datasets/lm.py
+++ b/returnn/datasets/lm.py
@@ -303,6 +303,10 @@ def supports_seq_order_sorting(self) -> bool:
         """supports sorting"""
         return True
 
+    def get_total_num_seqs(self) -> int:
+        """total num seqs"""
+        return len(self.orths)
+
     def _reduce_log_skipped_seqs(self):
         if isinstance(self.log_skipped_seqs, bool):
             return


Test patch: