Finnish-NLP
/

llama-7b-finnish-v2

Text Generation

text-generation-inference

Model card Files Files and versions Community

aapot commited on Mar 13

Commit

daa4cb1

•

1 Parent(s): d2f0aae

fix

Files changed (1) hide show

EasyLM/data.py +2 -0

EasyLM/data.py CHANGED Viewed

@@ -217,6 +217,8 @@ class HuggingfaceDataset(object):
             if self._eval_dataset:
                 break
             else:
                 self._dataset_loc = 0
                 self._train_epochs += 1

             if self._eval_dataset:
                 break
             else:
+                if self._train_epochs == 0:
+                    self._dataset = self._dataset.shuffle(seed=42, buffer_size=10000)
                 self._dataset_loc = 0
                 self._train_epochs += 1