Repository: pytorch/text

Issue:
BPTTIterator: Result of Slicing is an empty tensor
I can't iterate through all the batches in the iterator returned by BPTTITerator.
For example, this will give an error: batches = [batch for batch in train_iter]
On the last batch, I'll get error: Result of Slicing is an empty tensor

How should I fix it?

Thanks!
BPTTIterator: Result of Slicing is an empty tensor
I can't iterate through all the batches in the iterator returned by BPTTITerator.
For example, this will give an error: batches = [batch for batch in train_iter]
On the last batch, I'll get error: Result of Slicing is an empty tensor

How should I fix it?

Thanks!


Discussion:
I can't reproduce with `datasets.WikiText2` (I had to fix some unrelated bugs in that first, though). This code works for me after said unrelated fixes (PR coming momentarily):

```
from torchtext import data, datasets
TEXT = data.Field(sequential=True)
train, dev, test = datasets.WikiText2.splits(TEXT)
TEXT.build_vocab(train, dev, test)
train_iter, dev_iter, test_iter = data.BPTTIterator.splits(
    (train, dev, test), batch_size=20, bptt_len=35, device=-1, repeat=False)
for batch in train_iter:
    print(batch.text.size())
```
and prints sizes of 35 x 20 until the last batch of size 26 x 20.

Can you provide a snippet that's buggy?
Hi Sure!

So In utils.py, we have:
`
def load_ptb(ptb_path='data.zip', ptb_dir='data', bptt_len=5, batch_size=1, gpu=False, reuse=False,repeat=False, shuffle=True):

    print ("Loading Data")

    if (not reuse) or (not os.path.exists(ptb_dir)):
        f = zipfile.ZipFile(ptb_path, 'r')
        f.extractall('.')
        f.close()

    DEV = 0 if gpu else -1
    text_field = data.Field(lower=True, batch_first=True)
    train = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'train.txt'), text_field, newline_eos=False)
    val = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'valid.txt'), text_field, newline_eos=False)
    test = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'test.txt'), text_field, newline_eos=False)

    train_iter, val_iter, test_iter = data.BPTTIterator.splits((train, val, test), 
        batch_size=batch_size, bptt_len=bptt_len, device=DEV, repeat=repeat,
        shuffle=shuffle)
    text_field.build_vocab(train, vectors=torchtext.vocab.GloVe(name='6B', dim=50))
    return train_iter, val_iter, test_iter, text_field
`

And then in main.py function
Even `batches = [batch for batch in train_iter]` doesn't work

Thanks so much!
I can't reproduce with `datasets.WikiText2` (I had to fix some unrelated bugs in that first, though). This code works for me after said unrelated fixes (PR coming momentarily):

```
from torchtext import data, datasets
TEXT = data.Field(sequential=True)
train, dev, test = datasets.WikiText2.splits(TEXT)
TEXT.build_vocab(train, dev, test)
train_iter, dev_iter, test_iter = data.BPTTIterator.splits(
    (train, dev, test), batch_size=20, bptt_len=35, device=-1, repeat=False)
for batch in train_iter:
    print(batch.text.size())
```
and prints sizes of 35 x 20 until the last batch of size 26 x 20.

Can you provide a snippet that's buggy?
Hi Sure!

So In utils.py, we have:
`
def load_ptb(ptb_path='data.zip', ptb_dir='data', bptt_len=5, batch_size=1, gpu=False, reuse=False,repeat=False, shuffle=True):

    print ("Loading Data")

    if (not reuse) or (not os.path.exists(ptb_dir)):
        f = zipfile.ZipFile(ptb_path, 'r')
        f.extractall('.')
        f.close()

    DEV = 0 if gpu else -1
    text_field = data.Field(lower=True, batch_first=True)
    train = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'train.txt'), text_field, newline_eos=False)
    val = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'valid.txt'), text_field, newline_eos=False)
    test = datasets.LanguageModelingDataset(os.path.join(ptb_dir, 'test.txt'), text_field, newline_eos=False)

    train_iter, val_iter, test_iter = data.BPTTIterator.splits((train, val, test), 
        batch_size=batch_size, bptt_len=bptt_len, device=DEV, repeat=repeat,
        shuffle=shuffle)
    text_field.build_vocab(train, vectors=torchtext.vocab.GloVe(name='6B', dim=50))
    return train_iter, val_iter, test_iter, text_field
`

And then in main.py function
Even `batches = [batch for batch in train_iter]` doesn't work

Thanks so much!

Resolving patch:
diff --git a/torchtext/data/iterator.py b/torchtext/data/iterator.py
--- a/torchtext/data/iterator.py
+++ b/torchtext/data/iterator.py
@@ -224,7 +224,7 @@ def __init__(self, dataset, batch_size, bptt_len, **kwargs):
         super(BPTTIterator, self).__init__(dataset, batch_size, **kwargs)
 
     def __len__(self):
-        return math.ceil(len(self.dataset[0].text) /
+        return math.ceil((len(self.dataset[0].text) - 1) /
                          (self.batch_size * self.bptt_len))
 
     def __iter__(self):
@@ -240,7 +240,7 @@ def __iter__(self):
             ('text', TEXT), ('target', TEXT)])
         while True:
             for i in range(0, len(self) * self.bptt_len, self.bptt_len):
-                seq_len = min(self.bptt_len, len(data) - 1 - i)
+                seq_len = min(self.bptt_len, len(data) - i - 1)
                 yield Batch.fromvars(
                     dataset, self.batch_size, train=self.train,
                     text=data[i:i + seq_len],


Test patch: