Repository: pytorch/vision

Issue:
Dataset MovingMNIST: `split=None` returns test dataset
### 🐛 Describe the bug

I've found a bug in the code for torchvision's MovingMNIST dataset, which causes only the test dataset to be returned when split=None. According to the documentation, when split is set to None, the entire dataset should be returned. However, this is not currently happening.
https://github.com/pytorch/vision/blob/b403bfc771e0caf31efd06d43860b09004f4ac61/torchvision/datasets/moving_mnist.py#L13-L19

I've tested this with the following code:
```python
from torchvision import datasets
import torch

dataset = datasets.MovingMNIST(root="data", download=True)
dataset[0].size() # returns torch.Size([10, 1, 64, 64]), but I expected torch.Size([20, 1, 64, 64])
```

I believe the bug is caused by lines 58-62 in the code, which handle None and test splits together:
https://github.com/pytorch/vision/blob/b403bfc771e0caf31efd06d43860b09004f4ac61/torchvision/datasets/moving_mnist.py#L42-L62

To fix this, I propose the following two changes:
- Separate the handling of None and test splits in the code.
- Only process lines 46-50 when split is not None.

Reference issue: #6981 

I'm happy to help on this issue, please assign to me on this one.

### Versions

PyTorch version: 2.0.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.2.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:26:08) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.2.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.2
[pip3] torch==2.0.0
[pip3] torch-tb-profiler==0.4.1
[pip3] torchvision==0.15.1
[conda] numpy                     1.24.2          py310h3d2048e_0    conda-forge
[conda] pytorch                   2.0.0                  py3.10_0    pytorch
[conda] torch                     2.0.0                    pypi_0    pypi
[conda] torch-tb-profiler         0.4.1                    pypi_0    pypi
[conda] torchvision               0.15.1                   pypi_0    pypi

cc @pmeier
Dataset MovingMNIST: `split=None` returns test dataset
### 🐛 Describe the bug

I've found a bug in the code for torchvision's MovingMNIST dataset, which causes only the test dataset to be returned when split=None. According to the documentation, when split is set to None, the entire dataset should be returned. However, this is not currently happening.
https://github.com/pytorch/vision/blob/b403bfc771e0caf31efd06d43860b09004f4ac61/torchvision/datasets/moving_mnist.py#L13-L19

I've tested this with the following code:
```python
from torchvision import datasets
import torch

dataset = datasets.MovingMNIST(root="data", download=True)
dataset[0].size() # returns torch.Size([10, 1, 64, 64]), but I expected torch.Size([20, 1, 64, 64])
```

I believe the bug is caused by lines 58-62 in the code, which handle None and test splits together:
https://github.com/pytorch/vision/blob/b403bfc771e0caf31efd06d43860b09004f4ac61/torchvision/datasets/moving_mnist.py#L42-L62

To fix this, I propose the following two changes:
- Separate the handling of None and test splits in the code.
- Only process lines 46-50 when split is not None.

Reference issue: #6981 

I'm happy to help on this issue, please assign to me on this one.

### Versions

PyTorch version: 2.0.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.2.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:26:08) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.2.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.2
[pip3] torch==2.0.0
[pip3] torch-tb-profiler==0.4.1
[pip3] torchvision==0.15.1
[conda] numpy                     1.24.2          py310h3d2048e_0    conda-forge
[conda] pytorch                   2.0.0                  py3.10_0    pytorch
[conda] torch                     2.0.0                    pypi_0    pypi
[conda] torch-tb-profiler         0.4.1                    pypi_0    pypi
[conda] torchvision               0.15.1                   pypi_0    pypi

cc @pmeier


Discussion:
Thanks for the detailed report @Shu-Wan! This indeed seems wrong. IIUC, it should be enough to replace the `else` branch with an `elif`:

```diff
diff --git a/torchvision/datasets/moving_mnist.py b/torchvision/datasets/moving_mnist.py
index afff0bfa3b..ac5a2b1503 100644
--- a/torchvision/datasets/moving_mnist.py
+++ b/torchvision/datasets/moving_mnist.py
@@ -58,7 +58,7 @@ class MovingMNIST(VisionDataset):
         data = torch.from_numpy(np.load(os.path.join(self._base_folder, self._filename)))
         if self.split == "train":
             data = data[: self.split_ratio]
-        else:
+        elif self.split == "test":
             data = data[self.split_ratio :]
         self.data = data.transpose(0, 1).unsqueeze(2).contiguous()
```

Wondering why our tests missed this.

> I'm happy to help on this issue, please assign to me on this one.

Go for it!
Welp, that was not smart:

https://github.com/pytorch/vision/blob/f0a1df315db3c0fdec3d638895f6173809030bba/test/test_datasets.py#L1520-L1529

We should assert there that the second dimension has 20 elements for `split=None`.
Let's also change 

https://github.com/pytorch/vision/blob/f0a1df315db3c0fdec3d638895f6173809030bba/test/test_datasets.py#L1510

to a different number like 5 or whatever to avoid confusing the number of samples with the number of frames.
@Shu-Wan we are preparing for the `0.15.2` bug fix release. Since `MovingMNIST` was released with 0.15, it would be good to get this fix in. Do you happen to have time to send patch soon (within one week)? If not, are you ok with me taking over so we can get it released?
Hi Philip,

I will take care of this today. It should be an easy fix.

On Mar 22, 2023, at 4:44 AM, Philip Meier ***@***.***> wrote:

﻿

@Shu-Wan<https://github.com/Shu-Wan> we are preparing for the 0.15.2 bug fix release. Since MovingMNIST was released with 0.15, it would be good to get this fix in. Do you happen to have time to send patch soon? If not, are you ok with me taking over so we can get it released?

—
Reply to this email directly, view it on GitHub<https://github.com/pytorch/vision/issues/7439#issuecomment-1479408790>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADZWW7PU6LFZRMQWOKOZ6YTW5LQZJANCNFSM6AAAAAAWDFFL5U>.
You are receiving this because you were mentioned.Message ID: ***@***.***>

Thanks for the detailed report @Shu-Wan! This indeed seems wrong. IIUC, it should be enough to replace the `else` branch with an `elif`:

```diff
diff --git a/torchvision/datasets/moving_mnist.py b/torchvision/datasets/moving_mnist.py
index afff0bfa3b..ac5a2b1503 100644
--- a/torchvision/datasets/moving_mnist.py
+++ b/torchvision/datasets/moving_mnist.py
@@ -58,7 +58,7 @@ class MovingMNIST(VisionDataset):
         data = torch.from_numpy(np.load(os.path.join(self._base_folder, self._filename)))
         if self.split == "train":
             data = data[: self.split_ratio]
-        else:
+        elif self.split == "test":
             data = data[self.split_ratio :]
         self.data = data.transpose(0, 1).unsqueeze(2).contiguous()
```

Wondering why our tests missed this.

> I'm happy to help on this issue, please assign to me on this one.

Go for it!
Welp, that was not smart:

https://github.com/pytorch/vision/blob/f0a1df315db3c0fdec3d638895f6173809030bba/test/test_datasets.py#L1520-L1529

We should assert there that the second dimension has 20 elements for `split=None`.
Let's also change 

https://github.com/pytorch/vision/blob/f0a1df315db3c0fdec3d638895f6173809030bba/test/test_datasets.py#L1510

to a different number like 5 or whatever to avoid confusing the number of samples with the number of frames.
@Shu-Wan we are preparing for the `0.15.2` bug fix release. Since `MovingMNIST` was released with 0.15, it would be good to get this fix in. Do you happen to have time to send patch soon (within one week)? If not, are you ok with me taking over so we can get it released?
Hi Philip,

I will take care of this today. It should be an easy fix.

On Mar 22, 2023, at 4:44 AM, Philip Meier ***@***.***> wrote:

﻿

@Shu-Wan<https://github.com/Shu-Wan> we are preparing for the 0.15.2 bug fix release. Since MovingMNIST was released with 0.15, it would be good to get this fix in. Do you happen to have time to send patch soon? If not, are you ok with me taking over so we can get it released?

—
Reply to this email directly, view it on GitHub<https://github.com/pytorch/vision/issues/7439#issuecomment-1479408790>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADZWW7PU6LFZRMQWOKOZ6YTW5LQZJANCNFSM6AAAAAAWDFFL5U>.
You are receiving this because you were mentioned.Message ID: ***@***.***>


Resolving patch:
diff --git a/torchvision/datasets/moving_mnist.py b/torchvision/datasets/moving_mnist.py
--- a/torchvision/datasets/moving_mnist.py
+++ b/torchvision/datasets/moving_mnist.py
@@ -58,7 +58,7 @@ def __init__(
         data = torch.from_numpy(np.load(os.path.join(self._base_folder, self._filename)))
         if self.split == "train":
             data = data[: self.split_ratio]
-        else:
+        elif self.split == "test":
             data = data[self.split_ratio :]
         self.data = data.transpose(0, 1).unsqueeze(2).contiguous()
 


Test patch:
diff --git a/test/test_datasets.py b/test/test_datasets.py
--- a/test/test_datasets.py
+++ b/test/test_datasets.py
@@ -1504,14 +1504,16 @@ class MovingMNISTTestCase(datasets_utils.DatasetTestCase):
 
     ADDITIONAL_CONFIGS = combinations_grid(split=(None, "train", "test"), split_ratio=(10, 1, 19))
 
+    _NUM_FRAMES = 20
+
     def inject_fake_data(self, tmpdir, config):
         base_folder = os.path.join(tmpdir, self.DATASET_CLASS.__name__)
         os.makedirs(base_folder, exist_ok=True)
-        num_samples = 20
+        num_samples = 5
         data = np.concatenate(
             [
                 np.zeros((config["split_ratio"], num_samples, 64, 64)),
-                np.ones((20 - config["split_ratio"], num_samples, 64, 64)),
+                np.ones((self._NUM_FRAMES - config["split_ratio"], num_samples, 64, 64)),
             ]
         )
         np.save(os.path.join(base_folder, "mnist_test_seq.npy"), data)
@@ -1519,14 +1521,13 @@ def inject_fake_data(self, tmpdir, config):
 
     @datasets_utils.test_all_configs
     def test_split(self, config):
-        if config["split"] is None:
-            return
-
-        with self.create_dataset(config) as (dataset, info):
+        with self.create_dataset(config) as (dataset, _):
             if config["split"] == "train":
                 assert (dataset.data == 0).all()
-            else:
+            elif config["split"] == "test":
                 assert (dataset.data == 1).all()
+            else:
+                assert dataset.data.size()[1] == self._NUM_FRAMES
 
 
 class DatasetFolderTestCase(datasets_utils.ImageDatasetTestCase):