internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/vision

Issue:
python_type_check CI is broken
This looks like an upstream change that broke CI, see https://app.circleci.com/pipelines/github/pytorch/vision/4305/workflows/9ae30f70-6fa0-419f-966b-8db2e249a12b/jobs/254769 for an example

The error we get is
```
torchvision/datasets/mnist.py:446: error:
Incompatible
return
value
type
(got
"GzipFile",
expected
"IO[Any]")

[return-value]
            return gzip.open(path, 'rb')
                   ^
Found 1 error in 1 file (checked 97 source files)
```

cc @seemethere @pmeier


Discussion:
I'm on it.
This is caused by [`mypy==0.790`](https://mypy-lang.blogspot.com/2020/10/mypy-0790-released.html) released two days ago. It includes python/typeshed@846d922df238d307a99347f74d91ccb362f876ad which amongst others makes the return types of the `gzip` module more precise.

Resolving patch:
diff --git a/torchvision/datasets/mnist.py b/torchvision/datasets/mnist.py
--- a/torchvision/datasets/mnist.py
+++ b/torchvision/datasets/mnist.py
@@ -7,6 +7,8 @@
 import torch
 import codecs
 import string
+import gzip
+import lzma
 from typing import Any, Callable, Dict, IO, List, Optional, Tuple, Union
 from .utils import download_url, download_and_extract_archive, extract_archive, \
     verify_str_arg
@@ -435,17 +437,15 @@ def get_int(b: bytes) -> int:
     return int(codecs.encode(b, 'hex'), 16)
 
 
-def open_maybe_compressed_file(path: Union[str, IO]) -> IO:
+def open_maybe_compressed_file(path: Union[str, IO]) -> Union[IO, gzip.GzipFile]:
     """Return a file object that possibly decompresses 'path' on the fly.
        Decompression occurs when argument `path` is a string and ends with '.gz' or '.xz'.
     """
     if not isinstance(path, torch._six.string_classes):
         return path
     if path.endswith('.gz'):
-        import gzip
         return gzip.open(path, 'rb')
     if path.endswith('.xz'):
-        import lzma
         return lzma.open(path, 'rb')
     return open(path, 'rb')
 


Test patch:
Repository: pytorch/vision

Issue:
Improve error message for RandomCrop
## 🚀 Feature

In case if we apply `RandomCrop(256)` on images smaller than 256, current error message does not provide the information about size mismatch
```
import torchvision.transforms as T

t = T.RandomCrop(224)
t(torch.randint(0, 256, size=(3, 32, 32), dtype=torch.uint8))
> 
  File "/user/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 575, in forward
    i, j, h, w = self.get_params(img, self.size)
  File "/user/lib/python3.7/site-packages/torchvision/transforms/transforms.py", line 538, in get_params
    i = torch.randint(0, h - th + 1, size=(1, )).item()
RuntimeError: random_ expects 'from' to be less than 'to', but got from=0 >= to=-191
``` 

It would be nice to have a more user-friendly error message if `h - th + 1 < 0` saying that  we have a size mismatch.


cc @vfdev-5


Discussion:

Resolving patch:
diff --git a/torchvision/transforms/transforms.py b/torchvision/transforms/transforms.py
--- a/torchvision/transforms/transforms.py
+++ b/torchvision/transforms/transforms.py
@@ -532,6 +532,12 @@ def get_params(img: Tensor, output_size: Tuple[int, int]) -> Tuple[int, int, int
         """
         w, h = F._get_image_size(img)
         th, tw = output_size
+
+        if h + 1 < th or w + 1 < tw:
+            raise ValueError(
+                "Required crop size {} is larger then input image size {}".format((th, tw), (h, w))
+            )
+
         if w == tw and h == th:
             return 0, 0, h, w
 


Test patch:
diff --git a/test/test_transforms.py b/test/test_transforms.py
--- a/test/test_transforms.py
+++ b/test/test_transforms.py
@@ -298,6 +298,11 @@ def test_random_crop(self):
         self.assertEqual(result.size(1), height + 1)
         self.assertEqual(result.size(2), width + 1)
 
+        t = transforms.RandomCrop(48)
+        img = torch.ones(3, 32, 32)
+        with self.assertRaisesRegex(ValueError, r"Required crop size .+ is larger then input image size .+"):
+            t(img)
+
     def test_pad(self):
         height = random.randint(10, 32) * 2
         width = random.randint(10, 32) * 2
Repository: pytorch/vision

Issue:
RandomResizedCrop aspect_ratio is wrong
https://github.com/pytorch/vision/blob/1a04d3c265679e1a508e7cd627006aaa9ef1ccfb/torchvision/transforms/transforms.py#L699-L714

Doc says that `ratio` is the range of the ratio of the ***original aspect ratio***, yet the sampling code implies that the range is just directly the crop aspect ratio.


Discussion:
I agree the documentation can be improved to mention that the aspect ratio is associated with the aspect ratio of the crop before resizing.

PRs improving the documentation are welcome

Resolving patch:
diff --git a/torchvision/transforms/transforms.py b/torchvision/transforms/transforms.py
--- a/torchvision/transforms/transforms.py
+++ b/torchvision/transforms/transforms.py
@@ -738,8 +738,8 @@ class RandomResizedCrop(torch.nn.Module):
         size (int or sequence): expected output size of each edge. If size is an
             int instead of sequence like (h, w), a square output size ``(size, size)`` is
             made. If provided a tuple or list of length 1, it will be interpreted as (size[0], size[0]).
-        scale (tuple of float): range of size of the origin size cropped
-        ratio (tuple of float): range of aspect ratio of the origin aspect ratio cropped.
+        scale (tuple of float): scale range of the cropped image before resizing, relatively to the origin image.
+        ratio (tuple of float): aspect ratio range of the cropped image before resizing.
         interpolation (int): Desired interpolation enum defined by `filters`_.
             Default is ``PIL.Image.BILINEAR``. If input is Tensor, only ``PIL.Image.NEAREST``, ``PIL.Image.BILINEAR``
             and ``PIL.Image.BICUBIC`` are supported.


Test patch:
Repository: pytorch/vision

Issue:
[DOC] Documentation for sigmoid focal loss
## 📚 Documentation

Sigmoid focal loss was added to torchvision, but I think docs are not generated for it

It was initalized in `init` as well [here](https://github.com/pytorch/vision/blob/v0.8.0/torchvision/ops/__init__.py#L11)

I think it's really minor fix, we might need `..autodoc::sigmoid_focal_loss` and it should be fine.


Discussion:
A PR adding it would be great!

Resolving patch:
diff --git a/torchvision/ops/focal_loss.py b/torchvision/ops/focal_loss.py
--- a/torchvision/ops/focal_loss.py
+++ b/torchvision/ops/focal_loss.py
@@ -3,8 +3,8 @@
 
 
 def sigmoid_focal_loss(
-    inputs,
-    targets,
+    inputs: torch.Tensor,
+    targets: torch.Tensor,
     alpha: float = 0.25,
     gamma: float = 2,
     reduction: str = "none",
@@ -12,11 +12,12 @@ def sigmoid_focal_loss(
     """
     Original implementation from https://github.com/facebookresearch/fvcore/blob/master/fvcore/nn/focal_loss.py .
     Loss used in RetinaNet for dense detection: https://arxiv.org/abs/1708.02002.
-    Args:
+
+    Arguments:
         inputs: A float tensor of arbitrary shape.
                 The predictions for each example.
         targets: A float tensor with the same shape as inputs. Stores the binary
-                 classification label for each element in inputs
+                classification label for each element in inputs
                 (0 for the negative class and 1 for the positive class).
         alpha: (optional) Weighting factor in range (0,1) to balance
                 positive vs negative examples or -1 for ignore. Default = 0.25


Test patch:
Repository: pytorch/vision

Issue:
warning: ignoring #pragma omp parallel
## 🐛 Bug

I'm seeing a lot of `warning: ignoring #pragma omp parallel` like this:
<img width="1085" alt="image" src="https://user-images.githubusercontent.com/5203025/95647645-b5553600-0b03-11eb-94e1-4ad6caf7215d.png">

Maybe there should be a `-fopenmp`?

## Environment

 - PyTorch / torchvision Version (e.g., 1.0 / 0.4.0): master/master
 - OS (e.g., Linux): Ubuntu 18.04
 - How you installed PyTorch / torchvision (`conda`, `pip`, source): source
 - Build command you used (if compiling from source): cmake+ninja+gcc-8
 - Python version: 3.6
 - CUDA/cuDNN version: 11.0/8


Discussion:
Thanks for the report. We don't directly use omp parallel in nms_cpu, so this might be somewhere else.

We will have a look.
@vfdev-5 Any news so far?
@xkszltl i'll try to take a look at this issue this week.
@vfdev-5 any update? :) I'm having the same problem
@klvnptr Sorry, seems like I missed this issue in my task list. Let me check this on monday. Thanks. 
thanks for getting back @vfdev-5 
for me this solved the problem:
$ apt-get install libopenmpi-dev
$ CFLAGS="-fopenmp" python3 setup.py bdist_wheel

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -175,7 +175,9 @@ def get_extensions():
 
     define_macros = []
 
-    extra_compile_args = {}
+    extra_compile_args = {
+        'cxx': []
+    }
     if (torch.cuda.is_available() and ((CUDA_HOME is not None) or is_rocm_pytorch)) \
             or os.getenv('FORCE_CUDA', '0') == '1':
         extension = CUDAExtension
@@ -190,16 +192,13 @@ def get_extensions():
         else:
             define_macros += [('WITH_HIP', None)]
             nvcc_flags = []
-        extra_compile_args = {
-            'cxx': [],
-            'nvcc': nvcc_flags,
-        }
+        extra_compile_args['nvcc'] = nvcc_flags
 
     if sys.platform == 'win32':
         define_macros += [('torchvision_EXPORTS', None)]
-
-        extra_compile_args.setdefault('cxx', [])
         extra_compile_args['cxx'].append('/MP')
+    elif sys.platform == 'linux':
+        extra_compile_args['cxx'].append('-fopenmp')
 
     debug_mode = os.getenv('DEBUG', '0') == '1'
     if debug_mode:


Test patch:
Repository: pytorch/vision

Issue:
How to install torchvision to use video_reader backend?
I simply installed torchvision from conda (as advertised on pytorch.org). But `torchvision.set_video_backend('video_reader')` prints `video_reader video backend is not available. Please compile torchvision from source and try again`. This should be mentioned in https://pytorch.org/docs/stable/torchvision/index.html#torchvision.set_video_backend and in torchvision README (including if the `video_reader` is temporarily not supported)

cc @bjuncek


Discussion:
Hi Vadim - thanks for raising this - I'll add this to the docs.

As for how to do it, you need to have FFMPEG installed and build torchvision from source. 
If you're using the latest master, I believe you need to add `NO_FFMPEG=False` flag as well (pls check[ this issue](https://github.com/pytorch/vision/issues/3188) for clarification). 


Resolving patch:
diff --git a/torchvision/__init__.py b/torchvision/__init__.py
--- a/torchvision/__init__.py
+++ b/torchvision/__init__.py
@@ -64,6 +64,10 @@ def set_video_backend(backend):
             The :mod:`video_reader` package includes a native C++ implementation on
             top of FFMPEG libraries, and a python API of TorchScript custom operator.
             It is generally decoding faster than :mod:`pyav`, but perhaps is less robust.
+
+    .. note::
+        Building with FFMPEG is disabled by default in the latest master. If you want to use the 'video_reader'
+        backend, please compile torchvision from source.
     """
     global _video_backend
     if backend not in ["pyav", "video_reader"]:


Test patch:
Repository: pytorch/vision

Issue:
a question about  segmentation model loading
## ❓ Questions and Help
Why they are different？
### Please note that this issue tracker is not a help form and this issue will be closed.
![image](https://user-images.githubusercontent.com/32593161/106227085-8d5d2000-6223-11eb-9c66-fcb037faac92.png)

We have a set of [listed resources available on the website](https://pytorch.org/resources). Our primary means of support is our discussion forum:

- [Discussion Forum](https://discuss.pytorch.org/)


cc @vfdev-5


Discussion:
I want to load fcn_resnet101, why it downloads resnet101?
@njzyxiong by default, segmentation model setups an ImageNet pretrained backbone, resnet101 in your case. If you would like to disable this, please set `pretrained_backbone=False`, but I agree that if `pretrained=True` we could automatically disable it.

Resolving patch:
diff --git a/torchvision/models/segmentation/segmentation.py b/torchvision/models/segmentation/segmentation.py
--- a/torchvision/models/segmentation/segmentation.py
+++ b/torchvision/models/segmentation/segmentation.py
@@ -69,6 +69,7 @@ def _segm_model(name, backbone_name, num_classes, aux, pretrained_backbone=True)
 def _load_model(arch_type, backbone, pretrained, progress, num_classes, aux_loss, **kwargs):
     if pretrained:
         aux_loss = True
+        kwargs["pretrained_backbone"] = False
     model = _segm_model(arch_type, backbone, num_classes, aux_loss, **kwargs)
     if pretrained:
         _load_weights(model, arch_type, backbone, progress)


Test patch:
Repository: pytorch/vision

Issue:
What is expected Kinetics400 dataset directory structure?
Given that the dataset does not come with official downloader scripts and that most roll their own or hack some third-party scripts, it would be much clearer if https://pytorch.org/docs/stable/torchvision/datasets.html#kinetics-400 explained what directory structure is expected by `torchvision.datasets.Kinetics400`

What is the expected dataset size? and the video file extensions?

Thanks!

cc @pmeier


Discussion:
@vadimkantorov maybe notes from this ref example could help: https://github.com/pytorch/vision/tree/master/references/video_classification#data-preparation

cc @bjuncek 
This is not very helpful, since it does not mention explicitly the class-label directory organisation. It is also not clear if root_dir should contain the splits or if the root_dir is the path to a chosen split.
Unfortunately, I removed the dataset from my storage where I tried a minimal training of the ref example. 
Maybe, Bruno could share more info about that...
The problem is that the dataset does not have official download or dataset stats scripts or recommended directory structure. From what I found, the root_dir should be the path to the split, the split directory should contain directories for every class labels with videos inside them. But this should better be mentioned explicitly in the docs
I agree. I had a similar question in my case... If you'd like to improve the docs, feel free to send a PR.

Resolving patch:
diff --git a/torchvision/datasets/kinetics.py b/torchvision/datasets/kinetics.py
--- a/torchvision/datasets/kinetics.py
+++ b/torchvision/datasets/kinetics.py
@@ -23,7 +23,18 @@ class Kinetics400(VisionDataset):
     Internally, it uses a VideoClips object to handle clip creation.
 
     Args:
-        root (string): Root directory of the Kinetics-400 Dataset.
+        root (string): Root directory of the Kinetics-400 Dataset. Should be structured as follows:
+            .. code::
+
+                root/
+                ├── class1
+                │   ├── clip1.avi
+                │   ├── clip2.avi
+                │   └── ...
+                └── class2
+                    ├── clipx.avi
+                    └── ...
+
         frames_per_clip (int): number of frames in a clip
         step_between_clips (int): number of frames between each clip
         transform (callable, optional): A function/transform that  takes in a TxHxWxC video


Test patch:
Repository: pytorch/vision

Issue:
Add Docs for AutoAugment methods.
## 📚 Documentation

Add documentation for AutoAugment,
[AutoAugmentation](https://pytorch.org/blog/pytorch-1.8-new-library-releases/#stable-autoaugment) was added as a stable feature in the latest 0.9 release.

I think we are missing the docs on transforms page for both [master](https://pytorch.org/vision/master/transforms.html) as well as [0.9 docs](https://pytorch.org/vision/0.9/transforms.html)


I think the following needs to be added.

- [x] A small note and description as done in the [release page](https://github.com/pytorch/vision/releases/tag/v0.9.0)
- [x] Autodoc the methods in [`torchvision/transforms/autoaugment.py`](https://github.com/pytorch/vision/blob/v0.9.0/torchvision/transforms/autoaugment.py) file.
- [x] Also add a `__all__`in above file. I think only [AutoAugment](https://github.com/pytorch/vision/blob/01dfa8ea81972bb74b52dc01e6a1b43b26b62020/torchvision/transforms/autoaugment.py#L127) and [AutoAugmentPolicy](https://github.com/pytorch/vision/blob/01dfa8ea81972bb74b52dc01e6a1b43b26b62020/torchvision/transforms/autoaugment.py#L11) are supposed to be added.

I can send a PR for this. Do let me know!




Discussion:
@oke-aditya Good catch! 

I should have done this when I sent the original PR. Could you please send a PR to fix your points?

Note that `AutoAugmentPolicy` is also supposed to be public (I edited your issue in place to indicate this).
Great. Let me send a PR for master.

Resolving patch:
diff --git a/torchvision/transforms/autoaugment.py b/torchvision/transforms/autoaugment.py
--- a/torchvision/transforms/autoaugment.py
+++ b/torchvision/transforms/autoaugment.py
@@ -7,9 +7,12 @@
 
 from . import functional as F, InterpolationMode
 
+__all__ = ["AutoAugmentPolicy", "AutoAugment"]
+
 
 class AutoAugmentPolicy(Enum):
     """AutoAugment policies learned on different datasets.
+    Available policies are IMAGENET, CIFAR10 and SVHN.
     """
     IMAGENET = "imagenet"
     CIFAR10 = "cifar10"


Test patch:
Repository: pytorch/vision

Issue:
Places365 dataset redundant path.join under _check_integrity
## 🐛 Bug

Redundant path.join at line 165 under _check_integrity in torchvision/datasets/places365.py

path.join is already called at line 116 and line 131 under load_categories and load_file_list

cc @pmeier


Discussion:
While I at first was under the impression that this is a serious bug, it turns out @YellowPig-zp was right and it is just redundant. The documentation of `os.path.join` [states](https://docs.python.org/3/library/os.path.html#os.path.join):

> If a component is an absolute path, all previous components are thrown away and joining continues from the absolute path component.

Example:

```python
import os

root = os.path.join(os.sep, "root")
filename = "file"
file = os.path.join(root, filename)


assert os.path.join(root, file) == file

```

Since `file` is an absolute path after

https://github.com/pytorch/vision/blob/7992eb5da9c2e67469734e43f3d07e242d4f5273/torchvision/datasets/places365.py#L115-L116

or

https://github.com/pytorch/vision/blob/7992eb5da9c2e67469734e43f3d07e242d4f5273/torchvision/datasets/places365.py#L130-L131

the `path.join` in 

https://github.com/pytorch/vision/blob/7992eb5da9c2e67469734e43f3d07e242d4f5273/torchvision/datasets/places365.py#L165

is equal to `file`.

---



Resolving patch:
diff --git a/torchvision/datasets/places365.py b/torchvision/datasets/places365.py
--- a/torchvision/datasets/places365.py
+++ b/torchvision/datasets/places365.py
@@ -162,7 +162,7 @@ def _verify_split(self, split: str) -> str:
         return verify_str_arg(split, "split", self._SPLITS)
 
     def _check_integrity(self, file: str, md5: str, download: bool) -> bool:
-        integrity = check_integrity(path.join(self.root, file), md5=md5)
+        integrity = check_integrity(file, md5=md5)
         if not integrity and not download:
             raise RuntimeError(
                 f"The file {file} does not exist or is corrupted. You can set download=True to download it."


Test patch:
Repository: pytorch/vision

Issue:
STL10 does not download
## 🐛 Bug

I am currently working on the STL10 dataset, but it seems that there is some problem downloading the data. It gets stuck while downloading judging from the stack trace. I hunted down the URL from the dataset (http://ai.stanford.edu/~acoates/stl10/stl10_binary.tar.gz) and there does not seem to be a problem from their side. Does somebody else have the same problem?

## To Reproduce

Steps to reproduce the behaviour:

1. Import torchvision
1. Instantiate a STL10 Dataset

```
import torchvision

torchvision.datasets.STL10(root="datasets", split='unlabeled', folds=None, download=True)
```

## Expected behaviour

The dataset should be downloaded and extracted.

## Environment
PyTorch version: 1.8.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.16.3

Python version: 3.8 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.1
[pip3] pytorch-lightning==1.1.8
[pip3] torch==1.8.0+cpu
[pip3] torchaudio==0.8.0
[pip3] torchvision==0.9.0


cc @pmeier


Discussion:

Resolving patch:
diff --git a/torchvision/datasets/utils.py b/torchvision/datasets/utils.py
--- a/torchvision/datasets/utils.py
+++ b/torchvision/datasets/utils.py
@@ -61,18 +61,20 @@ def check_integrity(fpath: str, md5: Optional[str] = None) -> bool:
     return check_md5(fpath, md5)
 
 
-def _get_redirect_url(url: str, max_hops: int = 10) -> str:
-    import requests
-
-    for hop in range(max_hops + 1):
-        response = requests.get(url)
+def _get_redirect_url(url: str, max_hops: int = 3) -> str:
+    initial_url = url
+    headers = {"Method": "HEAD", "User-Agent": USER_AGENT}
 
-        if response.url == url or response.url is None:
-            return url
+    for _ in range(max_hops + 1):
+        with urllib.request.urlopen(urllib.request.Request(url, headers=headers)) as response:
+            if response.url == url or response.url is None:
+                return url
 
-        url = response.url
+            url = response.url
     else:
-        raise RecursionError(f"Too many redirects: {max_hops + 1})")
+        raise RecursionError(
+            f"Request to {initial_url} exceeded {max_hops} redirects. The last redirect points to {url}."
+        )
 
 
 def _get_google_drive_file_id(url: str) -> Optional[str]:


Test patch:
Repository: pytorch/vision

Issue:
EMNIST download fails
Oh no, something went wrong in the scheduled workflow tests/download. 
Please look into it:

https://github.com/pytorch/vision/actions/runs/626923202

Feel free to close this if this was just a one-off error.


cc @pmeier


Discussion:
Does somebody possibly know when the fix will propagate to Pypi ?
@wittenator There is currently no fix. We haven't decided yet what the way forward should look like. Either we add another mirror or disable the download for `EMNIST` all together.
Ah my bad, I thought that this would be a duplicate of the redirection bug for STL10. Do you possibly know when this will available in Pypi?

Resolving patch:
diff --git a/torchvision/datasets/mnist.py b/torchvision/datasets/mnist.py
--- a/torchvision/datasets/mnist.py
+++ b/torchvision/datasets/mnist.py
@@ -268,11 +268,7 @@ class EMNIST(MNIST):
         target_transform (callable, optional): A function/transform that takes in the
             target and transforms it.
     """
-    # Updated URL from https://www.nist.gov/node/1298471/emnist-dataset since the
-    # _official_ download link
-    # https://cloudstor.aarnet.edu.au/plus/s/ZNmuFiuQTqZlu9W/download
-    # is (currently) unavailable
-    url = 'http://www.itl.nist.gov/iaui/vip/cs_links/EMNIST/gzip.zip'
+    url = 'https://www.itl.nist.gov/iaui/vip/cs_links/EMNIST/gzip.zip'
     md5 = "58c8d27c78d21e728a6bc7b3cc06412e"
     splits = ('byclass', 'bymerge', 'balanced', 'letters', 'digits', 'mnist')
     # Merged Classes assumes Same structure for both uppercase and lowercase version


Test patch: