internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/pytorch

Issue:
[utils.bottleneck] Handle user script arguments better
Right now,
`python -m torch.utils.bottleneck script.py arg1 arg2 arg2`
fails with a cryptic errors to the user as their user script doesn't receive the args correctly. The recommended usage right now is `python -m torch.utils.bottleneck -- script.py arg1 arg2 arg2`.

One of the following should be expected:
- bottleneck prints out a nice error message politely telling the user they've omitted the '--'
- bottleneck should be able to run without the '--'

I'm working on fixing this

cc @fmassa for the original report


Discussion:

Resolving patch:
diff --git a/torch/utils/bottleneck/__main__.py b/torch/utils/bottleneck/__main__.py
--- a/torch/utils/bottleneck/__main__.py
+++ b/torch/utils/bottleneck/__main__.py
@@ -218,7 +218,7 @@ def parse_args():
     parser.add_argument('scriptfile', type=str,
                         help='Path to the script to be run. '
                         'Usually run with `python path/to/script`.')
-    parser.add_argument('args', type=str, nargs='*',
+    parser.add_argument('args', type=str, nargs=argparse.REMAINDER,
                         help='Command-line arguments to be passed to the script.')
     return parser.parse_args()
 


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -406,12 +406,9 @@ def _run_bottleneck(self, test_file, scriptargs=''):
         curdir = os.path.dirname(os.path.abspath(__file__))
         filepath = '{}/{}'.format(curdir, test_file)
         if scriptargs != '':
-            mark = '-- '
             scriptargs = ' {}'.format(scriptargs)
-        else:
-            mark = ''
         rc, out, err = self._run(
-            'python -m torch.utils.bottleneck {}{}{}'.format(mark, filepath, scriptargs))
+            'python -m torch.utils.bottleneck {}{}'.format(filepath, scriptargs))
         return rc, out, err
 
     def _check_run_args(self):
@@ -463,7 +460,7 @@ def _check_cuda(self, output):
             self.assertIsNone(results, self._fail_msg('Should not tell users about CUDA', output))
 
     @unittest.skipIf(torch.cuda.is_available(), 'CPU-only test')
-    def test_cpu_only(self):
+    def test_bottleneck_cpu_only(self):
         rc, out, err = self._run_bottleneck('bottleneck/test.py')
         self.assertEqual(rc, 0, 'Run failed with\n{}'.format(err))
 
@@ -475,7 +472,7 @@ def test_cpu_only(self):
 
     @unittest.skipIf(IS_WINDOWS, "FIXME: Intermittent CUDA out-of-memory error")
     @unittest.skipIf(not torch.cuda.is_available(), 'No CUDA')
-    def test_cuda(self):
+    def test_bottleneck_cuda(self):
         rc, out, err = self._run_bottleneck('bottleneck/test_cuda.py')
         self.assertEqual(rc, 0, 'Run failed with\n{}'.format(err))
Repository: pytorch/pytorch

Issue:
Error when exiting tutorial script
OS: 
Ubuntu 18.04

PyTorch version:  
0.3.1.post2

Python version:
3.6.5

How you installed PyTorch:
conda install pytorch-cpu torchvision -c pytorch

Script to reproduce the bug:
the tutorial script for cifar10 classification: http://pytorch.org/tutorials/_downloads/cifar10_tutorial.py

At the end of the script execution, I have the following error message:
```
Exception ignored in: <bound method DataLoaderIter.__del__ of <torch.utils.data.dataloader.DataLoaderIter object at 0x7f0b579b8470>>
Traceback (most recent call last):
  File "/home/snakeone/anaconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 333, in __del__
  File "/home/snakeone/anaconda3/envs/torch/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 319, in _shutdown_workers
  File "/home/snakeone/anaconda3/envs/torch/lib/python3.6/multiprocessing/queues.py", line 337, in get
ImportError: sys.meta_path is None, Python is likely shutting down
```



Discussion:
I am aware of this and will have a fix.

Resolving patch:
diff --git a/torch/utils/data/dataloader.py b/torch/utils/data/dataloader.py
--- a/torch/utils/data/dataloader.py
+++ b/torch/utils/data/dataloader.py
@@ -320,17 +320,21 @@ def _shutdown_workers(self):
             if not self.shutdown:
                 self.shutdown = True
                 self.done_event.set()
-                # if worker_manager_thread is waiting to put, make place for it
+                for q in self.index_queues:
+                    q.put(None)
+                # if some workers are waiting to put, make place for them
                 try:
-                    while not self.data_queue.empty():
-                        self.data_queue.get()
-                except FileNotFoundError:
+                    while not self.worker_result_queue.empty():
+                        self.worker_result_queue.get()
+                except (FileNotFoundError, ImportError):
+                    # Many weird errors can happen here due to Python
+                    # shutting down. These are more like obscure Python bugs.
                     # FileNotFoundError can happen when we rebuild the fd
                     # fetched from the queue but the socket is already closed
-                    # from the worker side (e.g. due to Python shutting down).
+                    # from the worker side.
+                    # ImportError can happen when the unpickler loads the
+                    # resource from `get`.
                     pass
-                for q in self.index_queues:
-                    q.put(None)
                 # done_event should be sufficient to exit worker_manager_thread,
                 # but be safe here and put another None
                 self.worker_result_queue.put(None)


Test patch:
Repository: pytorch/pytorch

Issue:
Default initial hidden states for recurrent layers
The recurrent layers (nn.RNN, nn.LSTM, and nn.GRU) all expect initial hidden (+cell for LSTM) states as a required argument. Would it be possible for these to default to zero if not provided?


Discussion:
Sure, that makes sense.
@apaszke Can I work on this issue? Is there a process to do that?
Sure! As far as I know no one is working on that, so go ahead if you want.
Can we add this to the RNN documentation page as well? Took me a while to find this information here..
@amrsharaf a change adding that has just been merged. Hopefully, noone will need to come here to find that out.

Thanks for suggesting that!
great, thanks!
It seems that document for [LSTMCell](http://pytorch.org/docs/stable/nn.html#lstmcell) did not add default statement while document for [LSTM](http://pytorch.org/docs/stable/nn.html#lstm) does mention the default settings.

Resolving patch:
diff --git a/torch/nn/modules/rnn.py b/torch/nn/modules/rnn.py
--- a/torch/nn/modules/rnn.py
+++ b/torch/nn/modules/rnn.py
@@ -533,6 +533,7 @@ class RNNCell(RNNCellBase):
         - **input** of shape `(batch, input_size)`: tensor containing input features
         - **hidden** of shape `(batch, hidden_size)`: tensor containing the initial hidden
           state for each element in the batch.
+          Defaults to zero if not provided.
 
     Outputs: h'
         - **h'** of shape `(batch, hidden_size)`: tensor containing the next hidden state
@@ -625,6 +626,8 @@ class LSTMCell(RNNCellBase):
         - **c_0** of shape `(batch, hidden_size)`: tensor containing the initial cell state
           for each element in the batch.
 
+          If `(h_0, c_0)` is not provided, both **h_0** and **c_0** default to zero.
+
     Outputs: h_1, c_1
         - **h_1** of shape `(batch, hidden_size)`: tensor containing the next hidden state
           for each element in the batch
@@ -706,6 +709,7 @@ class GRUCell(RNNCellBase):
         - **input** of shape `(batch, input_size)`: tensor containing input features
         - **hidden** of shape `(batch, hidden_size)`: tensor containing the initial hidden
           state for each element in the batch.
+          Defaults to zero if not provided.
 
     Outputs: h'
         - **h'** of shape `(batch, hidden_size)`: tensor containing the next hidden state


Test patch:
Repository: pytorch/pytorch

Issue:
incorrect doc for torch.nn.functional.relu
In line 613-621 of `torch/nn/functional.py`, it states

```python
def relu(input, inplace=False):
    r"""relu(input, threshold, value, inplace=False) -> Tensor
    Applies the rectified linear unit function element-wise. See
    :class:`~torch.nn.ReLU` for more details.
    """
    if inplace:
        return torch.relu_(input)
    return torch.relu(input)
```

It looks like the doc is incorrect. I think it should something like
```python
def relu(input, inplace=False):
    r"""relu(input, inplace=False) -> Tensor
    Applies the rectified linear unit function element-wise. See
    :class:`~torch.nn.ReLU` for more details.
    """
    if inplace:
        return torch.relu_(input)
    return torch.relu(input)
```

which causes the docs on the website to be incorrect.



Discussion:

Resolving patch:
diff --git a/torch/nn/functional.py b/torch/nn/functional.py
--- a/torch/nn/functional.py
+++ b/torch/nn/functional.py
@@ -611,7 +611,7 @@ def threshold(input, threshold, value, inplace=False):
 
 
 def relu(input, inplace=False):
-    r"""relu(input, threshold, value, inplace=False) -> Tensor
+    r"""relu(input, inplace=False) -> Tensor
 
     Applies the rectified linear unit function element-wise. See
     :class:`~torch.nn.ReLU` for more details.


Test patch:
Repository: pytorch/pytorch

Issue:
Sampler not exposed to torch.utils.data
## Issue description

class Sampler is not exposed to torch.utils.data, which is not consistent with the error information.
https://github.com/pytorch/pytorch/blob/master/torch/utils/data/sampler.py#L123

## Code example


```
>>> import torch
>>> torch.utils.data.Sampler
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Sampler'
>>> torch.utils.data.sampler.Sampler
```


Discussion:

Resolving patch:
diff --git a/torch/utils/data/sampler.py b/torch/utils/data/sampler.py
--- a/torch/utils/data/sampler.py
+++ b/torch/utils/data/sampler.py
@@ -120,7 +120,7 @@ class BatchSampler(object):
     def __init__(self, sampler, batch_size, drop_last):
         if not isinstance(sampler, Sampler):
             raise ValueError("sampler should be an instance of "
-                             "torch.utils.data.Sampler, but got sampler={}"
+                             "torch.utils.data.sampler.Sampler, but got sampler={}"
                              .format(sampler))
         if not isinstance(batch_size, _int_classes) or isinstance(batch_size, bool) or \
                 batch_size <= 0:


Test patch:
Repository: pytorch/pytorch

Issue:
[PyTorch] tensor.tolist has no docstring



Discussion:

Resolving patch:
diff --git a/torch/_tensor_docs.py b/torch/_tensor_docs.py
--- a/torch/_tensor_docs.py
+++ b/torch/_tensor_docs.py
@@ -1028,7 +1028,7 @@ def add_docstr_all(method, docstr):
 item() -> number
 
 Returns the value of this tensor as a standard Python number. This only works
-for tensors with one element.
+for tensors with one element. For other cases, see :meth:`~Tensor.tolist`.
 
 This operation is not differentiable.
 
@@ -2107,6 +2107,26 @@ def callable(a, b) -> number
 In-place version of :meth:`~Tensor.tanh`
 """)
 
+add_docstr_all('tolist',
+               r""""
+tolist() -> list or number
+
+Returns the tensor as a (nested) list. For scalars, a standard
+Python number is returned, just like with :meth:`~Tensor.item`.
+Tensors are automatically moved to the CPU first if necessary.
+
+This operation is not differentiable.
+
+Examples::
+
+    >>> a = torch.randn(2, 2)
+    >>> a.tolist()
+    [[0.012766935862600803, 0.5415473580360413],
+     [-0.08909505605697632, 0.7729271650314331]]
+    >>> a[0,0].tolist()
+    0.012766935862600803
+""")
+
 add_docstr_all('topk',
                r"""
 topk(k, dim=None, largest=True, sorted=True) -> (Tensor, LongTensor)


Test patch:
Repository: pytorch/pytorch

Issue:
repr of tensor does't rescale values
Printing a float tensor with < 1 values factors out a common multiplicative power.
But in some cases, it forgets to rescale the values that are printed to take into account the factored value:
```python
torch.tensor(0.05524)
# this is right
# tensor(1.00000e-02 *
#        5.5240)

# but
torch.tensor([0.05524])
# prints
# tensor(1.00000e-02 *
#        [ 0.0552])
```

Also, I think we should be a bit more strict in the cases we want to print the common multiplicative factor (in those cases, we are more verbose and still lose information). Numpy doesn't factor out the aforementioned cases
```python
np.array(0.05224)
# prints array(0.05224)
np.array([0.05224])
# prints array([0.05224])
```

I'm using `torch.__version__ = '0.5.0a0+4caea64'`


Discussion:
CC @li-roy 

Resolving patch:
diff --git a/torch/_tensor_str.py b/torch/_tensor_str.py
--- a/torch/_tensor_str.py
+++ b/torch/_tensor_str.py
@@ -168,7 +168,7 @@ def _vector_str(self, indent, fmt, scale, sz, summarize):
                 [' ...'] +
                 [fmt(val / scale) for val in self[-PRINT_OPTS.edgeitems:].tolist()])
     else:
-        data = [fmt(val) for val in self.tolist()]
+        data = [fmt(val / scale) for val in self.tolist()]
 
     data_lines = [data[i:i + elements_per_line] for i in range(0, len(data), elements_per_line)]
     lines = [', '.join(line) for line in data_lines]


Test patch:
Repository: pytorch/pytorch

Issue:
PackedSequence is missing `to` method
The following will die as PackedSequence doesn't have `to` method
```(python)
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
torch.nn.utils.rnn.pad_sequence(sequences).to(device)
```
So instead one has to do old-fashioned `.cuda()` call.


Discussion:

Resolving patch:
diff --git a/torch/nn/utils/rnn.py b/torch/nn/utils/rnn.py
--- a/torch/nn/utils/rnn.py
+++ b/torch/nn/utils/rnn.py
@@ -85,6 +85,23 @@ def byte(self):
         r"""Returns copy with `self.data` cast to byte type"""
         return type(self)(self.data.byte(), self.batch_sizes)
 
+    def to(self, *args, **kwargs):
+        r"""Performs dtype and/or device conversion on `self.data`.
+
+        It has similar signature as :meth:`torch.Tensor.to`.
+
+        .. note::
+
+            If the ``self.data`` Tensor already has the correct :class:`torch.dtype`
+            and :class:`torch.device`, then ``self`` is returned.
+            Otherwise, returns a copy with the desired configuration.
+        """
+        data = self.data.to(*args, **kwargs)
+        if data is self.data:
+            return self
+        else:
+            return type(self)(data, self.batch_sizes)
+
     @property
     def is_cuda(self):
         r"""Returns true if `self.data` stored on a gpu"""


Test patch:
diff --git a/test/test_nn.py b/test/test_nn.py
--- a/test/test_nn.py
+++ b/test/test_nn.py
@@ -161,6 +161,24 @@ def err_fn():
                     ref_output = torch.cat([no_extra_pad, extra_pad], 0)
                 self.assertEqual(unpacked, ref_output)
 
+    def test_to(self):
+        padded, lengths = self._padded_sequence(torch.IntTensor)
+        a = rnn_utils.pack_padded_sequence(padded, lengths).cpu()
+
+        self.assertIs(a, a.to('cpu'))
+        self.assertIs(a, a.to('cpu', dtype=torch.int32))
+        self.assertEqual(a.long(), a.to(torch.int64))
+
+        if torch.cuda.is_available():
+            for cuda in ['cuda', 'cuda:0' if torch.cuda.device_count() == 1 else 'cuda:1']:
+                b = a.cuda(device=cuda)
+                self.assertIs(b, b.to(cuda))
+                self.assertEqual(a, b.to('cpu'))
+                self.assertEqual(b, a.to(cuda))
+                self.assertEqual(a, b.to('cpu', dtype=torch.int32))
+                self.assertIs(b, b.to(dtype=torch.int32))
+                self.assertEqual(b.long(), b.to(dtype=torch.int64))
+
 
 def default_tensor_type(type):
     type_str = torch.typename(type)
Repository: pytorch/pytorch

Issue:
return NotImplemented
This is minor and not urgent, but in batch normalization: https://github.com/pytorch/pytorch/blob/76d3c30783d3e808b070f8350d9f102bb2396944/torch/nn/modules/batchnorm.py#L48

It seems to be `raise NotImplementedError`, according to the [python doc](https://docs.python.org/3.7/library/constants.html#NotImplemented).


Discussion:
@ydawei one never uses the _BatchNorm base class. So we never ran into this :), we should correct it.

Resolving patch:
diff --git a/torch/nn/modules/batchnorm.py b/torch/nn/modules/batchnorm.py
--- a/torch/nn/modules/batchnorm.py
+++ b/torch/nn/modules/batchnorm.py
@@ -45,7 +45,7 @@ def reset_parameters(self):
             self.bias.data.zero_()
 
     def _check_input_dim(self, input):
-        return NotImplemented
+        raise NotImplementedError
 
     def forward(self, input):
         self._check_input_dim(input)
diff --git a/torch/nn/modules/instancenorm.py b/torch/nn/modules/instancenorm.py
--- a/torch/nn/modules/instancenorm.py
+++ b/torch/nn/modules/instancenorm.py
@@ -9,7 +9,7 @@ def __init__(self, num_features, eps=1e-5, momentum=0.1, affine=False,
             num_features, eps, momentum, affine, track_running_stats)
 
     def _check_input_dim(self, input):
-        return NotImplemented
+        raise NotImplementedError
 
     def _load_from_state_dict(self, state_dict, prefix, strict, missing_keys, unexpected_keys, error_msgs):
         try:


Test patch:
Repository: pytorch/pytorch

Issue:
Minor error in nn.MultiMarginLoss documentation
Hello,

In torch.nn.MultiMarginLoss in the Pytorch documentation : https://pytorch.org/docs/stable/nn.html?highlight=multimarginloss#torch.nn.MultiMarginLoss.

The loss in the case where there is weights is : 
\[\text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} - x[y] - x[i]))^p)}{\text{x.size}(0)}\]
and should be
\[\text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} - x[y] + x[i]))^p)}{\text{x.size}(0)}\]
The + has change to a - between the loss without weight and with weight , which is probably not what is coded.



Discussion:

Resolving patch:
diff --git a/torch/nn/modules/loss.py b/torch/nn/modules/loss.py
--- a/torch/nn/modules/loss.py
+++ b/torch/nn/modules/loss.py
@@ -912,7 +912,7 @@ class MultiMarginLoss(_WeightedLoss):
     The loss function then becomes:
 
     .. math::
-        \text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} - x[y] - x[i]))^p)}{\text{x.size}(0)}
+        \text{loss}(x, y) = \frac{\sum_i \max(0, w[y] * (\text{margin} - x[y] + x[i]))^p)}{\text{x.size}(0)}
 
     Args:
         p (int, optional): Has a default value of `1`. `1` and `2` are the only


Test patch:
Repository: pytorch/pytorch

Issue:
Pollution of GPU-0 memory
## Issue description
Copying a tensor to a GPU other than 0 uses up memory on GPU 0 too.

## Code example
```
import torch
a = torch.zeros(1).cuda(1)
```

nvidia-smi now shows:
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0     93271    C   python                                         306MiB |
|    1     93271    C   python                                         306MiB |
+-----------------------------------------------------------------------------+

## System Info
PyTorch version: 0.4.0
Is debug build: No
CUDA used to build PyTorch: 8.0.61

OS: Ubuntu 16.04.3 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 8.0.61
GPU models and configuration: 
GPU 0: Tesla K80
GPU 1: Tesla K80
GPU 2: Tesla K80
GPU 3: Tesla K80

Nvidia driver version: 367.48
cuDNN version: Probably one of the following:
/usr/local/cuda-8.0/lib64/libcudnn.so.5.1.10
/usr/local/cuda-8.0/lib64/libcudnn_static.a

Versions of relevant libraries:
[pip3] numpy (1.14.3)
[pip3] torch (0.4.0)
[pip3] torchvision (0.2.1)
[conda] Could not collect



Discussion:
I remember that we always create context on first observable GPU. Not sure if it is the correct behavior.
We really shouldn't, but that might be the case because we call some driver functions to do initialization, before we switch to the desired GPU. A fix might be complicated 😕 
This:
```
import torch
a = torch.zeros(1).cuda(1)
```
works nicely in 0.3.1 (memory is taken only from the GPU 1)
The problem is the `THCRandom_getGenerator` call :

https://github.com/pytorch/pytorch/blob/71626491c40c86eb1ee410e695c467486084144c/aten/src/ATen/CUDAGenerator.cpp#L17-L23

via the call chain:

```
#1  0x00007fff622869eb in at::CUDAGenerator::CUDAGenerator(at::Context*) () from /data/users/sgross/pytorch/torch/lib/libATen.so
#2  0x00007fff62286d98 in at::Context::doInitCUDA() () from /data/users/sgross/pytorch/torch/lib/libATen.so
```

That call looks incorrect: the `THCGenerator*` is per-device but we have one global default  `CUDAGenerator`. It's also not really used, since the actual THC bindings look like:

```
    auto generator_ = check_generator<CUDAGenerator>(generator, &context->defaultGenerator(backend()));
    (void) generator_; //silence unused warning
```

Resolving patch:
diff --git a/aten/src/ATen/gen.py b/aten/src/ATen/gen.py
--- a/aten/src/ATen/gen.py
+++ b/aten/src/ATen/gen.py
@@ -141,7 +141,7 @@ def check_all_files_written(self):
     },
     'CUDAGenerator.h': {
         'name': 'CUDA',
-        'th_generator': 'THCGenerator * generator;',
+        'th_generator': '',
         'header': 'THC/THC.h'
     },
 }


Test patch:
Repository: pytorch/pytorch

Issue:
[feature request] make DataParallel and DistributedDataParallel's module as public
After training the DataParallel or DistributedDataParallel's model, we may resume the trained model, however [resume the parallel model while not unpack it](https://github.com/pytorch/examples/blob/83f1b5c2667c820b672852d8a5b6971835e6406f/imagenet/main.py#L104) is not a good idea while DataParallel and DistributedDataParallel are just used to accelerate training via parallel. So we would like to unpack them first and save the raw model sometimes.

However, although the `module` is public in the source code, this is not explicit in the doc. If we only reference the explicit and public code, we have to unpack the model like `a = list(b.modules())[0]` while `b` is the paralleled model, it is ugly. So I advice that make DataParallel and DistributedDataParallel's module as public so that we can unpack it comfortably.


Discussion:
what do you mean by "make it public", I am not sure I understand.
Tell everyone in the doc that `module` is a public member of `DataParallel` or `DistributedDataParallel` that we can use this API .
You can safely access `.module`, we will never change that. If you'd like this clarified, please send a PR with a doc fix!
OK, please wait for a few days because I don’t have a computer at hand.

Resolving patch:
diff --git a/torch/nn/parallel/data_parallel.py b/torch/nn/parallel/data_parallel.py
--- a/torch/nn/parallel/data_parallel.py
+++ b/torch/nn/parallel/data_parallel.py
@@ -31,6 +31,9 @@ class DataParallel(Module):
         device_ids: CUDA devices (default: all devices)
         output_device: device location of output (default: device_ids[0])
 
+    Attributes:
+        module (Module): the module to be parallelized
+
     Example::
 
         >>> net = torch.nn.DataParallel(model, device_ids=[0, 1, 2])
diff --git a/torch/nn/parallel/distributed.py b/torch/nn/parallel/distributed.py
--- a/torch/nn/parallel/distributed.py
+++ b/torch/nn/parallel/distributed.py
@@ -72,6 +72,9 @@ class DistributedDataParallel(Module):
         device_ids: CUDA devices (default: all devices)
         output_device: device location of output (default: device_ids[0])
 
+    Attributes:
+        module (Module): the module to be parallelized
+
     Example::
 
         >>> torch.distributed.init_process_group(world_size=4, init_method='...')


Test patch: