internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/pytorch

Issue:
Pytorch docs：RNN weight_ih_l[k]  shape error
In pytorch docs，torch.nn.RNN parts show that variable **weight_ih_l[k]**  has shape of input_size x hidden_size，however it should be hidden_size x input_size.


Discussion:

Resolving patch:
diff --git a/torch/nn/modules/rnn.py b/torch/nn/modules/rnn.py
--- a/torch/nn/modules/rnn.py
+++ b/torch/nn/modules/rnn.py
@@ -296,7 +296,8 @@ class RNN(RNNBase):
 
     Attributes:
         weight_ih_l[k]: the learnable input-hidden weights of the k-th layer,
-            of shape `(input_size x hidden_size)`
+            of shape `(hidden_size x input_size)` for k=0. Otherwise, the shape is
+            `(hidden_size x hidden_size)`
         weight_hh_l[k]: the learnable hidden-hidden weights of the k-th layer,
             of shape `(hidden_size x hidden_size)`
         bias_ih_l[k]: the learnable input-hidden bias of the k-th layer,


Test patch:
Repository: pytorch/pytorch

Issue:
UnboundLocalError: local variable 'distutils' referenced before assignment
This error happens building from source in Linux. Introduced in https://github.com/pytorch/pytorch/pull/3993


Discussion:

Resolving patch:
diff --git a/tools/setup_helpers/ninja_builder.py b/tools/setup_helpers/ninja_builder.py
--- a/tools/setup_helpers/ninja_builder.py
+++ b/tools/setup_helpers/ninja_builder.py
@@ -73,6 +73,7 @@ def patch(obj, attr_name, val):
             orig_link = orig_compiler.link
             orig_spawn = orig_compiler.spawn
         else:
+            import distutils.unixccompiler
             orig_compiler = distutils.unixccompiler.UnixCCompiler
             orig_compile = orig_compiler._compile
             orig_link = orig_compiler.link


Test patch:
Repository: pytorch/pytorch

Issue:
StepLR - example
Hi,
In the https://github.com/pytorch/pytorch/blob/e9ef20eab5e5cf361bdc7a425c7f8b873baad9d3/torch/optim/lr_scheduler.py#L86-L89 is written: "lr values are [0.05, 0.005, ...]."
If I understand it correctly, it should be [0.5, 0.05, 0.005, ...]. 


Discussion:
That is correct. Please feel free to send a PR to fix it! (otherwise, I can)
If you don't mind to change it yourself, I'm not contributor yet and don't want to start just for one comment. I guess `MultiStepLR` is the same case. 
  

Resolving patch:
diff --git a/torch/optim/lr_scheduler.py b/torch/optim/lr_scheduler.py
--- a/torch/optim/lr_scheduler.py
+++ b/torch/optim/lr_scheduler.py
@@ -83,7 +83,7 @@ class StepLR(_LRScheduler):
         last_epoch (int): The index of last epoch. Default: -1.
 
     Example:
-        >>> # Assuming optimizer uses lr = 0.5 for all groups
+        >>> # Assuming optimizer uses lr = 0.05 for all groups
         >>> # lr = 0.05     if epoch < 30
         >>> # lr = 0.005    if 30 <= epoch < 60
         >>> # lr = 0.0005   if 60 <= epoch < 90
@@ -118,7 +118,7 @@ class MultiStepLR(_LRScheduler):
         last_epoch (int): The index of last epoch. Default: -1.
 
     Example:
-        >>> # Assuming optimizer uses lr = 0.5 for all groups
+        >>> # Assuming optimizer uses lr = 0.05 for all groups
         >>> # lr = 0.05     if epoch < 30
         >>> # lr = 0.005    if 30 <= epoch < 80
         >>> # lr = 0.0005   if epoch >= 80


Test patch:
Repository: pytorch/pytorch

Issue:
RuntimeError: Unsupported op descriptor: thnn_batch_norm_forward-5-eps-momentum-training. File a bug report.
I started getting this with the latest code while using JIT:

RuntimeError: Unsupported op descriptor: thnn_batch_norm_forward-5-eps-momentum-training. File a bug report.

It did not happen 7 days ago and still does not happen if I roll back pytorch to the version I had 7 days ago.  Do you need any simplified test case to fix it?
  


Discussion:
here is a simple code that reproduces it:


```python

import torch
from torch import jit
import torch.nn as nn
from torch.autograd import Variable
import torch.optim as optim        


@torch.jit.compile(nderivs=1)
class TestNet(nn.Module):
    def __init__(self):
        super(TestNet, self).__init__()
        self.net1 = nn.Linear(100, 200)
        self.net2 = nn.Linear(200, 1)
        self.bn = nn.BatchNorm1d(200)
        self.sigmoid = nn.Sigmoid()
        self.ReLU = nn.ReLU(inplace=False)
        self.drop = nn.Dropout(0.5)
               
    def forward(self, V):
        return self.sigmoid(self.net2(self.drop(self.ReLU(self.bn(self.net1(V)))))).squeeze() 


use_cuda = False
net = TestNet()
criterion = nn.BCELoss()
if use_cuda:
    net.cuda()
    criterion.cuda()
    V = Variable(torch.randn(100, 100)).cuda()
    label = Variable(torch.randn(100)).cuda()
else:
    V = Variable(torch.randn(100, 100))
    label = Variable(torch.randn(100))


optim_betas = (0.9, 0.999)
optimizer = optim.Adam(filter(lambda p: p.requires_grad, net.parameters()), lr=0.01, betas=optim_betas)
criterion = nn.BCELoss()

net.train()
for i in range(0,1000000):
    net.zero_grad()
    r = net(V)
    err = criterion(r, label)
    err.backward()   

```



  
Ah, that's probably because batchnorm got bound as an ATen method and I was apparently supposed implement some more JIT dispatch support but didn't.
I think we auto-generate everything from `derivatives.yaml`
In #4395, `thnn_batch_norm` was renamed to `thnn_batch_norm_forward` in `derivatives.yaml`, but all functions ends with `_forward` are not recognized as JIT funciton in `gen_jit_dispatch.py`

https://github.com/pytorch/pytorch/blob/a0ab48575e8cb6cb0edb5e9966c14bf4404384fb/tools/jit/gen_jit_dispatch.py#L49-L52
@colesbury, do you think you could take a look?
I can take a look later today. I wrote that script

Resolving patch:
diff --git a/tools/jit/gen_jit_dispatch.py b/tools/jit/gen_jit_dispatch.py
--- a/tools/jit/gen_jit_dispatch.py
+++ b/tools/jit/gen_jit_dispatch.py
@@ -49,7 +49,6 @@
 def is_jit_op(decl):
     return (not decl['api_name'].endswith('_') and
             not decl['name'].endswith('_out') and
-            not decl['name'].endswith('_forward') and
             not any(arg['simple_type'] == 'Generator' for arg in decl['arguments']) and
             not any(arg['simple_type'] == 'SparseTensor' for arg in decl['arguments']) and
             not any(arg['simple_type'] == 'Storage' for arg in decl['arguments']) and


Test patch:
Repository: pytorch/pytorch

Issue:
Spelling error in lr_scheduler.py
Trivial report, unfortunately I can't make a PR at the moment, but I can submit a report. 

line 329 in `torch/optim/lr_scheduler.py`

```python
        if threshold_mode not in {'rel', 'abs'}:
            raise ValueError('threshold mode ' + mode + ' is unknown!')
```

should replace `mode` with `threshold_mode`


Discussion:

Resolving patch:
diff --git a/torch/optim/lr_scheduler.py b/torch/optim/lr_scheduler.py
--- a/torch/optim/lr_scheduler.py
+++ b/torch/optim/lr_scheduler.py
@@ -326,7 +326,7 @@ def _init_is_better(self, mode, threshold, threshold_mode):
         if mode not in {'min', 'max'}:
             raise ValueError('mode ' + mode + ' is unknown!')
         if threshold_mode not in {'rel', 'abs'}:
-            raise ValueError('threshold mode ' + mode + ' is unknown!')
+            raise ValueError('threshold mode ' + threshold_mode + ' is unknown!')
         if mode == 'min' and threshold_mode == 'rel':
             rel_epsilon = 1. - threshold
             self.is_better = lambda a, best: a < best * rel_epsilon


Test patch:
Repository: pytorch/pytorch

Issue:
Bug in CosineAnnealingLR in Python 2 (?)
The learning rate only takes two values `base_lr` or `eta_min` if the `T_max` parameter passed in the constructor is an integer. This is probably because `self.last_epoch / self.T_max` evaluates to an integer in Python 2.


Discussion:
Ouch, can you please send a PR?

Resolving patch:
diff --git a/torch/optim/lr_scheduler.py b/torch/optim/lr_scheduler.py
--- a/torch/optim/lr_scheduler.py
+++ b/torch/optim/lr_scheduler.py
@@ -194,7 +194,7 @@ def __init__(self, optimizer, T_max, eta_min=0, last_epoch=-1):
 
     def get_lr(self):
         return [self.eta_min + (base_lr - self.eta_min) *
-                (1 + math.cos(self.last_epoch / self.T_max * math.pi)) / 2
+                (1 + math.cos(math.pi * self.last_epoch / self.T_max)) / 2
                 for base_lr in self.base_lrs]
 
 


Test patch:
diff --git a/test/test_optim.py b/test/test_optim.py
--- a/test/test_optim.py
+++ b/test/test_optim.py
@@ -503,7 +503,7 @@ def test_cos_anneal_lr(self):
         epochs = 10
         eta_min = 1e-10
         single_targets = [eta_min + (0.05 - eta_min) *
-                          (1 + math.cos(x / epochs * math.pi)) / 2
+                          (1 + math.cos(math.pi * x / epochs)) / 2
                           for x in range(epochs)]
         targets = [single_targets, list(map(lambda x: x * epochs, single_targets))]
         scheduler = CosineAnnealingLR(self.opt, T_max=epochs, eta_min=eta_min)
Repository: pytorch/pytorch

Issue:
Update documentation for Embedding layer
The documentation corresponding [torch.nn.Embedding](http://pytorch.org/docs/master/nn.html) mentions that ```Keep in mind that only a limited number of optimizers support sparse gradients: currently it’s optim.SGD (cuda and cpu), and optim.Adagrad (cpu)```. This is outdated and now `SparseAdam` is also supported.
Update documentation for Embedding layer
The documentation corresponding [torch.nn.Embedding](http://pytorch.org/docs/master/nn.html) mentions that ```Keep in mind that only a limited number of optimizers support sparse gradients: currently it’s optim.SGD (cuda and cpu), and optim.Adagrad (cpu)```. This is outdated and now `SparseAdam` is also supported.


Discussion:

Resolving patch:
diff --git a/torch/nn/modules/sparse.py b/torch/nn/modules/sparse.py
--- a/torch/nn/modules/sparse.py
+++ b/torch/nn/modules/sparse.py
@@ -34,7 +34,7 @@ class Embedding(Module):
     Notes:
         Keep in mind that only a limited number of optimizers support
         sparse gradients: currently it's `optim.SGD` (`cuda` and `cpu`),
-        and `optim.Adagrad` (`cpu`)
+        `optim.SparseAdam` (`cuda` and `cpu`) and `optim.Adagrad` (`cpu`)
 
     Examples::
 


Test patch:
Repository: pytorch/pytorch

Issue:
cuDNN half-precision batch normalization binding does not work
The VariableType binding code for cuDNN batch norm looks like:

![26695836_146541072795362_1826711580_o](https://user-images.githubusercontent.com/655866/34699474-835d6eea-f4ab-11e7-9374-f5cff4dfa2c1.png)

The problem is that inputs are of mixed types, but `unpack` only handles types that exactly match `input`. We should probably use `unpack_any` here.

As reported by Yuxin


Discussion:
So, how do we spell this in ATen `native_functions.yaml`? Similarly, can we stop calling `ByteTensor` `BoolTensor`?
Huh, it used to work for us before #4285, I guess we should finally add a test ;-)
Longer term we should stop checking precise types in VariableType.cpp, and rely on the underlying `Type` to check precise types (float vs. double). We're not doing this now because `unpack_any` is slower than `unpack`.

Short term, we should just do whatever to make this function work. Special-casing in `gen_variable_type.py` is fine with me.

Resolving patch:
diff --git a/tools/autograd/gen_variable_type.py b/tools/autograd/gen_variable_type.py
--- a/tools/autograd/gen_variable_type.py
+++ b/tools/autograd/gen_variable_type.py
@@ -71,7 +71,8 @@
 # concrete type of arguments. Eventually all VariableType functions should only
 # check that arguments are Variables.
 USE_UNPACK_ANY = {
-    'sparse_coo_tensor',
+    'sparse_coo_tensor', 'cudnn_batch_norm', 'cudnn_batch_norm_forward',
+    'cudnn_batch_norm_backward',
 }
 
 METHOD_DECLARATION = CodeTemplate("""\
@@ -442,11 +443,11 @@ def requires_unpack(arg):
         return 'Tensor' in arg['dynamic_type']
 
     def get_suffix(dynamic_type, is_nullable):
-        if is_nullable:
+        if use_unpack_any:
+            return '_any' if not is_nullable else '_any_opt'
+        elif is_nullable:
             assert dynamic_type == 'Tensor'
             return '_opt'
-        elif use_unpack_any:
-            return '_any'
         elif dynamic_type == 'IndexTensor':
             return '_long'
         elif dynamic_type == 'BoolTensor':


Test patch:
diff --git a/test/test_nn.py b/test/test_nn.py
--- a/test/test_nn.py
+++ b/test/test_nn.py
@@ -3325,6 +3325,15 @@ def func(root):
         gradcheck(func, [v])
         gradgradcheck(func, [v])
 
+    @unittest.skipIf(not TEST_CUDA, "CUDA unavailable")
+    @unittest.skipIf(not TEST_CUDNN, "cuDNN unavailable")
+    def test_batchnorm_cudnn_half(self):
+        input = Variable(torch.rand(2, 3, 2, 2).half().cuda())
+        m = nn.BatchNorm2d(3).float().cuda()
+        output = m(input)
+        output.sum().backward()
+        self.assertEqual(output.type(), input.type())
+
     def test_batchnorm_raises_error_if_running_mean_is_not_same_size_as_input(self):
         input = Variable(torch.rand(2, 10))
         running_var = torch.rand(10)
Repository: pytorch/pytorch

Issue:
unsupported Python type error from set_default_tensor_type()
Hi,
I've updated pytorch to the lastest master (`a54407a7`) from github, but got an error when I use `torch.set_default_tensor_type("torch.cuda.FloatTensor")`:
```
terminate called after throwing an instance of 'std::invalid_argument'
  what():  unsupported Python type
Aborted (core dumped)
```

What to do for workaround? Thanks!
Jin


Discussion:
Here's a workaround. I think the problem is that pytorch hasn't initialized CUDA yet. At any rate, the following works for me:
```
import torch
x = torch.randn(1) # initialize cuda
torch.set_default_tensor_type("torch.cuda.FloatTensor")
```

Resolving patch:
diff --git a/torch/__init__.py b/torch/__init__.py
--- a/torch/__init__.py
+++ b/torch/__init__.py
@@ -129,6 +129,11 @@ def set_default_tensor_type(t):
     global Storage
     Tensor = _import_dotted_name(t)
     Storage = _import_dotted_name(t.replace('Tensor', 'Storage'))
+
+    if 'cuda' in t:
+        import torch.cuda
+        torch.cuda.init()
+
     _C._set_default_tensor_type(Tensor)
 
 


Test patch:
Repository: pytorch/pytorch

Issue:
`F.mse_loss` has strange broadcasting semantics
```
import torch
import torch.nn.functional as F
from torch.autograd import Variable

output = Variable(torch.Tensor([1, 2]), requires_grad=True)
target = Variable(torch.Tensor([0, 1]))

train_loss1 = F.mse_loss(output.view(2, 1), target.view(2, 1, 1, 1), True)  # returns 1
train_loss2 = ((output.view(2, 1) - target.view(2, 1, 1, 1))**2).mean()  # returns 1.5
```

I expect one of the following two things:
1) If `mse_loss` doesn't support broadcasting then it should throw an error.
2) If `mse_loss` supports broadcasting, then it should return 1.5 to be consistent.


Discussion:

Resolving patch:
diff --git a/torch/nn/modules/loss.py b/torch/nn/modules/loss.py
--- a/torch/nn/modules/loss.py
+++ b/torch/nn/modules/loss.py
@@ -332,8 +332,6 @@ class MSELoss(_Loss):
             \operatorname{sum}(L),  & \text{if}\; \text{size_average} = \text{False}.
         \end{cases}
 
-    `x` and `y` arbitrary shapes with a total of `n` elements each.
-
     The sum operation still operates over all the elements, and divides by `n`.
 
     The division by `n` can be avoided if one sets the internal variable


Test patch:
diff --git a/test/test_nn.py b/test/test_nn.py
--- a/test/test_nn.py
+++ b/test/test_nn.py
@@ -2394,6 +2394,24 @@ def test_RNN_cell(self):
 
                 hx.sum().backward()
 
+    def _test_loss_equal_input_target_shape(self, cast):
+        # Tests losses whose inputs should have the same size.
+        losses = {
+            'mse_loss': lambda x, y: F.mse_loss(x, y),
+            'l1_loss': lambda x, y: F.l1_loss(x, y),
+            'smooth_l1_loss': lambda x, y: F.smooth_l1_loss(x, y),
+            'kl_div': lambda x, y: F.kl_div(x, y),
+            'poisson_nll_loss': lambda x, y: F.poisson_nll_loss(x, y),
+        }
+
+        input = Variable(cast(torch.randn(3, 5)))
+        target = Variable(cast(torch.randn(5, 3)))
+        for name, fn in losses.items():
+            self.assertRaises(Exception, lambda: fn(input, target))
+
+    def test_loss_equal_input_target_shape(self):
+        self._test_loss_equal_input_target_shape(lambda x: x)
+
     def test_RNN_cell_no_broadcasting(self):
         def test(cell_module, input, hx, input_size, hidden_size):
             cell = cell_module(input_size, hidden_size)
Repository: pytorch/pytorch

Issue:
torch.nn.DataParallel supporting unequal sizes
As documented [here](http://pytorch.org/docs/master/_modules/torch/nn/parallel/data_parallel.html):
```
The batch size should be larger than the number of GPUs used. It should
    also be an integer multiple of the number of GPUs so that each chunk is the
    same size (so that each GPU processes the same number of samples).
```
To use `torch.nn.DataParallel`, people should carefully set the batch size according to the number of gpus they plan to use, otherwise it will pop up errors. 

This issue becomes more subtle when using `torch.utils.data.DataLoader` with `drop_last=False` by default. As the total number of training/validation samples varies with the dataset, the size of the last batch of data loaded by `torch.utils.data.DataLoader` is easy to become indivisible by the number of GPUs (e.g., 2,3,4,8,...).

A feature request would be:
supporting `torch.nn.DataParallel` with batch size indivisible by the number of GPUs used.


Discussion:
Actually setting an indivisible `batch size` works in my master branch, I recommend you to try it in your version. The document should clarify the ability to run with indivisible `batch size` anyway. 
We may need to update docs if the code already supports that?

Resolving patch:
diff --git a/torch/nn/parallel/data_parallel.py b/torch/nn/parallel/data_parallel.py
--- a/torch/nn/parallel/data_parallel.py
+++ b/torch/nn/parallel/data_parallel.py
@@ -14,9 +14,7 @@ class DataParallel(Module):
     and each replica handles a portion of the input. During the backwards
     pass, gradients from each replica are summed into the original module.
 
-    The batch size should be larger than the number of GPUs used. It should
-    also be an integer multiple of the number of GPUs so that each chunk is the
-    same size (so that each GPU processes the same number of samples).
+    The batch size should be larger than the number of GPUs used.
 
     See also: :ref:`cuda-nn-dataparallel-instead`
 


Test patch:
Repository: pytorch/pytorch

Issue:
ONNX export failed on Constant because torch.onnx.symbolic.Constant does not exist 
Hi!

- OS: Linux
- PyTorch version: 0.4.0a0+ca5071d
- How you installed PyTorch (conda, pip, source): source
- Python version: 3.6

- Context around what you are trying to do
I am trying to convert layer with torch.index_select() operation to onnx

- A script to reproduce the bug. Please try to provide as minimal of a test case as possible.
```
import torch
from torch import nn
from torch.autograd import Variable

class  IndexLayer(nn.Module): 
    def __init__(self):
        super(IndexLayer, self).__init__()
        self.dim = 1
        self.index = [0,4,8,12,1,5,9,13,2,6,10,14,3,7,11,15]
    def forward(self, x):
        x = torch.index_select(x, dim=self.dim, index=Variable(torch.LongTensor(self.index)))
        return x

dummy_input = Variable(torch.randn(1,16,14,14), volatile=True)
model = IndexLayer()
torch.onnx.export(model, dummy_input, "./IndexLayer.pb", verbose=True)
```

- Error messages and/or stack traces of the bug
```
graph(%0 : Float(1, 16, 14, 14)) {
  %1 : Long(16) = Constant[value=<Tensor>](), scope: IndexLayer
  %2 : Float(1, 16, 14, 14) = Gather[axis=1](%0, %1), scope: IndexLayer
  return (%2);
}
UserWarning: ONNX export failed on Constant because torch.onnx.symbolic.Constant does not exist
  .format(op_name, op_name))
```

What is the right way to export torch.index_select() operation?


Discussion:
It kind of sounds like we forgot to implement constant export, which is embarrassing and should be fixed.
Are there any chances it will be fixed soon?
@Dnekhaev I took a look, and actually, it's just a spurious warning, the model seems to export successfully into `IndexLayer.pb`

Resolving patch:
diff --git a/torch/onnx/symbolic.py b/torch/onnx/symbolic.py
--- a/torch/onnx/symbolic.py
+++ b/torch/onnx/symbolic.py
@@ -120,6 +120,10 @@ def unused(g):
     return g.op("Undefined")
 
 
+def Constant(g, value):
+    return g.op("Constant", value_t=value)
+
+
 def add(g, self, other, alpha):
     if _scalar(alpha) != 1:
         return _unimplemented("add", "alpha != 1")


Test patch:
Repository: pytorch/pytorch

Issue:
doc-push is broken
```
+ make html
python source/scripts/build_activation_images.py
/var/lib/jenkins/workspace/pytorch/docs/source/_static/img/activation/
ELU(alpha=1.0)
Traceback (most recent call last):
  File "source/scripts/build_activation_images.py", line 69, in <module>
    pylab.clf()
  File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 677, in clf
    gcf().clf()
  File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 592, in gcf
    return figure()
  File "/opt/conda/lib/python3.6/site-packages/matplotlib/pyplot.py", line 539, in figure
    **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 171, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/opt/conda/lib/python3.6/site-packages/matplotlib/backends/backend_tkagg.py", line 1049, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/opt/conda/lib/python3.6/tkinter/__init__.py", line 2017, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
make: *** [figures] Error 1
Makefile:16: recipe for target 'figures' failed
```

CC @apaszke who was asking me about doc generation yesterday. Full log https://ci.pytorch.org/jenkins/job/pytorch-builds/job/doc-push/3092//console

And CC @pmitros 


Discussion:
You probably know why this is happening, but just in case: 
`matplotlib.use('Agg')` fixes this. (as per: https://stackoverflow.com/questions/37604289/tkinter-tclerror-no-display-name-and-no-display-environment-variable)

Matplotlib is trying to open a display window on what I assume is a screen-less server.

Resolving patch:
diff --git a/docs/source/scripts/build_activation_images.py b/docs/source/scripts/build_activation_images.py
--- a/docs/source/scripts/build_activation_images.py
+++ b/docs/source/scripts/build_activation_images.py
@@ -7,6 +7,10 @@
 import os.path
 import torch.nn.modules.activation
 import torch.autograd
+import matplotlib
+
+matplotlib.use('Agg')
+
 import pylab
 
 


Test patch: