internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/pytorch

Issue:
ProgressMonitor (Trainer plugin) prints incorrect progress value and percentage
Code to replicate behaviour, extract of [jupyter notebook](https://github.com/recastrodiaz/courses/blob/master/deeplearning1/nbs/lesson1-pytorch.ipynb) In [9]:

```PYTHON
t = trainer.Trainer(model, criterion, optimizer, train_loader)
t.register_plugin(trainer.plugins.ProgressMonitor())
t.register_plugin(trainer.plugins.AccuracyMonitor())
t.register_plugin(trainer.plugins.LossMonitor())
t.register_plugin(trainer.plugins.TimeMonitor())
t.register_plugin(trainer.plugins.Logger(['progress', 'accuracy', 'loss', 'time']))

epochs = 1
t.run(epochs)
```

Prints:
```LOG
progress: 23000/360 (6388.89%)	accuracy: 100.00%  (99.31%)	loss: 0.0058  (0.0368)	time: 705ms    (1249ms)
```

Should print instead:
```
progress: 360/360 (100.00%)	accuracy: 100.00%  (99.31%)	loss: 0.0058  (0.0368)	time: 705ms    (1249ms)
```

I'll send over a PR for this.


Discussion:

Resolving patch:
diff --git a/torch/utils/trainer/plugins/progress.py b/torch/utils/trainer/plugins/progress.py
--- a/torch/utils/trainer/plugins/progress.py
+++ b/torch/utils/trainer/plugins/progress.py
@@ -19,7 +19,7 @@ def register(self, trainer):
 
     def iteration(self, iteration, input, *args):
         stats = self.trainer.stats.setdefault(self.stat_name, {})
-        stats['samples_used'] += input.size(0)
+        stats['samples_used'] += 1
         stats['percent'] = 100. * stats['samples_used'] / stats['epoch_size']
 
     def epoch(self, *args):


Test patch:
Repository: pytorch/pytorch

Issue:
support ~ operator on ByteTensor
Currently not supported.
See: https://discuss.pytorch.org/t/how-do-use-logical-not-on-bytetensor/2074


Discussion:

Resolving patch:
diff --git a/torch/tensor.py b/torch/tensor.py
--- a/torch/tensor.py
+++ b/torch/tensor.py
@@ -381,6 +381,11 @@ def __ixor__(self, other):
             raise RuntimeError('logical operations are supported on ByteTensors only')
         return self.copy_((self + other).eq(1))
 
+    def __invert__(self):
+        if type(self).__name__ != 'ByteTensor':
+            raise RuntimeError('logical operations are supported on ByteTensors only')
+        return (1 - self)
+
     def __hash__(self):
         return id(self)
 


Test patch:
diff --git a/test/test_torch.py b/test/test_torch.py
--- a/test/test_torch.py
+++ b/test/test_torch.py
@@ -3129,6 +3129,10 @@ def test_logical_ops(self):
             else:
                 self.assertFalse(x[idx] ^ y[idx])
 
+        invert_result = ~x
+        for idx in iter_indices(x):
+            self.assertEqual(1 - x[idx], invert_result[idx])
+
         x_clone = x.clone()
         x_clone &= y
         self.assertEqual(x_clone, and_result)
Repository: pytorch/pytorch

Issue:
Dropout documentation is incorrect
"Randomly zeroes some of the elements of the input tensor. The elements to zero are randomized on every forward call."

This is incorrect; the function also scales up by 1/(1-p), which the implementation correctly does.


Discussion:

Resolving patch:
diff --git a/torch/nn/modules/dropout.py b/torch/nn/modules/dropout.py
--- a/torch/nn/modules/dropout.py
+++ b/torch/nn/modules/dropout.py
@@ -3,9 +3,19 @@
 
 
 class Dropout(Module):
-    r"""Randomly zeroes some of the elements of the input tensor.
+    r"""During training, randomly zeroes some of the elements of the input
+    tensor with probability *p* using samples from a bernoulli distribution.
     The elements to zero are randomized on every forward call.
 
+    This has proven to be an effective technique for regularization and
+    preventing the co-adaptation of neurons as described in the paper
+    `Improving neural networks by preventing co-adaptation of feature
+    detectors`_ .
+
+    Furthermore, the outputs are scaled by a factor of *1/(1-p)* during
+    training. This means that during evaluation the module simply computes an
+    identity function.
+
     Args:
         p: probability of an element to be zeroed. Default: 0.5
         inplace: If set to True, will do this operation in-place. Default: false
@@ -19,6 +29,8 @@ class Dropout(Module):
         >>> m = nn.Dropout(p=0.2)
         >>> input = autograd.Variable(torch.randn(20, 16))
         >>> output = m(input)
+
+    .. _Improving neural networks by preventing co-adaptation of feature detectors: https://arxiv.org/abs/1207.0580
     """
 
     def __init__(self, p=0.5, inplace=False):


Test patch:
Repository: pytorch/pytorch

Issue:
Unexpected logical AND behavior
I was using self built PyTorch 0.1.12+ccd5f4d on python 3.6.0 on MacOS 10.12.3. Here is what I got, which has a different result from equivalent NumPy expressions:

```python
x = torch.arange(0, 4, 1).view(2, 2).type(torch.ByteTensor)
y = torch.zeros(2, 2).type(torch.ByteTensor)
x & y  # returns [[0, 1], [0, 0]]
```


Discussion:

Resolving patch:
diff --git a/torch/tensor.py b/torch/tensor.py
--- a/torch/tensor.py
+++ b/torch/tensor.py
@@ -345,42 +345,6 @@ def __ge__(self, other):
         return self.ge(other)
 
     # TODO: add native add or and xor in the libs
-    def __and__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return (self + other).eq(2)
-
-    def __or__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return (self + other).gt(0)
-
-    def __xor__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return (self + other).eq(1)
-
-    def __iand__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return self.mul_(other)
-
-    def __ior__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return self.copy_((self + other).gt(0))
-
-    def __ixor__(self, other):
-        if (type(self).__name__ != 'ByteTensor' or
-                type(other).__name__ != 'ByteTensor'):
-            raise RuntimeError('logical operations are supported on ByteTensors only')
-        return self.copy_((self + other).eq(1))
-
     def __invert__(self):
         if type(self).__name__ != 'ByteTensor':
             raise RuntimeError('logical operations are supported on ByteTensors only')


Test patch:
Repository: pytorch/pytorch

Issue:
Check compile-time cuDNN version matches runtime
We should check that the version from cudnn.h matches the version from the loaded shared library.

Really weird things happen if they don't match.


Discussion:

Resolving patch:
diff --git a/tools/cwrap/plugins/CuDNNPlugin.py b/tools/cwrap/plugins/CuDNNPlugin.py
--- a/tools/cwrap/plugins/CuDNNPlugin.py
+++ b/tools/cwrap/plugins/CuDNNPlugin.py
@@ -135,7 +135,7 @@ def process_declarations(self, declarations):
                     if arg['name'] in ['self', 'state', 'dataType', 'handle']:
                         arg['ignore_check'] = True
             declaration['options'] = self.filter_unique_options(declaration['options'])
-        return declarations
+        return [d for d in declarations if not d.get('only_register', False)]
 
     def filter_unique_options(self, options):
         def signature(option):
diff --git a/torch/backends/cudnn/__init__.py b/torch/backends/cudnn/__init__.py
--- a/torch/backends/cudnn/__init__.py
+++ b/torch/backends/cudnn/__init__.py
@@ -17,6 +17,12 @@ def _libcudnn():
         if hasattr(lib, 'cudnnGetErrorString'):
             lib.cudnnGetErrorString.restype = ctypes.c_char_p
             __cudnn_version = lib.cudnnGetVersion()
+            compile_version = torch._C._cudnn_version()
+            # Check that cuDNN major and minor versions match
+            if (__cudnn_version // 100) != (compile_version // 100):
+                raise RuntimeError(
+                    'cuDNN version mismatch: PyTorch was compiled against {} '
+                    'but linked against {}'.format(compile_version, __cudnn_version))
         else:
             lib = None
     return lib


Test patch:
Repository: pytorch/pytorch

Issue:
torch.is_tensor(torch.HalfTensor()) returns False. 
The problem is [here](https://github.com/pytorch/pytorch/blob/master/torch/__init__.py#L274).


Discussion:
This is not intentional, is it? 
I don't think so?  It's inconsistent with torch7:

```
th> a=torch.HalfTensor()
th> torch.isTensor(a)
true
```

HalfStorage/isStorage looks to have the same issue.
@gchanan Looks like an easy-fix. 

Resolving patch:
diff --git a/torch/__init__.py b/torch/__init__.py
--- a/torch/__init__.py
+++ b/torch/__init__.py
@@ -268,12 +268,12 @@ def storage_type(cls):
 
 _storage_classes = {
     DoubleStorage, FloatStorage, LongStorage, IntStorage, ShortStorage,
-    CharStorage, ByteStorage,
+    CharStorage, ByteStorage, HalfStorage
 }
 
 _tensor_classes = {
     DoubleTensor, FloatTensor, LongTensor, IntTensor, ShortTensor,
-    CharTensor, ByteTensor,
+    CharTensor, ByteTensor, HalfTensor
 }
 
 


Test patch:
diff --git a/test/test_torch.py b/test/test_torch.py
--- a/test/test_torch.py
+++ b/test/test_torch.py
@@ -3590,12 +3590,12 @@ def test_half_tensor(self):
             torch.save(xh, f)
             f.seek(0)
             xh2 = torch.load(f)
-            self.assertEqual(xh, xh2)
+            self.assertEqual(xh.float(), xh2.float())
 
     @unittest.skipIf(not torch.cuda.is_available(), 'no CUDA')
     def test_half_tensor_cuda(self):
         x = torch.randn(5, 5).half()
-        self.assertEqual(x.cuda().cpu(), x)
+        self.assertEqual(x.cuda(), x)
 
         xc = x.cuda()
         with tempfile.NamedTemporaryFile() as f:
@@ -3603,7 +3603,7 @@ def test_half_tensor_cuda(self):
             f.seek(0)
             xc2 = torch.load(f)
             self.assertIsInstance(xc2, type(xc))
-            self.assertEqual(xc, xc2)
+            self.assertEqual(xc.float(), xc2.float())
 
     @unittest.skipIf(not torch.cuda.is_available(), 'no CUDA')
     def test_serialization_cuda(self):
@@ -3748,6 +3748,8 @@ def test_from_file(self):
 
     def test_print(self):
         for t in torch._tensor_classes:
+            if t == torch.HalfTensor:
+                continue  # HalfTensor does not support fill
             if t in torch.sparse._sparse_tensor_classes:
                 continue
             if t.is_cuda and not torch.cuda.is_available():
Repository: pytorch/pytorch

Issue:
Small mistake in nn.Threshold documentation
Hello,

In the [documentation](http://pytorch.org/docs/master/nn.html?highlight=threshold#torch.nn.Threshold) it says 
```
y =  x      if x >= threshold
     value  if x <  threshold
```
So the following: 
`torch.nn.Threshold(1,0)(torch.Tensor([1]))`
should evaluate to 1, but instead returns 0.

Maybe it should be corrected to:
```
y =  x      if x > threshold
     value  if x <=  threshold
```



Discussion:

Resolving patch:
diff --git a/torch/nn/modules/activation.py b/torch/nn/modules/activation.py
--- a/torch/nn/modules/activation.py
+++ b/torch/nn/modules/activation.py
@@ -10,8 +10,8 @@ class Threshold(Module):
 
     Threshold is defined as::
 
-         y =  x        if x >= threshold
-              value    if x <  threshold
+         y =  x        if x >  threshold
+              value    if x <= threshold
 
     Args:
         threshold: The value to threshold at


Test patch:
Repository: pytorch/pytorch

Issue:
inplace division doesn't work in version 0.1.12_2
This is an ipython session. Note that the `id` doesn't remain the same for /= even though it works for div_

```python
In [1]: import torch

In [2]: foo = torch.ones(3, 3)

In [3]: foo
Out[3]: 

 1  1  1
 1  1  1
 1  1  1
[torch.FloatTensor of size 3x3]

In [4]: id(foo)
Out[4]: 140493992350728

In [5]: foo /= 2

In [6]: foo
Out[6]: 

 0.5000  0.5000  0.5000
 0.5000  0.5000  0.5000
 0.5000  0.5000  0.5000
[torch.FloatTensor of size 3x3]

In [7]: id(foo)
Out[7]: 140493991465672

In [8]: foo = torch.ones(3, 3)

In [9]: id(foo)
Out[9]: 140492895866120

In [10]: foo.div_(2)
Out[10]: 

 0.5000  0.5000  0.5000
 0.5000  0.5000  0.5000
 0.5000  0.5000  0.5000
[torch.FloatTensor of size 3x3]

In [11]: id(foo)
Out[11]: 140492895866120
```


Discussion:
Just to be clear, the reason this is an issue is that it means that functions can't do inplace operations on function arguments.
Also, this works fine for +=

Resolving patch:
diff --git a/torch/tensor.py b/torch/tensor.py
--- a/torch/tensor.py
+++ b/torch/tensor.py
@@ -333,6 +333,7 @@ def __rdiv__(self, other):
 
     def __idiv__(self, other):
         return self.div_(other)
+    __itruediv__ = __idiv__
 
     def __mod__(self, other):
         return self.remainder(other)


Test patch:
diff --git a/test/test_torch.py b/test/test_torch.py
--- a/test/test_torch.py
+++ b/test/test_torch.py
@@ -4054,6 +4054,13 @@ def test_big_transpose(self):
         t2 = torch.from_numpy(t.numpy().transpose())
         self.assertEqual(t1, t2)
 
+    def test_inplace_division(self):
+        t = torch.rand(5, 5)
+        id_before = id(t)
+        t /= 2
+        id_after = id(t)
+        self.assertEqual(id_before, id_after)
+
 # Functions to test negative dimension wrapping
 METHOD = 1
 INPLACE_METHOD = 2
Repository: pytorch/pytorch

Issue:
Why there is a `device_id` parameter in `nn.module.cpu()` call?
`pytorch/torch/nn/modules/modules.py`
```python
def cpu(self, device_id=None):
    """Moves all model parameters and buffers to the CPU."""
    return self._apply(lambda t: t.cpu())
```

as shown in source code, this parameters is not used. Any reason that we keep it?


Discussion:
this can be cleaned up.

Resolving patch:
diff --git a/torch/nn/modules/module.py b/torch/nn/modules/module.py
--- a/torch/nn/modules/module.py
+++ b/torch/nn/modules/module.py
@@ -147,7 +147,7 @@ def cuda(self, device_id=None):
         """
         return self._apply(lambda t: t.cuda(device_id))
 
-    def cpu(self, device_id=None):
+    def cpu(self):
         """Moves all model parameters and buffers to the CPU."""
         return self._apply(lambda t: t.cpu())
 


Test patch:
Repository: pytorch/pytorch

Issue:
Negative dimension behaviour in torch.stack
Support for using a negative dimension in `torch.stack` currently works as follows:

```python
if dim < 0:
    dim += sequence[0].dim()
```

This is not consistent with the behaviour of `torch.unsqueeze`, which uses `dim += input.dim() + 1`. The unsqueeze behaviour is better since it is possible to use negative indexing to insert a new dimension as the last dimension.

## Example

`torch.stack` with `dim=-1` adds a new dimension in the second-to-last position. This is confusing. It is not possible to use negative indexing to add a dimension in the last position (ie can't achieve a 3x2 result).

```python
>>> a = torch.Tensor([1, 2, 3])
>>> b = torch.Tensor([4, 5, 6])
>>> torch.stack([a, b], -1)

 1  2  3
 4  5  6
[torch.FloatTensor of size 2x3]
```

Contrast this to using `torch.unsqueeze` and `torch.cat` with `dim=-1`, which adds a new dimension in the last position:

```python
>>> a = torch.Tensor([1, 2, 3])
>>> b = torch.Tensor([4, 5, 6])
>>> torch.cat([a.unsqueeze(-1), b.unsqueeze(-1)], -1)

 1  4
 2  5
 3  6
[torch.FloatTensor of size 3x2]
```


Discussion:
this looks like a bug / unintentional. 

Resolving patch:
diff --git a/torch/functional.py b/torch/functional.py
--- a/torch/functional.py
+++ b/torch/functional.py
@@ -58,7 +58,7 @@ def stack(sequence, dim=0, out=None):
     if len(sequence) == 0:
         raise ValueError("stack expects a non-empty sequence of tensors")
     if dim < 0:
-        dim += sequence[0].dim()
+        dim += sequence[0].dim() + 1
     inputs = [t.unsqueeze(dim) for t in sequence]
     if out is None:
         return torch.cat(inputs, dim)


Test patch:
diff --git a/test/test_torch.py b/test/test_torch.py
--- a/test/test_torch.py
+++ b/test/test_torch.py
@@ -1644,7 +1644,9 @@ def test_stack(self):
         z = torch.rand(2, 3, 4)
         for dim in range(4):
             res = torch.stack((x, y, z), dim)
+            res_neg = torch.stack((x, y, z), dim - 4)
             expected_size = x.size()[:dim] + (3,) + x.size()[dim:]
+            self.assertEqual(res, res_neg)
             self.assertEqual(res.size(), expected_size)
             self.assertEqual(res.select(dim, 0), x, 0)
             self.assertEqual(res.select(dim, 1), y, 0)
@@ -4130,7 +4132,6 @@ def idx_tensor(size, max_val):
     ('index_select', (10, 10), lambda: [DIM_ARG, idx_tensor((10,), 10)], [METHOD, FUNCTIONAL]),
     ('split', (10, 20), lambda: [5, DIM_ARG], [METHOD, FUNCTIONAL]),
     ('squeeze', (10, 1, 20, 1), lambda: [DIM_ARG], [METHOD, INPLACE_METHOD, FUNCTIONAL]),
-    ('stack', [(2, 3, 4), (2, 3, 4)], lambda: [DIM_ARG], [FUNCTIONAL]),
     ('unbind', (2, 3, 4), lambda: [DIM_ARG], [FUNCTIONAL]),
     ('unsqueeze', (10, 20), lambda: [DIM_ARG], [METHOD, INPLACE_METHOD, FUNCTIONAL], 1),
     ('cumprod', (10, 20), lambda: [DIM_ARG], [METHOD, FUNCTIONAL]),
Repository: pytorch/pytorch

Issue:
DataParallel tests are currently broken 
https://github.com/pytorch/pytorch/pull/2121/commits/d69669efcfe4333c223f53249185c2e22f76ed73 has broken DataParallel tests. Now that device_ids are explicitly sent to parallel_apply, this assert https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/parallel_apply.py#L30 gets triggered if inputs are not big enough to be on all devices (e.g. batch size of 20 on 8 GPUs gets chunked into 6*3+2, so that 8-th GPU is idle, and assert gets triggered). 



Discussion:

@apaszke your latest fix in https://github.com/pytorch/pytorch/pull/2193 doesn't fix this completely.

Here's a failing stack-trace:
```
FAIL: test_data_parallel_nested_output (__main__.TestNN)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_nn.py", line 1299, in test_data_parallel_nested_output
    output = dp.data_parallel(Net(), i, gpus)
  File "/data/users/soumith/pytorch/torch/nn/parallel/data_parallel.py", line 104, in data_parallel
    outputs = parallel_apply(replicas, inputs, module_kwargs, device_ids)
  File "/data/users/soumith/pytorch/torch/nn/parallel/parallel_apply.py", line 30, in parallel_apply
    assert len(modules) == len(devices)
AssertionError
```
How is it possible that the tests didn't catch that in the PR?
it's because our build box which only has 2 GPUs doesn't hit this case. It's failing on 4-GPU setting (maybe batch size becomes too small then)

Resolving patch:
diff --git a/torch/nn/parallel/data_parallel.py b/torch/nn/parallel/data_parallel.py
--- a/torch/nn/parallel/data_parallel.py
+++ b/torch/nn/parallel/data_parallel.py
@@ -100,6 +100,7 @@ def data_parallel(module, inputs, device_ids=None, output_device=None, dim=0, mo
     inputs, module_kwargs = scatter_kwargs(inputs, module_kwargs, device_ids, dim)
     if len(device_ids) == 1:
         return module(*inputs[0], **module_kwargs[0])
-    replicas = replicate(module, device_ids[:len(inputs)])
-    outputs = parallel_apply(replicas, inputs, module_kwargs, device_ids)
+    used_device_ids = device_ids[:len(inputs)]
+    replicas = replicate(module, used_device_ids)
+    outputs = parallel_apply(replicas, inputs, module_kwargs, used_device_ids)
     return gather(outputs, output_device, dim)


Test patch:
Repository: pytorch/pytorch

Issue:
Why there is a `device_id` parameter in `nn.module.cpu()` call?
`pytorch/torch/nn/modules/modules.py`
```python
def cpu(self, device_id=None):
    """Moves all model parameters and buffers to the CPU."""
    return self._apply(lambda t: t.cpu())
```

as shown in source code, this parameters is not used. Any reason that we keep it?


Discussion:
this can be cleaned up.

Resolving patch:
diff --git a/torch/nn/modules/module.py b/torch/nn/modules/module.py
--- a/torch/nn/modules/module.py
+++ b/torch/nn/modules/module.py
@@ -147,7 +147,7 @@ def cuda(self, device_id=None):
         """
         return self._apply(lambda t: t.cuda(device_id))
 
-    def cpu(self, device_id=None):
+    def cpu(self):
         """Moves all model parameters and buffers to the CPU."""
         return self._apply(lambda t: t.cpu())
 
diff --git a/torch/optim/lr_scheduler.py b/torch/optim/lr_scheduler.py
--- a/torch/optim/lr_scheduler.py
+++ b/torch/optim/lr_scheduler.py
@@ -78,7 +78,7 @@ class StepLR(_LRScheduler):
         optimizer (Optimizer): Wrapped optimizer.
         step_size (int): Period of learning rate decay.
         gamma (float): Multiplicative factor of learning rate decay.
-            Default: -0.1.
+            Default: 0.1.
         last_epoch (int): The index of last epoch. Default: -1.
 
     Example:
@@ -113,7 +113,7 @@ class MultiStepLR(_LRScheduler):
         optimizer (Optimizer): Wrapped optimizer.
         milestones (list): List of epoch indices. Must be increasing.
         gamma (float): Multiplicative factor of learning rate decay.
-            Default: -0.1.
+            Default: 0.1.
         last_epoch (int): The index of last epoch. Default: -1.
 
     Example:


Test patch:
Repository: pytorch/pytorch

Issue:
nn.Embedding improperly sorts inputs when max_norm is specified
```python
import torch
import torch.nn as nn
from torch.autograd import Variable

ids = Variable(torch.LongTensor([2, 12, 4, 8, 8, 6]))
embed = nn.Embedding(22, 5, max_norm=1.0)
print(ids)  # before: 2 12 4 8 8 6
out = embed(ids)
print(ids)  # after: 2 4 6 8 8 12
```

The output is also incorrect. It's the output from the sorted indices, instead of the user specified indices.

Reported by @adampolyak 


Discussion:

Resolving patch:
diff --git a/torch/nn/_functions/thnn/sparse.py b/torch/nn/_functions/thnn/sparse.py
--- a/torch/nn/_functions/thnn/sparse.py
+++ b/torch/nn/_functions/thnn/sparse.py
@@ -10,12 +10,10 @@ class Embedding(Function):
 
     @staticmethod
     def _renorm(ctx, indices, weight, max_norm, norm_type):
-        if indices.dim() == 2:
-            indices = indices.clone().view(-1)
-
+        # clone indices since LookupTable_renorm modifies it in-place
         ctx._backend.LookupTable_renorm(
             ctx._backend.library_state,
-            indices,
+            indices.clone().view(-1),
             weight,
             max_norm,
             norm_type


Test patch:
diff --git a/test/test_nn.py b/test/test_nn.py
--- a/test/test_nn.py
+++ b/test/test_nn.py
@@ -822,6 +822,21 @@ def test_embedding_padding_idx(self):
         self.assertEqual(output[0][0].sum().data[0], 0)
         self.assertEqual(output[1][2].sum().data[0], 0)
 
+    def test_embedding_max_norm(self):
+        embedding = nn.Embedding(22, 5, max_norm=1.0)
+        input = Variable(torch.LongTensor([2, 8, 8, 6]))
+        output = embedding(input)
+        self.assertEqual(output[1], output[2])
+        self.assertTrue(output.data.norm(p=2, dim=1).le(1).all())
+
+    @unittest.skipIf(not TEST_CUDA, "CUDA unavailable")
+    def test_embedding_max_norm_cuda(self):
+        embedding = nn.Embedding(22, 5, max_norm=1.0).cuda()
+        input = Variable(torch.LongTensor([2, 8, 8, 6])).cuda()
+        output = embedding(input)
+        self.assertEqual(output[1], output[2])
+        self.assertTrue(output.data.norm(p=2, dim=1).le(1).all())
+
     def test_embedding_functional(self):
         a = Variable(torch.LongTensor([
             [1, 3, 2],