Change:
I have a straightforward use case. Given a token (or word), lookup the embedding of the word, pass that through a linear layer and perform L2 regression. If the token corresponds to the of the embedding layer, throws an error saying OS: macOS High Sierra PyTorch version: 0.3.1 How you installed PyTorch (conda, pip, source): Tried both conda and pip Python version: 3.6.4 CUDA/cuDNN version: CPU GPU models and configuration: NA GCC version (if compiling from source): NA A script to reproduce the bug:
I got the same trouble, and it throw this error after some epochs of training. Does anyone have some solutions?

Discussion:
grad = grad.index(c); } <del> int64_t num_features = grad.size(-1); </del> <ins> int64_t num_features = grad_.size(-1); </ins> auto weight_size = std::array<int64_t, 2>{{ num_weights, num_features }}; <ins> auto& dense_type = grad.type(); auto& sparse_type = dense_type.toBackend(grad.is_cuda() ? kSparseCUDA : kSparseCPU); // check if all our grad come from padding_idx if (grad.numel() == 0) { return sparse_type.sparse_coo_tensor(indices_.type().tensor(), dense_type.tensor(), weight_size); } </ins> auto index = indices.view({1, -1}); auto values = grad.contiguous().view({-1, num_features}); <del> auto& sparse_type = grad.type().toBackend(grad.is_cuda() ? kSparseCUDA : kSparseCPU); </del> return sparse_type.sparse_coo_tensor(index, values, weight_size); }

self.assertRaises(AssertionError, nn.Embedding, num_embeddings=10, embedding_dim=20, padding_idx=25) self.assertRaises(AssertionError, nn.Embedding, num_embeddings=10, embedding_dim=20, padding_idx=-25) <ins> # test backward when input contains padding_idx padding_idx = 0 embedding = nn.Embedding(5, 2, padding_idx=padding_idx) for n in (1, 2): for other_indices in ([], [1, 3], [2]): indices = torch.LongTensor(other_indices + [padding_idx] * n) pre = embedding.weight[padding_idx].clone() embedding(indices).sum().backward() after = (embedding.weight + embedding.weight.grad)[padding_idx] embedding.zero_grad() self.assertEqual(after, pre) </ins> def test_embedding_max_norm(self): embedding = nn.Embedding(22, 5, max_norm=1.0) input = Variable(torch.LongTensor([2, 8, 8, 6]))

padding_idx (int, optional): If given, pads the output with zeros whenever it encounters the index. max_norm (float, optional): If given, will renormalize the embeddings to always have a norm lesser than this norm_type (float, optional): The p of the p-norm to compute for the max_norm option <del> scale_grad_by_freq (boolean, optional): if given, this will scale gradients by the frequency of </del> <ins> scale_grad_by_freq (bool, optional): if given, this will scale gradients by the frequency of </ins> the words in the mini-batch. <del> sparse (boolean, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </del> <ins> sparse (bool, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </ins> more details regarding sparse gradients. Attributes: weight (Tensor): the learnable weights of the module of shape (num_embeddings, embedding_dim) Shape: <del> - Input: LongTensor `(N, W)`, N = mini-batch, W = number of indices to extract per mini-batch - Output: `(N, W, embedding_dim)` </del> <ins> - Input: LongTensor of arbitrary shape containing the indices to extract - Output: `(*, embedding_dim)`, where `*` is the input shape </ins> Notes: Keep in mind that only a limited number of optimizers support

embedding_dim (int): the size of each embedding vector max_norm (float, optional): If given, will renormalize the embeddings to always have a norm lesser than this norm_type (float, optional): The p of the p-norm to compute for the max_norm option <del> scale_grad_by_freq (boolean, optional): if given, this will scale gradients by the frequency of </del> <ins> scale_grad_by_freq (bool, optional): if given, this will scale gradients by the frequency of </ins> the words in the dictionary. mode (string, optional): 'sum' | 'mean'. Specifies the way to reduce the bag. Default: 'mean' <del> sparse (boolean, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </del> <ins> sparse (bool, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </ins> more details regarding sparse gradients. Attributes: