Issue 1:
Issue description

Using the +=, -=, *=, /= operators gives the wrong answer in some cases without any warning. The matrix multiply assignment operator @= seems to work fine. I have tested in both PyTorch 0.3.0.post4 and PyTorch 0.4.0 (the latest version at the time of this issue creation) on the CPU. I have not tested whether this bug happens with CUDA.

Code example

I was able to produce a minimal case of this bug:

import torch

## Addition
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 += t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 + t2)

## Subtraction
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 -= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 - t2)

## Multiplication
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 *= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 * t2)

## Division
x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

t1 /= t2
print("Incorrect answer")
print(t1)

x = torch.Tensor([[1],[2],[3]])
t1 = x.expand(3, 3)
t2 = x.t().expand(3, 3)

print("Correct answer")
print(t1 / t2)

Output

Incorrect answer
tensor([[ 7., 7., 7.],
[ 21., 21., 21.],
[ 62., 62., 62.]])
Correct answer
tensor([[ 2., 3., 4.],
[ 3., 4., 5.],
[ 4., 5., 6.]])
Incorrect answer
tensor([[-5., -5., -5.],
[-3., -3., -3.],
[ 0., 0., 0.]])
Correct answer
tensor([[ 0., -1., -2.],
[ 1., 0., -1.],
[ 2., 1., 0.]])
Incorrect answer
tensor([[ 6.0000e+00, 6.0000e+00, 6.0000e+00],
[ 4.3200e+02, 4.3200e+02, 4.3200e+02],
[ 6.0466e+07, 6.0466e+07, 6.0466e+07]])
Correct answer
tensor([[ 1., 2., 3.],
[ 2., 4., 6.],
[ 3., 6., 9.]])
Incorrect answer
tensor([[ 0.1667, 0.1667, 0.1667],
[ 0.3333, 0.3333, 0.3333],
[ 1.0000, 1.0000, 1.0000]])
Correct answer
tensor([[ 1.0000, 0.5000, 0.3333],
[ 2.0000, 1.0000, 0.6667],
[ 3.0000, 1.5000, 1.0000]])

System Info

PyTorch version: 0.4.0

Is debug build: No

CUDA used to build PyTorch: 8.0.61

OS: Ubuntu 17.10

GCC version: (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0

CMake version: version 3.9.1

Python version: 3.6

Is CUDA available: No

CUDA runtime version: No CUDA

GPU models and configuration: No CUDA

Nvidia driver version: No CUDA

cuDNN version: No CUDA

Versions of relevant libraries:

[pip3] numpy (1.14.3)

[pip3] torch (0.4.0)

[conda] Could not collect

Issue 2:
Sometimes an in-place division where the denominator comes from the same tensor will yield unpredictable results. For example, if you want an array that counts from 0 to 100 in each of 50 rows, but actually you want to normalize to the range [0..1], it seems natural to write:

import torch
c = torch.arange(101)[None,:].repeat(50, 1)
c /= c[:,100:]
print(c.max())

However, running this on CPU repeatedly will produce an unpredictable max, e.g., 82, 95, 87, etc, rather than the expected 1.0.

This operation appears to be reliable on numpy, and for many array shapes it also works fine in pytorch. But it fails on certain sizes, leading to oddly rare and difficult-to-find bugs.

Should c /= c[:,-1:] be reliable? If not, would it be reasonable to warn on such operations?