Repository: pytorch
Issue: Cannot cast float64 to float32
Labels: needs reproduction,module: crash,triaged,module: macos,module: numpy

### 🐛 Describe the bug

when i try to cast float64 to float32 tensor, it raises a segfault, but it seems nn.Linear requires float32 data, stuck in here.


```python
X = torch.from_numpy(X)
print(X.dtype)
y = torch.tensor(y, dtype=torch.float32)
train_data = TensorDataset(X, y)
train_loader = DataLoader(dataset=train_data, batch_size=batch_size, shuffle=True)

net = nn.Sequential(
    nn.Linear(X.shape[1], 16),
    nn.ReLU(),
    nn.Linear(16, 1)
)

trainer = torch.optim.Adam(net.parameters(), lr=lr)
loss = nn.MSELoss()
for epoch in range(epochs):
    for _X, _y in train_loader:
        trainer.zero_grad()
        l = loss(net(_X), _y)
        l.backward()
        trainer.step()
    with torch.no_grad():
        l = loss(net(X), y)
        print(f'epoch {epoch} train loss: {l}')
```

i got
```
torch.float64
Traceback (most recent call last):
  File "5_train_mlp.py", line 69, in <module>
    l = loss(net(_X), _y)
  File "/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: expected scalar type Float but found Double
```

then i try to make a cast
```python
X = torch.from_numpy(X).float()
print(X.dtype)
# same as above...
```
it returns most of the time
```
 48207 Segmentation fault: 11  python 5_train_mlp.py
```
the peculiar thing is for only once,  it didnt throw any exception but gave me a warning
```
torch.float32
/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/loss.py:530: UserWarning: Using a target size (torch.Size([16])) that is different to the input size (torch.Size([16, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/loss.py:530: UserWarning: Using a target size (torch.Size([3])) that is different to the input size (torch.Size([3, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
/Users/ben/miniconda3/envs/autogluon/lib/python3.7/site-packages/torch/nn/modules/loss.py:530: UserWarning: Using a target size (torch.Size([723])) that is different to the input size (torch.Size([723, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
  return F.mse_loss(input, target, reduction=self.reduction)
epoch 0 train loss: 36856.08203125
epoch 1 train loss: 446.7268371582031
epoch 2 train loss: 45.26797866821289
epoch 3 train loss: 43.33668518066406
...
```

So, i wonder what's wrong here and how can i fix it?

### Versions

❯ python collect_env.py
[1]    50310 segmentation fault  python collect_env.py

versions used: 
torch                         1.12.1
Python 3.7.12 

cc @malfet @albanD @mruberry @rgommers