Issue 1:
🐛 Bug

Inconsistent results of torch.argmax with tensors that have duplicated values

To Reproduce

cpu = torch.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 3])
cuda = torch.cuda.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cuda.argmax(dim=1) #tensor([4, 1], device='cuda:0')

When I did the same on another machine(pytorch 0.4.1, CUDA 9.0)

cpu = torch.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 1]).
cuda = torch.cuda.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cuda.argmax(dim=1) # tensor([1, 1, device='cuda:0')

Expected behavior

torch.argmax returns the same value for tensors that have same data in it, regardless of devices(or other factors).

Environment

PyTorch version: 0.4.1.post2

Is debug build: No

CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.5 LTS

GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609

CMake version: version 3.5.1

Python version: 3.6

Is CUDA available: Yes

CUDA runtime version: 9.0.176

GPU models and configuration:

GPU 0: GeForce GTX 1080 Ti

GPU 1: GeForce GTX 1080 Ti

Nvidia driver version: 410.79

cuDNN version: /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.3.1

Versions of relevant libraries:

[pip3] msgpack-numpy==0.4.4.1

[pip3] numpy==1.15.0

[pip3] torch==0.4.1.post2

[pip3] torchtext==0.3.1

[pip3] torchvision==0.2.1

[conda] blas 1.0 mkl

[conda] mkl 2018.0.3 1

[conda] mkl_fft 1.0.4 py36h4414c95_1

[conda] mkl_random 1.0.1 py36h4414c95_1

[conda] mxnet-mkl 1.3.0.post0

[conda] pytorch 0.4.1 py36_py35_py27__9.0.176_7.1.2_2 pytorch

[conda] torchtext 0.3.1

[conda] torchvision 0.2.1 py36_0

Additional context

It would be helpful if the documentation describes if torch.argmax returns the last or the first index if a tensor holds duplicated values.

Issue 2:
Currently there doesn't seem to be a way to skip tracing of certain functions. Encountered this error when trying to symbolically trace open source BERT model to quantize it.

Original error is from

https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L238

Unit test to repro -

import torch
from torch.fx import symbolic_trace
import torch.nn as nn

class M(torch.nn.Module):
def __init__(self):
super().__init__()

def transpose_inp(self, x):
new_shape = x.size()[:-1] + (5, 5)
x = x.view(*new_shape)
return x.permute(0, 2, 1, 3)

def forward(self, x):
tmp = self.transpose_inp(x)
return tmp

m = symbolic_trace(M())

Throws the following error

m = symbolic_trace(M())
File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/symbolic_trace.py", line 219, in symbolic_trace
graph.output(delegate.create_arg(fn(*args)))
File "scripts/test_fx.py", line 49, in forward
tmp = self.transpose_inp(x)
File "scripts/test_fx.py", line 44, in transpose_inp
x = x.view(*new_shape)
File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py", line 62, in __iter__
self._no_arg_unpack()
File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py", line 70, in _no_arg_unpack
raise TraceError('Proxy object cannot be unpacked as function argument')
torch.fx.proxy.TraceError: Proxy object cannot be unpacked as function argument

cc @gmagogsfm @jerryzh168, @vkuzo