Dynamo eager with sparse tensors gives wrong numeric results
Labels: triaged, oncall: pt2

Steps to reproduce:
1. Patch in https://github.com/pytorch/pytorch/pull/84641 (I think this fixes enough bugs that dynamo successfully captures some sparse code; you can compare with a run on master to confirm this)
2. Run `PYTORCH_TEST_WITH_DYNAMO=1 python test/test_sparse.py -k test_softmax_cpu_float64`

The unit test fails due to tensor difference.

Here is the info level log.

<details>

```
/data/users/ezyang/pytorch-tmp/torch/backends/cudnn/__init__.py:91: UserWarning: PyTorch was compiled without cuDNN/MIOpen support. To use cuDNN/MIOpen, rebuild PyTorch making sure the library is visible to the build system.
  warnings.warn(
/data/users/ezyang/torchdynamo/torchdynamo/eval_frame.py:106: DeprecationWarning: The 'warn' function is deprecated, use 'warning' instead
  logging.warn(
WARNING:root:Using TorchDynamo with a context manager will be deprecated soon.Please read https://github.com/pytorch/torchdynamo#usage-example to use TorchDynamo using an annotation.
torchdynamo.convert_frame: [ERROR] WON'T CONVERT setUp /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 97 
due to: 
Traceback (most recent call last):
  File "/data/users/ezyang/torchdynamo/torchdynamo/variables/user_defined.py", line 77, in call_method
    return super().call_method(tx, args, kwargs)
TypeError: call_method() missing 1 required positional argument: 'kwargs'

from user code:
   File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 98, in setUp
    TestCase.setUp(self)

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.symbolic_convert: [WARNING] Graph break: inline in skipfiles: genSparseTensor /data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_utils.py from user code at File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 116, in _gen_sparse
    x, i, v = self.genSparseTensor(with_size, sparse_dim, nnz, not coalesced, dtype=dtype, device=device)
torchdynamo.convert_frame: [INFO] ORIGINAL BYTECODE test_op /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3337 
3338           0 LOAD_GLOBAL              0 (isinstance)
               2 LOAD_FAST                2 (with_size)
               4 LOAD_GLOBAL              1 (Number)
               6 CALL_FUNCTION            2
               8 POP_JUMP_IF_FALSE       20

3339          10 LOAD_FAST                2 (with_size)
              12 BUILD_LIST               1
              14 LOAD_FAST                0 (sparse_dims)
              16 BINARY_MULTIPLY
              18 STORE_FAST               2 (with_size)

3341     >>   20 LOAD_DEREF               2 (self)
              22 LOAD_METHOD              2 (_gen_sparse)
              24 LOAD_FAST                0 (sparse_dims)
              26 LOAD_FAST                1 (nnz)
              28 LOAD_FAST                2 (with_size)
              30 LOAD_DEREF               1 (dtype)
              32 LOAD_DEREF               0 (device)
              34 LOAD_FAST                3 (coalesced)
              36 CALL_METHOD              6
              38 UNPACK_SEQUENCE          3
              40 STORE_FAST               4 (x)
              42 STORE_FAST               5 (i)
              44 STORE_FAST               6 (v)

3343          46 LOAD_CONST               1 (<code object sparse_log at 0x7fda3338cbe0, file "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3343>)
              48 LOAD_CONST               2 ('TestSparse.test_softmax.<locals>.test_op.<locals>.sparse_log')
              50 MAKE_FUNCTION            0
              52 STORE_FAST               7 (sparse_log)

3347          54 LOAD_GLOBAL              3 (range)
              56 LOAD_FAST                4 (x)
              58 LOAD_METHOD              4 (sparse_dim)
              60 CALL_METHOD              0
              62 LOAD_FAST                4 (x)
              64 LOAD_METHOD              5 (dense_dim)
              66 CALL_METHOD              0
              68 BINARY_ADD
              70 CALL_FUNCTION            1
              72 GET_ITER
         >>   74 EXTENDED_ARG             1
              76 FOR_ITER               462 (to 540)
              78 STORE_FAST               8 (dim)

3351          80 LOAD_DEREF               6 (sparse_softmax)
              82 LOAD_FAST                4 (x)
              84 LOAD_FAST                8 (dim)
              86 CALL_FUNCTION            2
              88 STORE_FAST               9 (y)

3352          90 LOAD_DEREF               5 (softmax_to_dense)
              92 LOAD_FAST                4 (x)
              94 LOAD_FAST                8 (dim)
              96 CALL_FUNCTION            2
              98 STORE_FAST              10 (r1)

3353         100 LOAD_FAST                9 (y)
             102 LOAD_METHOD              6 (to_dense)
             104 CALL_METHOD              0
             106 STORE_FAST              11 (r2)

3354         108 LOAD_GLOBAL              7 (print)
             110 LOAD_FAST               10 (r1)
             112 CALL_FUNCTION            1
             114 POP_TOP

3355         116 LOAD_GLOBAL              7 (print)
             118 LOAD_FAST               11 (r2)
             120 CALL_FUNCTION            1
             122 POP_TOP

3356         124 LOAD_DEREF               2 (self)
             126 LOAD_METHOD              8 (assertEqual)
             128 LOAD_FAST               10 (r1)
             130 LOAD_FAST               11 (r2)
             132 CALL_METHOD              2
             134 POP_TOP

3359         136 LOAD_GLOBAL              9 (torch)
             138 LOAD_ATTR               10 (sparse)
             140 LOAD_METHOD             11 (softmax)
             142 LOAD_FAST                4 (x)
             144 LOAD_FAST                8 (dim)
             146 CALL_METHOD              2
             148 STORE_FAST              12 (y1)

3360         150 LOAD_DEREF               2 (self)
             152 LOAD_METHOD              8 (assertEqual)
             154 LOAD_FAST                9 (y)
             156 LOAD_FAST               12 (y1)
             158 CALL_METHOD              2
             160 POP_TOP

3363         162 LOAD_GLOBAL              9 (torch)
             164 LOAD_ATTR               10 (sparse)
             166 LOAD_METHOD             12 (log_softmax)
             168 LOAD_FAST                4 (x)
             170 LOAD_FAST                8 (dim)
             172 CALL_METHOD              2
             174 STORE_FAST              13 (ly1)

3364         176 LOAD_DEREF               2 (self)
             178 LOAD_METHOD              8 (assertEqual)
             180 LOAD_FAST               13 (ly1)
             182 LOAD_FAST                7 (sparse_log)
             184 LOAD_FAST               12 (y1)
             186 CALL_FUNCTION            1
             188 CALL_METHOD              2
             190 POP_TOP

3369         192 LOAD_DEREF               7 (to_dense)
             194 LOAD_FAST                4 (x)
             196 LOAD_GLOBAL             13 (float)
             198 LOAD_CONST               3 ('-inf')
             200 CALL_FUNCTION            1
             202 LOAD_CONST               4 (('fill_value',))
             204 CALL_FUNCTION_KW         2
             206 STORE_FAST              14 (x1)

3370         208 LOAD_DEREF               3 (softmax_jacobian_analytic)
             210 LOAD_FAST               14 (x1)
             212 LOAD_FAST                8 (dim)
             214 CALL_FUNCTION            2
             216 STORE_FAST              15 (J)

3371         218 LOAD_FAST               15 (J)
             220 LOAD_ATTR               14 (shape)
             222 LOAD_CONST               5 (0)
             224 BINARY_SUBSCR
             226 LOAD_FAST                4 (x)
             228 LOAD_ATTR               14 (shape)
             230 LOAD_FAST                8 (dim)
             232 BINARY_SUBSCR
             234 COMPARE_OP               2 (==)
             236 POP_JUMP_IF_TRUE       242
             238 LOAD_ASSERTION_ERROR
             240 RAISE_VARARGS            1

3372     >>  242 LOAD_FAST               15 (J)
             244 LOAD_ATTR               14 (shape)
             246 LOAD_FAST                8 (dim)
             248 LOAD_CONST               6 (1)
             250 BINARY_ADD
             252 BINARY_SUBSCR
             254 LOAD_FAST                4 (x)
             256 LOAD_ATTR               14 (shape)
             258 LOAD_FAST                8 (dim)
             260 BINARY_SUBSCR
             262 COMPARE_OP               2 (==)
             264 EXTENDED_ARG             1
             266 POP_JUMP_IF_TRUE       272
             268 LOAD_ASSERTION_ERROR
             270 RAISE_VARARGS            1

3375     >>  272 LOAD_DEREF               4 (softmax_jacobian_autograd)
             274 LOAD_FAST               14 (x1)
             276 LOAD_FAST                8 (dim)
             278 CALL_FUNCTION            2
             280 STORE_FAST              16 (J2)

3376         282 LOAD_DEREF               2 (self)
             284 LOAD_METHOD              8 (assertEqual)
             286 LOAD_FAST               15 (J)
             288 LOAD_FAST               16 (J2)
             290 CALL_METHOD              2
             292 POP_TOP

3379         294 LOAD_DEREF               4 (softmax_jacobian_autograd)
             296 LOAD_FAST                4 (x)
             298 LOAD_FAST                8 (dim)
             300 CALL_FUNCTION            2
             302 STORE_FAST              17 (J3)

3380         304 LOAD_DEREF               2 (self)
             306 LOAD_METHOD              8 (assertEqual)
             308 LOAD_FAST               15 (J)
             310 LOAD_FAST               17 (J3)
             312 CALL_METHOD              2
             314 POP_TOP

3390         316 LOAD_DEREF               4 (softmax_jacobian_autograd)
             318 LOAD_FAST               14 (x1)
             320 LOAD_FAST                8 (dim)
             322 LOAD_CONST               7 (True)
             324 LOAD_CONST               8 (('log',))
             326 CALL_FUNCTION_KW         3
             328 STORE_FAST              18 (J2_log)

3393         330 LOAD_DEREF               4 (softmax_jacobian_autograd)
             332 LOAD_FAST                4 (x)
             334 LOAD_FAST                8 (dim)
             336 LOAD_CONST               7 (True)
             338 LOAD_CONST               8 (('log',))
             340 CALL_FUNCTION_KW         3
             342 STORE_FAST              19 (J3_log)

3395         344 LOAD_FAST               15 (J)
             346 LOAD_METHOD             15 (transpose)
             348 LOAD_CONST               5 (0)
             350 LOAD_FAST                8 (dim)
             352 LOAD_CONST               6 (1)
             354 BINARY_ADD
             356 CALL_METHOD              2
             358 STORE_FAST              15 (J)

3396         360 LOAD_FAST               18 (J2_log)
             362 LOAD_METHOD             15 (transpose)
             364 LOAD_CONST               5 (0)
             366 LOAD_FAST                8 (dim)
             368 LOAD_CONST               6 (1)
             370 BINARY_ADD
             372 CALL_METHOD              2
             374 STORE_FAST              18 (J2_log)

3397         376 LOAD_FAST               19 (J3_log)
             378 LOAD_METHOD             15 (transpose)
             380 LOAD_CONST               5 (0)
             382 LOAD_FAST                8 (dim)
             384 LOAD_CONST               6 (1)
             386 BINARY_ADD
             388 CALL_METHOD              2
             390 STORE_FAST              19 (J3_log)

3398         392 LOAD_DEREF               2 (self)
             394 LOAD_METHOD              8 (assertEqual)
             396 LOAD_FAST               15 (J)
             398 LOAD_FAST               18 (J2_log)
             400 LOAD_FAST               10 (r1)
             402 BINARY_MULTIPLY
             404 CALL_METHOD              2
             406 POP_TOP

3399         408 LOAD_DEREF               2 (self)
             410 LOAD_METHOD              8 (assertEqual)
             412 LOAD_FAST               15 (J)
             414 LOAD_FAST               19 (J3_log)
             416 LOAD_FAST               10 (r1)
             418 BINARY_MULTIPLY
             420 CALL_METHOD              2
             422 POP_TOP

3401         424 LOAD_FAST                8 (dim)
             426 LOAD_CONST               5 (0)
             428 COMPARE_OP               2 (==)
             430 POP_JUMP_IF_FALSE       74

3403         432 LOAD_GLOBAL              9 (torch)
             434 LOAD_ATTR               16 (float32)
             436 STORE_FAST              20 (other_dtype)

3404         438 LOAD_GLOBAL              9 (torch)
             440 LOAD_ATTR               10 (sparse)
             442 LOAD_ATTR               11 (softmax)
             444 LOAD_FAST                4 (x)
             446 LOAD_FAST                8 (dim)
             448 LOAD_FAST               20 (other_dtype)
             450 LOAD_CONST               9 (('dtype',))
             452 CALL_FUNCTION_KW         3
             454 STORE_FAST              21 (y2)

3405         456 LOAD_DEREF               2 (self)
             458 LOAD_METHOD              8 (assertEqual)
             460 LOAD_FAST               21 (y2)
             462 LOAD_ATTR               17 (dtype)
             464 LOAD_FAST               20 (other_dtype)
             466 CALL_METHOD              2
             468 POP_TOP

3406         470 LOAD_DEREF               2 (self)
             472 LOAD_METHOD              8 (assertEqual)
             474 LOAD_FAST               21 (y2)
             476 LOAD_FAST               12 (y1)
             478 LOAD_METHOD             18 (type)
             480 LOAD_FAST               20 (other_dtype)
             482 CALL_METHOD              1
             484 CALL_METHOD              2
             486 POP_TOP

3408         488 LOAD_GLOBAL              9 (torch)
             490 LOAD_ATTR               10 (sparse)
             492 LOAD_ATTR               12 (log_softmax)
             494 LOAD_FAST                4 (x)
             496 LOAD_FAST                8 (dim)
             498 LOAD_FAST               20 (other_dtype)
             500 LOAD_CONST               9 (('dtype',))
             502 CALL_FUNCTION_KW         3
             504 STORE_FAST              22 (ly2)

3409         506 LOAD_DEREF               2 (self)
             508 LOAD_METHOD              8 (assertEqual)
             510 LOAD_FAST               22 (ly2)
             512 LOAD_ATTR               17 (dtype)
             514 LOAD_FAST               20 (other_dtype)
             516 CALL_METHOD              2
             518 POP_TOP

3410         520 LOAD_DEREF               2 (self)
             522 LOAD_METHOD              8 (assertEqual)
             524 LOAD_FAST               22 (ly2)
             526 LOAD_FAST               13 (ly1)
             528 LOAD_METHOD             18 (type)
             530 LOAD_FAST               20 (other_dtype)
             532 CALL_METHOD              1
             534 CALL_METHOD              2
             536 POP_TOP
             538 JUMP_ABSOLUTE           74
         >>  540 LOAD_CONST               0 (None)
             542 RETURN_VALUE

 
torchdynamo.convert_frame: [INFO] MODIFIED BYTECODE test_op /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3337 
3337           0 LOAD_DEREF               2 (self)
               2 LOAD_ATTR                2 (_gen_sparse)
               4 LOAD_FAST                0 (sparse_dims)
               6 LOAD_FAST                1 (nnz)
               8 LOAD_FAST                2 (with_size)
              10 LOAD_DEREF               1 (dtype)
              12 LOAD_DEREF               0 (device)
              14 LOAD_FAST                3 (coalesced)
              16 CALL_FUNCTION            6
              18 LOAD_CLOSURE             0 (device)
              20 LOAD_CLOSURE             1 (dtype)
              22 LOAD_CLOSURE             2 (self)
              24 LOAD_CLOSURE             3 (softmax_jacobian_analytic)
              26 LOAD_CLOSURE             4 (softmax_jacobian_autograd)
              28 LOAD_CLOSURE             5 (softmax_to_dense)
              30 LOAD_CLOSURE             6 (sparse_softmax)
              32 LOAD_CLOSURE             7 (to_dense)
              34 BUILD_TUPLE              8
              36 LOAD_CONST              10 (<code object <graph break in test_op> at 0x7fd8309aef50, file "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3341>)
              38 LOAD_CONST              11 ('__resume_at_38_0')
              40 MAKE_FUNCTION            8 (closure)
              42 ROT_TWO
              44 CALL_FUNCTION            1
              46 RETURN_VALUE

 
torchdynamo.convert_frame: [INFO] GUARDS:
 - 
            local 'nnz' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ['___check_type_id(nnz, 94813225457600)', 'nnz == 10'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d1180; to 'type' at 0x563b6c9f97c0 (int)>
            }
            
 - 
            local 'self' TYPE_MATCH"
            {
                'guard_types': ['TYPE_MATCH'],
                'code': ['___check_type_id(self, 94814090822400)'],
                'obj_weakref': <weakref at 0x7fd83087cc20; to 'TestSparseCPU' at 0x7fd831093880>
                'guarded_class': <weakref at 0x7fd831508900; to 'type' at 0x563ba0340300 (TestSparseCPU)>
            }
            
 - 
            local 'dtype' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ["str(dtype) == 'torch.float64'"],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda32ff3270; to 'type' at 0x7fd9f88413a0 (dtype)>
            }
            
 - 
            local 'device' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ['___check_type_id(device, 94813225422688)', "device == 'cpu'"],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413e4e00; to 'type' at 0x563b6c9f0f60 (str)>
            }
            
 - 
            local 'coalesced' CONSTANT_MATCH"
            {
                'guard_types': ['ID_MATCH'],
                'code': ['___check_obj_id(coalesced, 94813225495552)'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d1450; to 'type' at 0x563b6ca02a40 (bool)>
            }
            
 - 
            local 'with_size' EQUALS_MATCH"
            {
                'guard_types': ['LIST_LENGTH', 'EQUALS_MATCH'],
                'code': ['___check_type_id(with_size, 94813225459872)', 'len(with_size) == 1', '___check_type_id(with_size[0], 94813225457600)', 'with_size == [3]'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d7d60; to 'type' at 0x563b6c9fa0a0 (list)>
            }
            
 - 
            local 'sparse_dims' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ['___check_type_id(sparse_dims, 94813225457600)', 'sparse_dims == 1'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d1180; to 'type' at 0x563b6c9f97c0 (int)>
            }
            
 - 
            local 'with_size[0]' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ['___check_type_id(with_size[0], 94813225457600)', 'with_size[0] == 3'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d1180; to 'type' at 0x563b6c9f97c0 (int)>
            }
            
 - 
            global 'Number' FUNCTION_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
 - 
            global 'isinstance' BUILTIN_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
torchdynamo.convert_frame: [ERROR] WON'T CONVERT <graph break in test_op> /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3341 
due to: 
Traceback (most recent call last):
  File "/data/users/ezyang/torchdynamo/torchdynamo/variables/tensor.py", line 204, in create
    torch.distributed.get_rank,
AttributeError: module 'torch.distributed' has no attribute 'get_rank'

from user code:
   File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3347, in <graph break in test_op>
    for dim in range(x.sparse_dim() + x.dense_dim()):

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.convert_frame: [ERROR] WON'T CONVERT sparse_softmax /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3172 
due to: 
Traceback (most recent call last):
  File "/data/users/ezyang/torchdynamo/torchdynamo/variables/tensor.py", line 204, in create
    torch.distributed.get_rank,
AttributeError: module 'torch.distributed' has no attribute 'get_rank'

from user code:
   File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3187, in sparse_softmax
    if dim < sparse.sparse_dim():

Set torchdynamo.config.verbose=True for more information
==========
torchdynamo.output_graph: [INFO] TRACED GRAPH
 __compiled_fn_1 <eval_with_key>.1 opcode         name      target                                                   args                kwargs
-------------  --------  -------------------------------------------------------  ------------------  ------------------------------------------------------
placeholder    sparse    sparse                                                   ()                  {}
call_method    coalesce  coalesce                                                 (sparse,)           {}
call_function  full      <built-in method full of type object at 0x7fd9f88450e0>  ((3,), -inf)        {'dtype': torch.float64, 'device': device(type='cpu')}
call_method    _indices  _indices                                                 (coalesce,)         {}
call_method    t         t                                                        (_indices,)         {}
call_method    _values   _values                                                  (coalesce,)         {}
call_function  softmax   <function softmax at 0x7fd9337c11f0>                     (full, 0)           {}
call_function  ne        <built-in function ne>                                   (softmax, softmax)  {}
call_function  setitem   <built-in function setitem>                              (softmax, ne, 0)    {}
output         output    output                                                   ((softmax,),)       {}

torchdynamo.convert_frame: [INFO] ORIGINAL BYTECODE softmax_to_dense /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3147 
3164           0 LOAD_FAST                0 (sparse)
               2 LOAD_ATTR                0 (dtype)
               4 STORE_FAST               2 (dtype)

3165           6 LOAD_FAST                0 (sparse)
               8 LOAD_ATTR                1 (device)
              10 STORE_FAST               3 (device)

3166          12 LOAD_DEREF               1 (to_dense)
              14 LOAD_FAST                0 (sparse)
              16 LOAD_GLOBAL              2 (float)
              18 LOAD_CONST               1 ('inf')
              20 CALL_FUNCTION            1
              22 UNARY_NEGATIVE
              24 LOAD_CONST               2 (('fill_value',))
              26 CALL_FUNCTION_KW         2
              28 STORE_FAST               4 (dense)

3167          30 LOAD_DEREF               0 (F)
              32 LOAD_METHOD              3 (softmax)
              34 LOAD_FAST                4 (dense)
              36 LOAD_FAST                1 (dim)
              38 CALL_METHOD              2
              40 STORE_FAST               5 (r)

3169          42 LOAD_CONST               3 (0)
              44 LOAD_FAST                5 (r)
              46 LOAD_FAST                5 (r)
              48 LOAD_FAST                5 (r)
              50 COMPARE_OP               3 (!=)
              52 STORE_SUBSCR

3170          54 LOAD_FAST                5 (r)
              56 RETURN_VALUE

 
torchdynamo.convert_frame: [INFO] MODIFIED BYTECODE softmax_to_dense /data/users/ezyang/pytorch-tmp/test/test_sparse.py line 3147 
3147           0 LOAD_GLOBAL              4 (__compiled_fn_1)
               2 LOAD_FAST                0 (sparse)
               4 CALL_FUNCTION            1
               6 UNPACK_SEQUENCE          1
               8 RETURN_VALUE

 
torchdynamo.convert_frame: [INFO] GUARDS:
 - 
            local 'F' FUNCTION_MATCH"
            {
                'guard_types': ['ID_MATCH'],
                'code': ['___check_obj_id(F, 140570849544208)'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda4138ca90; to 'type' at 0x563b6c9f6e60 (module)>
            }
            
 - 
            local 'dim' CONSTANT_MATCH"
            {
                'guard_types': ['EQUALS_MATCH'],
                'code': ['___check_type_id(dim, 94813225457600)', 'dim == 0'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413d1180; to 'type' at 0x563b6c9f97c0 (int)>
            }
            
 - 
            local 'sparse' TENSOR_MATCH"
            {
                'guard_types': ['TENSOR_MATCH'],
                'code': None,
                'obj_weakref': <weakref at 0x7fd83087d7c0; to 'Tensor' at 0x7fd83087ce00>
                'guarded_class': <weakref at 0x7fd9d63e2180; to 'torch._C._TensorMeta' at 0x563b706941b0 (Tensor)>
            }
            
 - 
            local 'to_dense' FUNCTION_MATCH"
            {
                'guard_types': ['ID_MATCH'],
                'code': ['___check_obj_id(to_dense, 140566504178304)'],
                'obj_weakref': None
                'guarded_class': <weakref at 0x7fda413f04a0; to 'type' at 0x563b6c9d5c00 (function)>
            }
            
 - 
            global 'zip' BUILTIN_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
 - 
            global 'float' BUILTIN_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
 - 
            global 'torch' FUNCTION_MATCH"
            {
                'guard_types': None,
                'code': None,
                'obj_weakref': None
                'guarded_class': None
            }
            
F
======================================================================
FAIL: test_softmax_cpu_float64 (__main__.TestSparseCPU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_device_type.py", line 378, in instantiated_test
    result = test(self, **param_kwargs)
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_utils.py", line 3276, in wrapped
    f(self, *args, **kwargs, coalesced=True)
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3412, in test_softmax
    test_op(1, 10, [3], coalesced)
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3337, in test_op
    def test_op(sparse_dims, nnz, with_size, coalesced):
  File "/data/users/ezyang/pytorch-tmp/test/test_sparse.py", line 3356, in <graph break in test_op>
    self.assertEqual(r1, r2)
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_internal/common_utils.py", line 2411, in assertEqual
    assert_equal(
  File "/data/users/ezyang/pytorch-tmp/torch/testing/_comparison.py", line 1093, in assert_equal
    raise error_metas[0].to_error(msg)
AssertionError: Tensor-likes are not close!

Mismatched elements: 3 / 3 (100.0%)
Greatest absolute difference: 0.6566471149084403 at index (0,) (up to 1e-07 allowed)
Greatest relative difference: 1.0 at index (0,) (up to 1e-07 allowed)

----------------------------------------------------------------------
Ran 1 test in 0.384s

FAILED (failures=1)
tensor([0., 0., 0.], dtype=torch.float64)
tensor([0.6566, 0.1139, 0.2295], dtype=torch.float64)
```

</details>

In particular, notice there is a sparse tensor in the guard set:

```
 -                                                       
            local 'sparse' TENSOR_MATCH"                 
            {                                            
                'guard_types': ['TENSOR_MATCH'],         
                'code': None,                                                                                     
                'obj_weakref': <weakref at 0x7f730555f360; to 'Tensor' at 0x7f73055d2b80>                         
                'guarded_class': <weakref at 0x7f74aa5913b0; to 'torch._C._TensorMeta' at 0x5627ea2b4af0 (Tensor)>
            } 
```

My suspicion is that it has to do with sparse tensors guards. I checked the TensorGuard logic, and there are no sparse specific checks as far as I can tell. But I couldn't figure out how to easily tell if a guard was matching or not, the debug level doesn't say it.

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh