Repository: pytorch
Issue: Tracing interpolate with tensor scale_factor is cursed
Labels: triaged,oncall: pt2,module: dynamic shapes,oncall: export

### 🐛 Describe the bug

This test

```
    def test_interpolate_tensor_scale_factor(self):
        x = torch.randn(4, 3, 32, 32)
        factor = torch.tensor(2.0)

        def f(x, factor):
            resized_tensor = torch.nn.functional.interpolate(x, scale_factor=factor.item(), mode='bilinear', recompute_scale_factor=True)

        make_fx(f, tracing_mode="symbolic")(x, factor)
```

fails with

```
  File "/data/users/ezyang/b/pytorch/test/test_proxy_tensor.py", line 1069, in f
    resized_tensor = torch.nn.functional.interpolate(x, scale_factor=factor.item(), mode='bilinear', recompute_scale_factor=True)
  File "/data/users/ezyang/b/pytorch/torch/nn/functional.py", line 3971, in interpolate
    output_size = [
  File "/data/users/ezyang/b/pytorch/torch/nn/functional.py", line 3972, in <listcomp>
    _sym_int(input.size(i + 2) * scale_factors[i])
  File "/data/users/ezyang/b/pytorch/torch/__init__.py", line 419, in sym_int
    return math.floor(a) if a >= 0 else math.ceil(a)  # type: ignore[arg-type, call-overload]
  File "/data/users/ezyang/b/pytorch/torch/__init__.py", line 352, in __bool__
    return self.node.bool_()
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 973, in bool_
    return self.guard_bool("", 0)
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 955, in guard_bool
    r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3461, in evaluate_expr
    concrete_val = self.size_hint(orig_expr)
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3262, in size_hint
    raise self._make_data_dependent_error(result_expr, expr)
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.)  The expression we were trying to evaluate is 32*f0 >= 0 (unhinted: f0*s2 >= 0).  Scroll up to see where each of these data-dependent accesses originally occurred.
```

If you remove the item() call (which was the case in the original torchvision model), you hit https://github.com/pytorch/pytorch/issues/108777

If you remove `recompute_scale_factor`, you get

```
TypeError: upsample_bilinear2d() received an invalid combination of arguments - got (FakeTensor, NoneType, bool, list), but expected one of:                                                                                                              
 * (Tensor input, tuple of ints output_size, bool align_corners, tuple of floats scale_factors)                              
      didn't match because some of the arguments have invalid types: (FakeTensor, NoneType, bool, list of [SymFloat, SymFloat])                                                                                                                           
 * (Tensor input, tuple of ints output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out) 
```

Overall, this is all kind of cursed and multiple things need to happen:

* If the scale factor is truly unbacked, then we need to make sure it is possible to insert enough constrain_range so that the data-dependent guards can be discharged. But this is extra cursed, because the guards in question are *floating point* operations, and we generally don't have good FP reasoning in sympy. The particular problem here requires us to reason that the scale factor is >= 0, which is not too bad, but there is then a subsequent call to arange that also has to be dealt with:

```
  File "/data/users/ezyang/b/pytorch/torch/_decomp/decompositions.py", line 3779, in arange_default                              return aten.arange.start_step(                                                                                           
  File "/data/users/ezyang/b/pytorch/torch/_ops.py", line 448, in __call__                                                   
    return self._op(*args, **kwargs or {})                                                                                   
  File "/data/users/ezyang/b/pytorch/torch/utils/_stats.py", line 20, in wrapper                                             
    return fn(*args, **kwargs)                                                                                               
  File "/data/users/ezyang/b/pytorch/torch/_subclasses/fake_tensor.py", line 1290, in __torch_dispatch__                     
    return self.dispatch(func, types, args, kwargs)                                                                          
  File "/data/users/ezyang/b/pytorch/torch/_subclasses/fake_tensor.py", line 1495, in dispatch                               
    return decomposition_table[func](*args, **kwargs)                                                                        
  File "/data/users/ezyang/b/pytorch/torch/_prims_common/wrappers.py", line 229, in _fn                                      
    result = fn(*args, **kwargs)                                                                                             
  File "/data/users/ezyang/b/pytorch/torch/_refs/__init__.py", line 4727, in arange                                          
    (step > 0 and end >= start) or (step < 0 and end <= start),                                                              
  File "/data/users/ezyang/b/pytorch/torch/__init__.py", line 352, in __bool__                                               
    return self.node.bool_()                                                                                                 
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 973, in bool_                           
    return self.guard_bool("", 0)                                                                                            
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 955, in guard_bool                          r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node)                                               File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3461, in evaluate_expr                  
    concrete_val = self.size_hint(orig_expr)                                                                                 
  File "/data/users/ezyang/b/pytorch/torch/fx/experimental/symbolic_shapes.py", line 3262, in size_hint                      
    raise self._make_data_dependent_error(result_expr, expr)                                                                 
torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and thus we do not know the true value.)  The expression we were trying to evaluate is floor(300*f0) >= 0 (unhinted: floor(300*f0) >= 0).  Scroll up to see where each of these data-dependent accesses originally occurred.  
```

* If you don't pass `recompute_scale_factors`, we must accept SymFloat scale factor in the kernel. But currently it only takes `float[]` so a schema change is necessary. But we don't support SymFloat in schema so the schema must change to `Scalar[]` which is BC-breaking.

I... kind of suspect that we actually do know the output size in the model in question? Going to check that out next.

Extracted from https://github.com/pytorch/pytorch/issues/108337

### Versions

main

cc @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519