Use of random and fx symbolic tracing

username: dskhudia
cooked: <p>Control flow with torch.rand or random.random (from python) gets traced but result is unexpected (see examples below). I understand that the docs <a href="https://pytorch.org/docs/stable/fx.html#miscellanea" rel="noopener nofollow ugc">here</a> point out this nondeterministic behavior and mention torch.rand is not traceable. Is <em>silently</em> tracing through such code with nondeterministic behavior reasonable?</p>
<pre><code class="lang-auto">import torch
import torch.fx

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(512, 512)

    def forward(self, x):
        x = self.linear(x)
        rand = torch.rand(1).item()
        if rand &lt; 0.5:
            x = torch.relu(x)
        return x

mod = MyModule()

traced_mod = torch.fx.symbolic_trace(mod)

traced_mod.graph.print_tabular()
"""
opcode       name    target    args       kwargs
-----------  ------  --------  ---------  --------
placeholder  x       x         ()         {}
call_module  linear  linear    (x,)       {}
output       output  output    (linear,)  {}

or 

opcode         name    target                                                   args       kwargs
-------------  ------  -------------------------------------------------------  ---------  --------
placeholder    x       x                                                        ()         {}
call_module    linear  linear                                                   (x,)       {}
call_function  relu    &lt;built-in method relu of type object at 0x7f14be3091a0&gt;  (linear,)  {}
output         output  output                                                   (relu,)    {}
"""
</code></pre>
<p>Wrapping with torch.fx.wrap also shows similar behavior.</p>
<pre><code class="lang-auto">import torch
import torch.fx

@torch.fx.wrap
def torch_rand(shape):
    return torch.rand(shape)

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = torch.nn.Linear(512, 512)

    def forward(self, x):
        x = self.linear(x)
        rand = torch_rand(1).item()
        if rand &lt; 0.5:
            x = torch.relu(x)
        return x

mod = MyModule()

traced_mod = torch.fx.symbolic_trace(mod)

traced_mod.graph.print_tabular()
"""
opcode         name    target                                                   args       kwargs
-------------  ------  -------------------------------------------------------  ---------  --------
placeholder    x       x                                                        ()         {}
call_module    linear  linear                                                   (x,)       {}
call_function  relu    &lt;built-in method relu of type object at 0x7f5ef0c251a0&gt;  (linear,)  {}
output         output  output                                                   (relu,)    {}

or 

opcode       name    target    args       kwargs
-----------  ------  --------  ---------  --------
placeholder  x       x         ()         {}
call_module  linear  linear    (x,)       {}
output       output  output    (linear,)  {}
"""
</code></pre>
<p>PyTorch version:</p>
<pre><code class="lang-auto">python -c "import torch; print(torch.__version__)"
1.10.2+cu113
</code></pre>