Unable to reproduce the same result with torch::manual_seed

username: ianlienfa
cooked: <p>Hello guys, I’m now training an DDPG model with libtorch c++ api<br>
before the network training, I’ve called</p>
<pre><code class="lang-auto">int seed = SEED;
srand(seed);
torch::manual_seed(SEED)
</code></pre>
<p>in the main function,<br>
but I still get different result every time.</p>
<p>Here are my network structures:<br>
Critic:</p>
<pre><code class="lang-auto">    net = register_module("Sequential", nn::Sequential(
        nn::Linear(ops.state_dim + ops.action_dim + ops.max_num_contour, 32),
        nn::ReLU(),
        nn::Linear(32, 64),
        nn::ReLU(),
        nn::Linear(64, 32),
        nn::ReLU(),
        nn::Linear(32, 1)
    ));
</code></pre>
<p>Actor:</p>
<pre><code class="lang-auto">    net = register_module("Sequential", 
        nn::Sequential(
            nn::Linear(ops.state_dim + ops.max_num_contour, 64),
            nn::ReLU(),
            nn::Linear(64, 32),
            nn::ReLU(),
            nn::Linear(32, (ops.action_range.second - 1) + 2)
        )
    );       
</code></pre>
<p>Optimizers:</p>
<pre><code class="lang-auto">    optimizer_q = std::make_shared&lt;torch::optim::Adam&gt;(net-&gt;q-&gt;parameters(), lr_q);
    optimizer_pi = std::make_shared&lt;torch::optim::Adam&gt;(net-&gt;pi-&gt;parameters(), lr_pi);    

</code></pre>
<p>Weight Initializations</p>
<pre><code class="lang-auto">    {
        torch::NoGradGuard no_grad;
        // weight init
        auto initialize_weights_norm = [](nn::Module&amp; module) {
            torch::NoGradGuard no_grad;
            if (auto* linear = module.as&lt;nn::Linear&gt;()) {
                torch::nn::init::kaiming_uniform_(linear-&gt;weight);
                torch::nn::init::constant_(linear-&gt;bias, 0.01);
            }
        };
        this-&gt;apply(initialize_weights_norm);
    }
</code></pre>
<p>And the c-library rand() is used for exploration.</p>
<p>Is it possible that there are other function or algorithm that has indeterministic behaviors that I didn’t notice?</p>

username: AidanShipperley
cooked: <p>Have you tried using <code>at::globalContext().setDeterministicCuDNN(true)</code>?</p>

username: ianlienfa
cooked: <aside class="quote no-group" data-username="AidanShipperley" data-post="2" data-topic="151348">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aidanshipperley/48/59501_2.png" class="avatar"> AidanShipperley:</div>
<blockquote>
<p>at::globalContext().setDeterministicCuDNN(true)</p>
</blockquote>
</aside>
<p>Hi, I’ve tried this and the randomness is still there.<br>
Btw, I didn’t use GPU in my code, so I don’t really get where the randomness comes from.</p>

username: AidanShipperley
cooked: <p>The only other thing I can think of would be to use deterministic algorithms as well. I achieved deterministic results in my PPO implementation that way. Code of my seeding:</p>
<pre><code class="lang-auto">int m_seed = 0;
bool m_torch_deterministic = true;
srand(m_seed);
torch::manual_seed(m_seed);
at::globalContext().setDeterministicCuDNN(m_torch_deterministic ? true : false);
// https://docs.nvidia.com/cuda/cublas/index.html#cublasApi_reproducibility
at::globalContext().setDeterministicAlgorithms(m_torch_deterministic ? true : false, true);
</code></pre>
<p>I would still do this even if you don’t use GPU. Other than that, I would try to check every aspect of your program through unit testing. It’s possible that maybe the way you designed your code may cause reseeding to occur at a time you don’t want to. Please also read <a href="https://pytorch.org/docs/stable/notes/randomness.html" class="inline-onebox" rel="noopener nofollow ugc">Reproducibility — PyTorch 1.12 documentation</a> and see if anything is there that you have not tried yet. Maybe if you are still running into issues you could share a small working example that I could test out. Let me know!</p>