How to initialize network to output zeros?

username: ElLoboLoco
cooked: <p>Hi,</p>
<p>I’m currently trying to implement the algorithm found in this paper : <a href="https://arxiv.org/pdf/1811.00164.pdf" rel="noopener nofollow ugc">https://arxiv.org/pdf/1811.00164.pdf</a>.</p>
<p>However, at some point, the algorithm need to initialize the neural network to return 0 for all inputs (line 1). <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba6eaa1402e7a31755c75f2e3cc7786f31559b14.png" data-download-href="https://discuss.pytorch.org/uploads/default/ba6eaa1402e7a31755c75f2e3cc7786f31559b14" title="Capture d’écran 2021-04-21 à 22.05.12"><img src="https://discuss.pytorch.org/uploads/default/original/3X/b/a/ba6eaa1402e7a31755c75f2e3cc7786f31559b14.png" alt="Capture d’écran 2021-04-21 à 22.05.12" data-base62-sha1="qBfUdO9OaXxlYezMpRrtXjOlNJi" width="690" height="220" data-dominant-color="EFEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Capture d’écran 2021-04-21 à 22.05.12</span><span class="informations">961×307 25.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So my question is simply how do I do that ? At first I thought to initialize the weights of the output layer to zero but it will prevent it from learning. I’m sure there is a simple method but I can’t find it.</p>

username: googlebot
cooked: <aside class="quote no-group" data-username="ElLoboLoco" data-post="1" data-topic="119007">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/elloboloco/48/30094_2.png" class="avatar"> ElLoboLoco:</div>
<blockquote>
<p>At first I thought to initialize the weights of the output layer to zero but it will prevent it from learning</p>
</blockquote>
</aside>
<p>It won’t, this only applies to hidden layers (that generate “artifical” features). For output layer, loss (eq.:  distance to target) provides a non-zero gradient.</p>

username: ScottD
cooked: <p>Hi, I was wondering if you had any luck figuring this out? I came across this topic because I was actually trying to implement that same algorithm, and I had the same question.</p>
<p>I tried your suggestion regarding the output layer weights, and didn’t get any improvement in performance. My advantage networks end up with a loss of around 30k mean squared error and I converge around 500mbb exploitability. I can’t figure out where I’m going wrong in replicating the neural net of the paper, but this is one line that I don’t understand. I’ve never seen such a technique discussed anywhere.</p>

username: ElLoboLoco
cooked: <p>Hi, initializing the weights of the output layer to zero seemed to work in my case. You can still look on public implementation of deep cfr such as :</p><aside class="onebox githubblob" data-onebox-src="https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py">
  <header class="source">

      <a href="https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py" target="_blank" rel="noopener nofollow ugc">deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py</a></h4>


    <pre><code class="lang-py"># Copyright 2019 DeepMind Technologies Ltd. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Implements Deep CFR Algorithm.

See https://arxiv.org/abs/1811.00164.

The algorithm defines an `advantage` and `strategy` networks that compute
advantages used to do regret matching across information sets and to approximate
</code></pre>


  This file has been truncated. <a href="https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/pytorch/deep_cfr.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>or</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/EricSteinberger/Deep-CFR">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/EricSteinberger/Deep-CFR" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/177bc08a93b237f184f422bc8a035791b867de083157205dad64a9fd39386eaf/EricSteinberger/Deep-CFR" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/EricSteinberger/Deep-CFR" target="_blank" rel="noopener nofollow ugc">GitHub - EricSteinberger/Deep-CFR: Scalable Implementation of Deep CFR and...</a></h3>

  <p>Scalable Implementation of Deep CFR and Single Deep CFR - GitHub - EricSteinberger/Deep-CFR: Scalable Implementation of Deep CFR and Single Deep CFR</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Hope that helps</p>

username: Alvaro_Budria
cooked: <p>Hi, I can’t seem to find the code initializing the weights in the websites you shared. Could you please indicate the code you used for initialization?</p>