Data Structure coding for GDAS NAS

promach: <p>how to code the following graph structure ?<br>
which kind of data structure is suitable in this case for <strong>both forward inference and backward propagation</strong> ?<br>
Would <a href="https://networkx.org/documentation/stable/tutorial.html#multigraphs" class="inline-onebox" rel="noopener nofollow ugc">Tutorial — NetworkX 3.2.1 documentation</a> or <a href="https://pytorch-geometric.readthedocs.io/en/latest/index.html" class="inline-onebox" rel="noopener nofollow ugc">PyG Documentation — pytorch_geometric documentation</a> or <a href="https://www.dgl.ai/" rel="noopener nofollow ugc">https://www.dgl.ai/</a> be suitable ?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/7/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15_2_480x500.jpeg" alt="image" data-base62-sha1="7WPnx2ArKZsIYbwcKH8COHTzDgx" width="480" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15_2_480x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15_2_720x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37b59ead1a75bf3ef27e1f1aa5ff2b8f5be65b15_2_960x1000.jpeg 2x" data-dominant-color="ECF0EF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">976×1015 285 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/D-X-Y/AutoDL-Projects/issues/99#issuecomment-835713058">
  <header class="source">

      <a href="https://github.com/D-X-Y/AutoDL-Projects/issues/99#issuecomment-835713058" target="_blank" rel="noopener nofollow ugc">github.com/D-X-Y/AutoDL-Projects</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/D-X-Y/AutoDL-Projects/issues/99#issuecomment-835713058" target="_blank" rel="noopener nofollow ugc">Questions about DARTS</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-05-01" data-time="04:47:17" data-timezone="UTC">04:47AM - 01 May 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/buttercutter" target="_blank" rel="noopener nofollow ugc">
          <img alt="buttercutter" src="https://avatars.githubusercontent.com/u/3324659?v=4" class="onebox-avatar-inline" width="20" height="20">
          buttercutter
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          question
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">1. For DARTS complexity analysis, anyone have any idea how to derive the *(k+1)\<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">*k/2* expression ?  Why 2 input nodes ?  How will the calculated value change if graph isomorphism is considered ?  Why "2+3+4+5" learnable edges ?  If there is lack of connection, the paper should not add 1 which does not actually contribute to learnable edges configurations at all ?

2. Why need to train the weights for normal cells and reduction cells **separately** as shown in Figures 4 and 5 below ?

3. How to arrange the nodes such that the NAS search will actually converge with minimum error ?   Note: Not all nodes are connected to each and every other nodes

4. Why is [GDAS](https://arxiv.org/pdf/1910.04465.pdf#page=7) 10 times faster than DARTS ?


![DARTS_normal_reduction_cells](https://user-images.githubusercontent.com/3324659/116770930-e97d4d80-aa79-11eb-8aae-ecf867aabe49.png)

![DARTS_complexity_analysis](https://user-images.githubusercontent.com/3324659/116770928-e2eed600-aa79-11eb-90b9-f00546958b12.png)</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p><a href="https://gist.github.com/promach/b6f526c56e20f029d68e6f9041c3f5c0#file-gdas-py-L106-L107" class="onebox" target="_blank" rel="noopener nofollow ugc">https://gist.github.com/promach/b6f526c56e20f029d68e6f9041c3f5c0#file-gdas-py-L106-L107</a></p>