How could I define layer contains parallel custom layers

deponce: <p>Hey,</p>
<p>I defined a custom convolution let call it “custom_conv2d”, which takes n channels input and outputs fix 8 channels.</p>
<p>How do I define a layer “custom_conv2d_layer()”, which contains several of my custom conv2d layers in parallel?</p>
<p>For example, “custom_conv2d_layer()”, take 64 input channels, and couput 128 channels, that means “custom_conv2d_layer()” contain 128/8 = 16 “custom_conv2d” layers parallelly.</p>
<p>Thanks in advance!</p>

KFrank: <p>Hi Deponce!</p>
<aside class="quote no-group" data-username="deponce" data-post="1" data-topic="134458" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ee7513/48.png" class="avatar"> deponce:</div>
<blockquote>
<p>How do I define a layer “custom_conv2d_layer()”, which contains several of my custom conv2d layers in parallel?</p>
</blockquote>
</aside>
<p>I don’t fully understand your use case, but take a look at <a href="https://pytorch.org/docs/1.9.1/generated/torch.nn.Module.html" rel="noopener nofollow ugc">torch.nn.Module</a>.</p>
<p>You could package your “custom_conv2d” as a <code>Module</code>.  (You don’t<br>
have to, but you could.)  Then you could define “custom_cond2d_layer”<br>
also as a <code>Module</code>.  Your “custom_cond2d_layer” <code>Module</code> could contain<br>
16 “custom_cond2d” <code>Module</code>s as “properties.”</p>
<p>Your “custom_cond2d_layer” <code>Module</code>’s <code>forward()</code> function would apply<br>
each “custom_cond2d” to the appropriate part of the input and then, for<br>
example, <code>cat()</code> together the results of the 16 “custom_cond2ds” to<br>
form the output returned by <code>forward()</code>.</p>
<p>Best.</p>
<p>K. Frank</p>

deponce: <p>Thanks for your reply KFrank!<br>
Yeah, I thought you fully understand my question. But my question is how could I implement such ‘custom_cond2d_layer’ Module, there are 16 'custom_conv2d’s, I could define them manually, but what if there are more than 1000 'custom_conv2d’s? Is there a method to automatically initiate the 'custom_conv2d’s by a for loop?</p>
<p>Thanks in advance!</p>

KFrank: <p>Hi Deponce!</p>
<aside class="quote no-group" data-username="deponce" data-post="3" data-topic="134458" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ee7513/48.png" class="avatar"> deponce:</div>
<blockquote>
<p>what if there are more than 1000 'custom_conv2d’s? Is there a method to automatically initiate the 'custom_conv2d’s by a for loop?</p>
</blockquote>
</aside>
<p>Yes, you can write a method that will “automatically” create and<br>
initialize a large number of <code>custom_conv2d</code>s with a for loop.</p>
<p>When you define your <code>custom_cond2d_layer</code> you will be defining a<br>
new class (that is a subclass of <code>Module</code>).  You will define for this<br>
class an <code>__init__()</code> function that gets called every time you create<br>
a new <em>instance</em> of the <code>custom_cond2d_layer</code> class.  This function<br>
initializes the new instance.  (In other languages such a function is<br>
often called the class’s <em>constructor.</em>)</p>
<p>You have to write the code, but this <code>__init__()</code> can run a loop that<br>
creates (<em>instantiates</em>) 1000 (or however many) instances of your<br>
<code>custom_conv2d</code> class and puts (references to) those instances into<br>
a <code>list</code> (or other container) that is a <em>property</em> of the new instance of<br>
your <code>custom_conv2d_layer</code> class.</p>
<p>(During the forward pass, the <code>forward()</code> function of your<br>
<code>custom_conv2d_layer</code> could iterate over the <code>list</code> (or other container)<br>
of <code>custom_conv2d</code>s, applying them to the appropriate “chunks” of the<br>
input, and then <code>cat()</code> together those results to create the final result<br>
returned by <code>forward()</code>.)</p>
<p>Best.</p>
<p>K. Frank</p>