Provide weights before concatenating the outputs of different branches of a model

username: sidhant_Naveria
cooked: <p>hi,<br>
I have a model with 2 branches. one branch is for the left side of the image and 2nd branch is for the right side. I am concatenating the features from both branches and applying those features to the classifier layer. In my case, the left side image has more patterns as compared to the right side image. I want to add weights before concatenation so that model will be little bias towards the left branch. kindly let me know how I can do that</p>

username: ptrblck
cooked: <p>If you want to scale the activations before concatenating them, you could simply multiply them with a <code>weight</code> tensor.<br>
Could you explain, how the weighting should be performed, in case I misunderstood the question?</p>

username: sidhant_Naveria
cooked: <p>weights should perform similar to the weights for multi modality models. what i require is that at the classification layer after concatenation, should know that layer has to focus more on the features extracted from the left side branch</p>

username: ptrblck
cooked: <p>You could try to scale the activations from the left branch to try to force the model to focus more on this branch. However, this would mean that the magnitude of the input features would directly correspond to the importance of these features.<br>
Do you know any publications for this claim?</p>

username: sidhant_Naveria
cooked: <p>by scale the activation you mean, just multiply a scalar value to the output of left branch and then concatenate it with the right branch.No i haven’t about anything which states left image will the more features, it just i come up to the conclusion after training left image and right image separately, which showed left image get trained faster with higher accuracy then the right image.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="sidhant_Naveria" data-post="5" data-topic="86152">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sidhant_naveria/48/25813_2.png" class="avatar"> sidhant_Naveria:</div>
<blockquote>
<p>by scale the activation you mean, just multiply a scalar value to the output of left branch and then concatenate it with the right branch.</p>
</blockquote>
</aside>
<p>Yes, I understood that would be your use case given “provide weights before concatenation”.<br>
If that’s not the case, please provide more information about the method you would like to use.</p>
<aside class="quote no-group" data-username="sidhant_Naveria" data-post="5" data-topic="86152">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sidhant_naveria/48/25813_2.png" class="avatar"> sidhant_Naveria:</div>
<blockquote>
<p>No i haven’t about anything which states left image will the more features, it just i come up to the conclusion after training left image and right image separately</p>
</blockquote>
</aside>
<p>I was asking about the general assumption that scaling a feature (or however your weighting should look like) corresponds to a higher “feature importance”.</p>