What is the title type of this ensemble?

Sammo1: <p>i used those functions to merge between two models but i don’t know the type of this ensemble</p>
<pre><code class="lang-auto">class MyEnsemble(nn.Module):
    def __init__(self, modelA, modelB):
        super(MyEnsemble, self).__init__()
        self.modelA = modelA
        self.modelB = modelB
    def forward(self, x):
        x1 = self.modelA(x.clone()) 
        x1 = x1.view(x1.size(0), -1)
        x2 = self.modelB(x)
        x2 = x2.view(x2.size(0), -1)
        x = torch.cat((x1, x2), dim=1)
</code></pre>
<p>will that be Bootstrap Aggregation Ensemble or K-fold Cross-Validation Ensemble or Boosting Ensemble?</p>

ptrblck: <p>I don’t think any of the mentioned techniques are seen in the code example.<br>
If I’m not mistaken the mentioned techniques are:</p>
<ul>
<li>Bootstrap Aggregation Ensemble (bagging): would train weak learners on subsets of the training data and would then create an ensemble for the final prediction</li>
<li>K-fold Cross-Validation Ensemble: I’m not familiar with this ensemble technique, but guess that you would be using k-fold CV to create weak learners again and put them into an ensemble. Note that k-fold CV is usually applied for hyperparameter search.</li>
<li>Boosting Ensemble: would sequentially create weak learners by feeding them with misclassified samples from the previous stage (or by applying a larger weight for misclassified samples).</li>
</ul>
<p>In your example you are only concatenating the outputs of two different models, so I also wouldn’t call it an ensemble without getting a final prediction.</p>

Sammo1: <p>thanks for replying but excuse me what do you mean by predictions … as I searched for how to ensemble two different models and found this way by concatenating the outputs of the two models then taking this merge and training the model with it … if it is not ensemble what is this technique called, please</p>

ptrblck: <aside class="quote no-group" data-username="Sammo1" data-post="3" data-topic="146552">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sammo1/48/46342_2.png" class="avatar"> Sammo1:</div>
<blockquote>
<p>by concatenating the outputs of the two models then taking this merge and training the model with it</p>
</blockquote>
</aside>
<p>Yes, this would be a 2-stage ensemble where the outputs of stage0 are fed to a classifier in stage1 (it could be any trained classifier, a voting classifier etc.).<br>
In your current code snippet you are just concatenating the outputs of two base models without adding any classifier on top, so I wouldn’t call it an ensemble.</p>

ptrblck: <p>I don’t want to nitpick, but I wouldn’t call it an ensemble model, since you are unable to get predictions using your current code snippet and are just concatenating features or different models.<br>
Depending on the training strategy and the classification process you are using, it could be used in an ensemble. Feature concatenation itself is also used in e.g. UNets and I wouldn’t call the internal UNet architecture an ensemble of models.</p>

Sammo1: <p>excuse me and thanks for your time as I’m conflicting in what I did. if you can help me if this way called merging model, not ensemble model as I searched but need to confirm … the basic idea is to get a model from two models to use it in finding the features of the images using PyTorch so I thought that deleting the classifier from both then merge is the right way … is there any way to use ensemble and what I did in the code called merging not ensemble?</p>
<p>appreciate your reply</p>