While Upsampling  I don't get the need size 

mohamed_mzoughi: <p>Hello, first of all, This is my code</p>
<h1><a name="coding-utf-8-1" class="anchor" href="#coding-utf-8-1"></a>-<em>- coding: utf-8 -</em>-</h1>
<p>“”"fork-of-siamese-network-family-relationship (1).ipynb</p>
<p>Automatically generated by Colaboratory.</p>
<p>Original file is located at<br>
<a href="https://colab.research.google.com/drive/18y6TBHtgI_heaKNI3EwpDSHCCr-QIhpM" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a><br>
“”"</p>
<p>from google.colab import drive<br>
drive.mount(‘/content/drive’)</p>
<p>!mkdir input<br>
! cp /content/drive/MyDrive/recognizing-faces-in-the-wild.zip /content/input</p>
<p>!pip install validators matplotlib<br>
!pip install timm==0.5.4</p>
<h1><a name="commented-out-ipython-magic-to-ensure-python-compatibility-2" class="anchor" href="#commented-out-ipython-magic-to-ensure-python-compatibility-2"></a>Commented out IPython magic to ensure Python compatibility.</h1>
<p>import torch<br>
from PIL import Image<br>
import torchvision.transforms as transforms<br>
import numpy as np<br>
import json<br>
import requests<br>
import matplotlib.pyplot as plt<br>
import warnings<br>
warnings.filterwarnings(‘ignore’)</p>
<h1><a name="matplotlib-inline-3" class="anchor" href="#matplotlib-inline-3"></a>%matplotlib inline</h1>
<p>device = torch.device(“cuda”) if torch.cuda.is_available() else torch.device(“cpu”)<br>
print(f’Using {device} for inference’)</p>
<p>!unzip /content/input/recognizing-faces-in-the-wild.zip -d /content/input</p>
<p>!mkdir train<br>
!unzip /content/input/train.zip -d ./input/train</p>
<p>import torch<br>
import torch.nn as nn<br>
import numpy as np<br>
import pandas as pd</p>
<p>from glob import glob<br>
from collections import defaultdict</p>
<p>“”“## Test”“”</p>
<p>import torchvision<br>
from random import choice<br>
from torch.utils.data import Dataset<br>
from PIL import Image<br>
class KinDataset(Dataset):<br>
def <strong>init</strong>(self, relations, person_to_images_map, transform=None):<br>
self.relations = relations # 가능한 모든 부모 - 자식 관계를 모은 csv 파일<br>
self.transform = transform<br>
self.person_to_images_map = person_to_images_map # (가족 id : 사람데이터) 형태의 데이터<br>
self.ppl = list(person_to_images_map.keys()) # 사람 리스트</p>
<pre><code>def __len__(self):
    return len(self.relations)*2

def __getitem__(self, idx):

    if (idx%2==0): #Positive samples
        p1, p2 = self.relations[idx//2]
        label = 1
    else:          #Negative samples
        while True:
            p1 = choice(self.ppl) # 사람 리스트에서 하나 뽑기
            p2 = choice(self.ppl) # 사람 리스트에서 하나 뽑기
            if p1 != p2 and (p1, p2) not in self.relations and (p2, p1) not in self.relations:
                # 만약 둘이 같은 사람이 아니고 가족 관계가 아닐 경우
                break
        label = 0

    # 고른 사진의 경로에서 이미지 불러오기
    path1, path2 = choice(self.person_to_images_map[p1]), choice(self.person_to_images_map[p2])
    img1, img2 = Image.open(path1), Image.open(path2)

    if self.transform:
        img1, img2 = self.transform(img1), self.transform(img2)
    imgs=torch.cat((img1,img2))

    return imgs, label
</code></pre>
<p>print(“Prepare data…”)<br>
train_file_path = “/content/input/train_relationships.csv”<br>
train_folders_path = “./input/train/”<br>
val_famillies = “F09”</p>
<p>all_images = glob(train_folders_path + “<em>/</em>/*.jpg”)</p>
<p>train_images = [x for x in all_images if val_famillies not in x]<br>
val_images = [x for x in all_images if val_famillies in x]</p>
<p>train_person_to_images_map = defaultdict(list)</p>
<p>ppl = [x.split(“/”)[-3] + “/” + x.split(“/”)[-2] for x in all_images]</p>
<p>for x in train_images:<br>
train_person_to_images_map[x.split(“/”)[-3] + “/” + x.split(“/”)[-2]].append(x)</p>
<p>val_person_to_images_map = defaultdict(list)</p>
<p>for x in val_images:<br>
val_person_to_images_map[x.split(“/”)[-3] + “/” + x.split(“/”)[-2]].append(x)</p>
<p>relationships = pd.read_csv(train_file_path)<br>
relationships = list(zip(relationships.p1.values, relationships.p2.values))<br>
relationships = [x for x in relationships if x[0] in ppl and x[1] in ppl]</p>
<p>train_relations = [x for x in relationships if val_famillies not in x[0]]<br>
val_relations  = [x for x in relationships if val_famillies in x[0]]</p>
<p>from torch.utils.data import DataLoader<br>
from torchvision import transforms</p>
<p>train_transform = transforms.Compose([<br>
transforms.Resize(160),<br>
transforms.Grayscale(num_output_channels=3),<br>
transforms.RandomHorizontalFlip(),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.5, 0.5, 0.5],<br>
std=[0.5, 0.5, 0.5])<br>
])<br>
val_transform = transforms.Compose([<br>
transforms.Resize(160),<br>
transforms.Grayscale(num_output_channels=3),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.5, 0.5, 0.5],<br>
std=[0.5, 0.5, 0.5])<br>
])</p>
<p>trainset = KinDataset(train_relations, train_person_to_images_map, train_transform)<br>
valset = KinDataset(val_relations, val_person_to_images_map, val_transform)</p>
<p>trainloader = DataLoader(trainset, batch_size=200, shuffle=True)<br>
valloader = DataLoader(valset, batch_size=200, shuffle=False)</p>
<p>“”"# Model part</p>
<h2><a name="siamesenet-4" class="anchor" href="#siamesenet-4"></a><img src="https://discuss.pytorch.org/images/emoji/apple/desktop_computer.png?v=12" title=":desktop_computer:" class="emoji" alt=":desktop_computer:" loading="lazy" width="20" height="20"> SiameseNet</h2>
<p>“”"</p>
<p>import torch<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import torchvision.models</p>
<p>class BasicConv2d(nn.Module):<br>
“”"<br>
basic convoluation model<br>
“”"</p>
<pre><code>def __init__(self, in_planes, out_planes, kernel_size, stride, padding=0):
    super().__init__()
    self.conv = nn.Conv2d(
        in_planes, out_planes,
        kernel_size=kernel_size, stride=stride,
        padding=padding, bias=False
    )  # verify bias false
    self.bn = nn.BatchNorm2d(
        out_planes,
        eps=0.001,  # value found in tensorflow
        momentum=0.1,  # default pytorch value
        affine=True
    )
    self.relu = nn.ReLU(inplace=False)

def forward(self, x):
    x = self.conv(x)
    x = self.bn(x)
    x = self.relu(x)
    return x
</code></pre>
<p>class res_unit(nn.Module):<br>
“”"<br>
this is the attention module before Residual structure<br>
“”"<br>
def <strong>init</strong>(self,channel,up_size = None):<br>
“”"</p>
<pre><code>    :param channel: channels of input feature map
    :param up_size: upsample size
    """
    super(res_unit,self).__init__()
    self.pool = nn.MaxPool2d(2,2)
    self.conv = nn.Conv2d(channel,channel,3,padding=1)
    if up_size == None:
        self.upsample = nn.Upsample(scale_factor=2,mode='bilinear',align_corners=False)
    else:
        self.upsample = nn.Upsample(size=(up_size,up_size), mode='bilinear', align_corners=False)
    self.sigmoid = nn.Sigmoid()
def forward(self,x):
    identity = x
    print("Intial Size",x.size(),identity.size())
    x = self.pool(x)
    print("Size After pooling",x.size(),identity.size())
    x = self.conv(x)
    print("Size After Convolution",x.size(),identity.size())
    x = self.upsample(x)
    print("Size after Upsampling",x.size(),identity.size())
    x = self.sigmoid(x)
    x = torch.mul(identity,x)
    return x
</code></pre>
<p>class basenet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.base = _attenNet()<br>
self.fea = nn.Sequential(<br>
nn.Dropout(),<br>
nn.Linear(128 * 9 * 9, 512),<br>
nn.ReLU(inplace=True),<br>
nn.Dropout(),<br>
nn.Linear(512, 128),<br>
)</p>
<pre><code>def forward(self,x):
    x = self.base(x)
    x = x.view(-1, 9 * 9 * 128)
    x = self.fea(x)
    return x
</code></pre>
<p>class _atten(nn.Module):<br>
“”"<br>
the attention Module in <br>
“”"<br>
def <strong>init</strong>(self):<br>
super(_atten,self).<strong>init</strong>()<br>
self.conv1 = nn.Conv2d(6,32,5)<br>
self.conv2 = nn.Conv2d(32,64,5)<br>
self.at1 = res_unit(32)<br>
self.at2 = res_unit(64)<br>
self.pool = nn.MaxPool2d(2,2)<br>
self.bn1 = nn.BatchNorm2d(32)<br>
self.bn2 = nn.BatchNorm2d(64)<br>
<span class="hashtag-raw">#self</span>.fc1 = nn.Linear((9<em>9</em>128),512)<br>
<span class="hashtag-raw">#self</span>.dp  = nn.Dropout()<br>
<span class="hashtag-raw">#self</span>.fc2 = nn.Linear(512,2)</p>
<pre><code>def forward(self,x):
    """
    :param x: 6x64x64
    :return:
    """
    x = self.conv1(x)
    identity1 = x
    x = self.at1(x)
    x = identity1+x
    x = self.bn1(x)
    x = self.pool(F.relu(x))

    x = self.conv2(x)
    identity2 = x
    x = self.at2(x)
    x = identity2 + x
    x = self.bn2(x)
    x = self.pool(F.relu((x)))


    # x = x.view(-1, 9*9*128)
    # x = F.relu(self.fc1(x))
    # x = self.dp(x)
    # x = self.fc2(x)
    return x
</code></pre>
<p>class _attenNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.base = _atten()<br>
self.conv3 = nn.Conv2d(64, 128, 5)<br>
self.at3 = res_unit(128,up_size=9)<br>
self.bn3 = nn.BatchNorm2d(128)</p>
<pre><code>def forward(self,x):
    x = self.base(x)
    x = self.conv3(x)
    identity3 = x
    x = self.at3(x)
    x = identity3 + x
    x = self.bn3(x)
    x = F.relu(x)
    return x
</code></pre>
<p>class res_addNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.conv3 = nn.Conv2d(64, 128, 5)<br>
self.at3 = res_unit(128, up_size=9)<br>
self.bn3 = nn.BatchNorm2d(128)</p>
<pre><code>def forward(self, x):
    x = self.conv3(x)
    identity3 = x
    x = self.at3(x)
    x = identity3 + x
    x = self.bn3(x)
    x = F.relu(x)
    return x
</code></pre>
<p>class each_brach(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.conv3 = nn.Conv2d(64, 128, 5)<br>
self.at3 = res_unit(128, up_size=9)<br>
self.bn3 = nn.BatchNorm2d(128)</p>
<pre><code>    self.fc = nn.Sequential(
        nn.Linear((9 * 9 * 128), 512),
        nn.ReLU(),
        nn.Dropout(),
        nn.Linear(512, 2)
    )
def forward(self, x):
    x = self.conv3(x)
    identity3 = x
    x = self.at3(x)
    x = identity3 + x
    x = self.bn3(x)
    x = F.relu(x)
    x = x.view(-1, 9*9*128)
    x = self.fc(x)
    return x
</code></pre>
<p>class My_Network(nn.Module):<br>
“”"<br>
concatenate 4x2 output + add loss layer<br>
“”"<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
self.base = _atten()</p>
<pre><code>    self.fd_fc = each_brach()

def forward(self, x):
    x = self.base(x)
    x = self.fd_fc(x)
    print("Done")
    return x
</code></pre>
<p>def train():<br>
net.train() # 학습 시작<br>
train_loss = 0.0<br>
running_loss = 0.0<br>
running_corrects = 0</p>
<pre><code>for i, batch in enumerate(trainloader):
    # 로더에서 하나씩 꺼내오며
    optimizer.zero_grad()

    img, label = batch
    img, label = img.to(device), label.float().view(-1,1).to(device)
    output = net(img)
    preds = output&gt;0.5
    loss = criterion(output, label)
    loss.backward()
    optimizer.step()

    train_loss += loss.item()
    running_loss += loss.item()
    running_corrects += torch.sum(preds == (label&gt;0.5))

    step = 100
    if i % step == step-1:
        print(' [{} - {:.2f}%],\ttrain loss: {:.5}'.format(epoch+1, 100*(i+1)/len(trainloader), running_loss/step/200))
        running_loss = 0

train_loss /= len(trainset)
running_corrects = running_corrects.item()/len(trainset)
print('[{}], \ttrain loss: {:.5}\tacc: {:.5}'.format(epoch+1, train_loss, running_corrects))
return train_loss, running_corrects
</code></pre>
<p>def validate():<br>
net.eval()<br>
val_loss = 0.0<br>
running_corrects = 0</p>
<pre><code>for batch in valloader:
    img ,label = batch
    img, label = img.to(device), label.float().view(-1,1).to(device)
    with torch.no_grad():
        output = net(img)
        preds = output&gt;0.5
        loss = criterion(output, label)

    val_loss += loss.item()
    running_corrects += torch.sum(preds == (label&gt;0.5))

val_loss /= len(valset)
running_corrects = running_corrects.item()/len(valset)
print('[{}], \tval loss: {:.5}\tacc: {:.5}'.format(epoch+1, val_loss, running_corrects))

return val_loss, running_corrects
</code></pre>
<p>“”“## <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=12" title=":grinning:" class="emoji" alt=":grinning:" loading="lazy" width="20" height="20"> Train”“”</p>
<p>print(“Initialize network…”)<br>
device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)<br>
net=My_Network().to(device)<br>
lr = 1e-3</p>
<p>criterion = nn.BCEWithLogitsLoss()<br>
optimizer = torch.optim.Adam(net.parameters(), lr=lr)</p>
<p>from torch.optim.lr_scheduler import ReduceLROnPlateau<br>
scheduler = ReduceLROnPlateau(optimizer, patience=10)</p>
<p>print(“Start training…”)<br>
num_epoch = 100</p>
<p>best_val_loss = 1000<br>
best_epoch = 0</p>
<p>history = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
accuracy = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for epoch in range(num_epoch):<br>
train_loss, train_acc = train()<br>
val_loss, val_acc = validate()<br>
history.append((train_loss, val_loss))<br>
accuracy.append((train_acc,val_acc))<br>
scheduler.step(val_loss)</p>
<pre><code>if val_loss &lt; best_val_loss:
    best_val_loss = val_loss
    best_epoch = epoch
    torch.torch.save(net.state_dict(), 'net_checkpoint.pth')
</code></pre>
<h2><a name="torchsavenetstate_dictnet_full_trainingpth-i-add-few-print-statement-for-debugging-in-which-i-remarked-that-upsampling-is-causing-the-error-here-is-the-result-of-the-debugging-start-training-intial-size-torchsize200-32-156-156-torchsize200-32-156-156-size-after-pooling-torchsize200-32-78-78-torchsize200-32-156-156-size-after-convolution-torchsize200-32-78-78-torchsize200-32-156-156-size-after-upsampling-torchsize200-32-156-156-torchsize200-32-156-156-intial-size-torchsize200-64-74-74-torchsize200-64-74-74-size-after-pooling-torchsize200-64-37-37-torchsize200-64-74-74-size-after-convolution-torchsize200-64-37-37-torchsize200-64-74-74-size-after-upsampling-torchsize200-64-74-74-torchsize200-64-74-74-intial-size-torchsize200-128-33-33-torchsize200-128-33-33-size-after-pooling-torchsize200-128-16-16-torchsize200-128-33-33-size-after-convolution-torchsize200-128-16-16-torchsize200-128-33-33-size-after-upsampling-torchsize200-128-9-9-torchsize200-128-33-33-5" class="anchor" href="#torchsavenetstate_dictnet_full_trainingpth-i-add-few-print-statement-for-debugging-in-which-i-remarked-that-upsampling-is-causing-the-error-here-is-the-result-of-the-debugging-start-training-intial-size-torchsize200-32-156-156-torchsize200-32-156-156-size-after-pooling-torchsize200-32-78-78-torchsize200-32-156-156-size-after-convolution-torchsize200-32-78-78-torchsize200-32-156-156-size-after-upsampling-torchsize200-32-156-156-torchsize200-32-156-156-intial-size-torchsize200-64-74-74-torchsize200-64-74-74-size-after-pooling-torchsize200-64-37-37-torchsize200-64-74-74-size-after-convolution-torchsize200-64-37-37-torchsize200-64-74-74-size-after-upsampling-torchsize200-64-74-74-torchsize200-64-74-74-intial-size-torchsize200-128-33-33-torchsize200-128-33-33-size-after-pooling-torchsize200-128-16-16-torchsize200-128-33-33-size-after-convolution-torchsize200-128-16-16-torchsize200-128-33-33-size-after-upsampling-torchsize200-128-9-9-torchsize200-128-33-33-5"></a>torch.save(net.state_dict(),‘net_full_training.pth’)<br>
I add few print statement for debugging in which I remarked that upsampling is causing the error  here is the result of the debugging :<br>
Start training…<br>
Intial Size torch.Size([200, 32, 156, 156]) torch.Size([200, 32, 156, 156])<br>
Size After pooling torch.Size([200, 32, 78, 78]) torch.Size([200, 32, 156, 156])<br>
Size After Convolution torch.Size([200, 32, 78, 78]) torch.Size([200, 32, 156, 156])<br>
Size after Upsampling torch.Size([200, 32, 156, 156]) torch.Size([200, 32, 156, 156])<br>
Intial Size torch.Size([200, 64, 74, 74]) torch.Size([200, 64, 74, 74])<br>
Size After pooling torch.Size([200, 64, 37, 37]) torch.Size([200, 64, 74, 74])<br>
Size After Convolution torch.Size([200, 64, 37, 37]) torch.Size([200, 64, 74, 74])<br>
Size after Upsampling torch.Size([200, 64, 74, 74]) torch.Size([200, 64, 74, 74])<br>
Intial Size torch.Size([200, 128, 33, 33]) torch.Size([200, 128, 33, 33])<br>
Size After pooling torch.Size([200, 128, 16, 16]) torch.Size([200, 128, 33, 33])<br>
Size After Convolution torch.Size([200, 128, 16, 16]) torch.Size([200, 128, 33, 33])<br>
Size after Upsampling torch.Size([200, 128, 9, 9]) torch.Size([200, 128, 33, 33])</h2>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in &lt;cell line: 9&gt;()<br>
8 accuracy = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
9 for epoch in range(num_epoch):<br>
—&gt; 10     train_loss, train_acc = train()<br>
11     val_loss, val_acc = validate()<br>
12     history.append((train_loss, val_loss))</p>
<p>9 frames<br>
 in forward(self, x)<br>
61         print(“Size after Upsampling”,x.size(),identity.size())<br>
62         x = self.sigmoid(x)<br>
—&gt; 63         x = torch.mul(identity,x)<br>
64         return x<br>
65</p>
<p>RuntimeError: The size of tensor a (33) must match the size of tensor b (9) at non-singleton dimension 3</p>

Gillian: <p>Hello,<br>
I haven’t checked what your code is doing exactly, but the prints that you have seems to be explanable by the following lines.</p>
<p>You have those two:</p>
<pre><code class="lang-auto">  self.at1 = res_unit(32)
  self.at2 = res_unit(64)
</code></pre>
<p>if i’m not wrong, they outpout your 2 first print blocks. You don’t defined a up_size, so your code execute since you wrote that when up_size is None, the scale factor is 2.</p>
<p>Further you call res_unit with a dedicated size:</p>
<p><code>self.at3 = res_unit(128,**up_size=9**)</code></p>
<p>That seems to fix your upsampling step size at 9, which trigger the error.</p>