Using model.pth pytorch to predict image

Itha_LItha: <p>Hello,<br>
I am a beginner in neural networks and I am trying a siamese neural network using Pytorch.</p>
<p>I tried someone’s project that was published on github, but the post only gave me the stage of making a model with the .pth format</p>
<p><strong>how can I make the model can predict the images that I put into the system?</strong></p>
<p>can anyone help me?</p>
<p>please</p>

ptrblck: <p>Once you have the model and load its <code>state_dict</code>, you should set it to evaluation mode (to use the running stats in batchnorm layers and disable dropout). Afterwards, you would have to use the same preprocessing pipeline, which was used during training to get reasonable results (e.g. normalizing with the same mean and stddev). Once you’ve created the input tensors, you could pass them to your model to get the output. Here is a small dummy code snippet:</p>
<pre><code class="lang-python"># Create the model
model = MyModel()

# Load state_dict
model.load_state_dict(torch.load(...))

# Create the preprocessing transformation here
transform = transforms.ToTensor()

# load your image(s)
img = Image.open(...)

# Transform
input = transform(img)

# unsqueeze batch dimension, in case you are dealing with a single image
input = input.unsquueeze(0)

# Set model to eval
model.eval()

# Get prediction
output = model(input)
</code></pre>

Itha_LItha: <p>thanks for the help, everything went smoothly until i found this error</p>
<p>can you help me in understanding the error and how can I fix it?</p>
<p>please<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/7/e7c70b5cdc72f39a8480c7b58dbe6de17fd29507.png" alt="Screen Shot 2020-03-13 at 11.31.55" data-base62-sha1="x4oKgMYCO9ke4BDgYiYaj6JF4mr" width="575" height="358"></p>

ptrblck: <p>Your model apparently needs a second input, which is missing in your call. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Itha_LItha: <p>so how can I use 2 image inputs?</p>
<p>is it enough to make it like this?</p>
<p>sorry if I asked a very basic question.</p>
<pre><code class="lang-auto">input = transforms(img1)
input = transforms(img2)
input = (img1,img2)
# unsqueeze batch dimension, in case you are dealing with a single image
input = input.unsqueeze(0)

# Set model to eval
model.eval()

# Get prediction
output = model(input)
</code></pre>

ptrblck: <p>Just pass two arguments to the model:</p>
<pre><code class="lang-python">output = model(x1, x2)
</code></pre>

Itha_LItha: <p>i’ve been tried, and i found this error, i feel so confused. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_frowning_face.png?v=12" title=":slightly_frowning_face:" class="emoji" alt=":slightly_frowning_face:" loading="lazy" width="20" height="20"><br>
the error said that input must be transforms to tensor?<br>
but i think the image has been transformed to tensor by <code>input=transforms(x1)</code><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/2/8273808cd8e0b6030fec05688ad3170b20a41aba.png" data-download-href="https://discuss.pytorch.org/uploads/default/8273808cd8e0b6030fec05688ad3170b20a41aba" title="Screen Shot 2020-03-13 at 15.41.45"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/8/2/8273808cd8e0b6030fec05688ad3170b20a41aba_2_690x192.png" alt="Screen Shot 2020-03-13 at 15.41.45" data-base62-sha1="iC1x4hoUe7ooGCMcz8B0IuT9oNk" width="690" height="192" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/8/2/8273808cd8e0b6030fec05688ad3170b20a41aba_2_690x192.png, https://discuss.pytorch.org/uploads/default/optimized/3X/8/2/8273808cd8e0b6030fec05688ad3170b20a41aba_2_1035x288.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/8/2/8273808cd8e0b6030fec05688ad3170b20a41aba.png 2x" data-dominant-color="ECEFED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-03-13 at 15.41.45</span><span class="informations">1355×379 136 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>You would have to transform the images (<code>x1</code> and <code>x2</code>) and pass the tensors to the model.<br>
You are currently reassigning <code>image</code> to both transformed tensors.<br>
This should work:</p>
<pre><code class="lang-python">x1 = transforms(x1)
x2 = transforms(x2)
output = model(x1, x2)
</code></pre>

Itha_LItha: <p>when i try that, then error comes again that said expected image is 4dimension, but my data is 3dimension.<br>
expected is [64,1,10,10] but my dataset is [105, 105, 1]<br>
and i still want to use my own dataset. i search how to solve it and some page said the image must unsequeeze before.</p>
<p>and i see, i dont unsequeeze x1 and x2<br>
but when x1 and x1 pass the unsequezeed, the same error comes again like this <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/d/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3.png" data-download-href="https://discuss.pytorch.org/uploads/default/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3" title="Screen Shot 2020-03-13 at 16.53.14"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/d/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3_2_690x129.png" alt="Screen Shot 2020-03-13 at 16.53.14" data-base62-sha1="xVTRPoH5A9t5V1TAvSubDPwbwwH" width="690" height="129" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/d/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3_2_690x129.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/d/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3_2_1035x193.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/d/edd34cbf125f41d51d6f06f0a9be9bc3f8d002f3_2_1380x258.png 2x" data-dominant-color="F2F5F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-03-13 at 16.53.14</span><span class="informations">1987×374 144 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

Itha_LItha: <p>this is my model</p>
<pre><code class="lang-auto">class Siamese(nn.Module):

    def __init__(self):
        super(Siamese, self).__init__()
        self.conv = nn.Sequential(
            nn.Conv2d(1, 64, 10),  # 64@96*96
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2),  # 64@48*48
            nn.Conv2d(64, 128, 7),
            nn.ReLU(),    # 128@42*42
            nn.MaxPool2d(2),   # 128@21*21
            nn.Conv2d(128, 128, 4),
            nn.ReLU(), # 128@18*18
            nn.MaxPool2d(2), # 128@9*9
            nn.Conv2d(128, 256, 4),
            nn.ReLU(),   # 256@6*6
        )
        self.liner = nn.Sequential(nn.Linear(9216, 4096), nn.Sigmoid())
        self.out = nn.Linear(4096, 1)

    def forward_one(self, x):
        x = self.conv(x)
        x = x.view(x.size()[0], -1)
        x = self.liner(x)
        return x

    def forward(self, x1, x2):
        out1 = self.forward_one(x1)
        out2 = self.forward_one(x2)
        dis = torch.abs(out1 - out2)
        out = self.out(dis)
    
        return out

</code></pre>
<p>when I want to test the resulting model, an error appears stating the requested input is 4 but entering 3 dimensional input.</p>
<p>I think there is a problem in the image that I will test</p>
<pre><code class="lang-auto">from PIL import Image
x1 = Image.open('t.PNG')
x2 = Image.open('g.PNG')
</code></pre>
<p>the error is said that I put an image with a size of 105x105, even though the test image that I input is not 105.</p>
<p>but i was wrong, maybe the error tell me about my data that i use to training.<br>
An image with a size of 105x105 is an image that I used for the train process to produce a siamese neural network model.</p>
<p>I follow this post in the siamese process (<a href="https://github.com/Ameyapores/one_shot_learning.git" rel="nofollow noopener">https://github.com/Ameyapores/one_shot_learning.git</a>)</p>
<p>please help me,</p>

simaiden: <p>That’s an error in your code in line 79,80, with <code>input</code> variable. Altought you shouldn’t do a squeeze before go as an input of the model, because model expects a batch dimension</p>
<p>edit: i didn’t notice that it’s an unsqueeze instead of squeeze</p>

Itha_LItha: <p>i’ve been tried, but still show the same error</p>

simaiden: <p>please post the error message</p>

Itha_LItha: <p>this is the error, help me</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf.png" data-download-href="https://discuss.pytorch.org/uploads/default/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf" title="Screen Shot 2020-03-13 at 21.45.16"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf_2_690x132.png" alt="Screen Shot 2020-03-13 at 21.45.16" data-base62-sha1="pArYzijCaXrlBaWabxSG7yBeoW3" width="690" height="132" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf_2_690x132.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf_2_1035x198.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/3/b3552f8ff0334cc53a96b39f7a3aa3073e1a8acf_2_1380x264.png 2x" data-dominant-color="EFF3F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-03-13 at 21.45.16</span><span class="informations">1992×383 157 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

simaiden: <p>Add this lines after transforms</p>
<pre><code class="lang-auto">
x1 = torch.stack([x1])
x2 = torch.stack([x2])
</code></pre>
<p>It’s a simple way to add the batch dimension with 1 sample</p>
<p>It’s the same with unsqueeze(0) , I guess that your error was with the lines 79,80 using a wrong variable</p>

Itha_LItha: <p>Wohooo… its works. Thankyou so much<br>
but when i print the output and just given this score,<br>
How do I tell the label of the score?</p>
<p>This neural network siamese model will predict the image, when the image is the same it will tell the type of the image. for example with a score of -0.005 it is “Table”. correct me if im wrong. hehehe</p>
<p>how to make the output like that? not just giving it a score<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/5/a5c3939bc64ea54a1c278f9db3efbf008fca041a.png" data-download-href="https://discuss.pytorch.org/uploads/default/a5c3939bc64ea54a1c278f9db3efbf008fca041a" title="Screen Shot 2020-03-13 at 21.53.05"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/5/a5c3939bc64ea54a1c278f9db3efbf008fca041a_2_690x408.png" alt="Screen Shot 2020-03-13 at 21.53.05" data-base62-sha1="nEpNOPWRIV3pLQDNtDd50uCARWW" width="690" height="408" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/5/a5c3939bc64ea54a1c278f9db3efbf008fca041a_2_690x408.png, https://discuss.pytorch.org/uploads/default/original/3X/a/5/a5c3939bc64ea54a1c278f9db3efbf008fca041a.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/5/a5c3939bc64ea54a1c278f9db3efbf008fca041a.png 2x" data-dominant-color="EBEDEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-03-13 at 21.53.05</span><span class="informations">945×560 103 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

simaiden: <p>Siamese network does not predict the label but the similarity between the inputs. Usually returns a 1 if the images are similar and a -1 if not, but depends on how you trained the network</p>

Itha_LItha: <p>hmm, oke… thankyou.</p>
<p>but how i can show does it 0 or 1?</p>

simaiden: <p>I looked the training code and it uses BCEWithLogitsLoss so is using binary cross entropy plus a sigmoid. To get the output between 0 and 1 you have to use a sigmoid like:</p>
<p><code>prob = nn.Sigmoid()(output)</code></p>

Itha_LItha: <p>this is the sigmoid model that I used before<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/3/03fa11dbf4cdf751f1839138caf693e91a7339a2.png" alt="Screen Shot 2020-03-13 at 22.07.50" data-base62-sha1="zbcFJOn2pLyVEPW6icCUQND2y6" width="637" height="46"><br>
mean I have to retrain?</p>