New to PyTorch, having trouble making predictions once data is loaded using Data Loader

username: falconzfan4ever
cooked: <p>I’m completely new to PyTorch (have previously used tensor flow) and I’m stuck on something I’m working on. I’ve been tasked with using a pretrained model to extract the features from application documents and then compute similarity scores to identify duplicates. I have all of the pdf’s converted to .jpg’s, and I’ve loaded the pretrained model and modified the last layer to extract features.</p>
<p>The folder structure is like this:<br>
root<br>
Application 1<br>
image 1<br>
image 2…<br>
Application 2<br>
image 1<br>
Image 2…</p>
<p>What I’m trying to do is extract features from the images in every sub-directory and calculate the euclidian distance between them and output a similarity matrix. Where I’m having an issue, and this may seem really basic, is actually making the predictions once the data is loaded. Below is the code I have so far, any help would be greatly appreciated.</p>
<p>def get_pretrained_model_notop(model_name): <span class="hashtag">#pull</span> the model and change last layer<br>
pretrained_model = model_name(pretrained=True) <span class="hashtag">#downloads</span> pretrained model weights<br>
for param in pretrained_model.parameters():<br>
param.requires_grad = False <span class="hashtag">#freezes</span> layers<br>
pretrained_model = nn.Sequential(*list(pretrained_model.children())[:-1]) <span class="hashtag">#drops</span> final layer, because we aren’t classifying 1000 imagenet classes<br>
pretrained_model.fc = nn.Sequential(<br>
nn.Flatten() <span class="hashtag">#adds</span> flatten layer at end of model<br>
)<br>
if torch.cuda.is_available(): <span class="hashtag">#uses</span> GPU if available<br>
pretrained_model = pretrained_model.cuda()<br>
return pretrained_model</p>
<p>def get_similarity(pretrained_model,train_imgs):   <span class="hashtag">#function</span> to extract features from the model and compute similarity scores<br>
bottleneck_feature_example = pretrained_model(train_imgs)<br>
similarity = euclidean_distances(bottleneck_feature_example)<br>
similarity=similarity/similarity.max()<br>
similarity_df = pd.DataFrame(similarity)<br>
similarity_df=1-similarity_df<br>
return np.round(similarity_df,4)</p>
<p>transforms =           transforms.Compose([transforms.Resize(256),<br>
transforms.CenterCrop(224),<br>
transforms.ToTensor(),<br>
transforms.Normalize([0.485, 0.456, 0.406],<br>
[0.229, 0.224, 0.225])])</p>
<p>img_dir=‘path’<br>
images = datasets.ImageFolder(img_dir,transform=transforms)<br>
data_loader = torch.utils.data.DataLoader(images,<br>
batch_size=32,<br>
shuffle=True,<br>
num_workers=4)</p>
<p>model_list=[models.densenet201]<br>
model_name=[‘densenet201’]<br>
pretrained_model=[get_pretrained_model_notop(selected_model) for selected_model in model_list]<br>
for data in data_loader:<br>
pred=[get_similarity(pretrained,data) for pretrained in pretrained_model]<br>
pred_label_ensemble=sum(pred) / len(pred)<br>
pred_label_ensemble.columns=page_numbers<br>
prob_output_folder = unzipped.replace(‘MF_loan_document’, ‘MF_loan_document_results’)<br>
pred_label_ensemble.to_csv(prob_output_folder+’/’+‘results.csv’,index=False)</p>

username: ptrblck
cooked: <p>I’m not sure I understand the use case completely and where you are stuck at the moment.<br>
The posted code will iterate the <code>DataLoader</code> and create the similarities for the current batch for all pretrained models.<br>
Would you like to calculate the similarities differently?</p>
<p>PS: You can post code snippets by wrapping them into three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: falconzfan4ever
cooked: <p>The predictions themselves weren’t running, I realized it’s because I needed to subset to data[0], since these images aren’t labeled. One follow up question I have is regarding the batches. The application I’m attempting to compute similarity on has 140 pages, so it’s been converted to 140 images. When I set batch size=10, I get a 10x10 matrix of similarity. But when I set batch size=1, it outputs nothing. Any idea why that might be? Ultimately, my goal is to compute pairwise similarity scores between every image.</p>

username: ptrblck
cooked: <p>You would have to create a similarity matrix for all <code>140x140</code> images.<br>
Based on the current similarity matrix it seems you are calculating it only on the current batch.<br>
For 10 images, you would thus get a <code>10x10</code> matrix and for a single image you should get a single similarity score (image compared to itself).</p>
<p>To calculate the similarities between all images, you could use a nested loop with two <code>DataLoaders</code>.</p>