Getting image name from tensor data after batch data loading

username: treadstone
cooked: <p>To debug my code I need to get name of images in my batch with its labels and prediction. As images in batch are in form of tensor, So I can not acess image name.</p>
<p>In my dataloader below part of code is used-</p>
<blockquote>
<p>def <strong>getitem</strong>(self, i):<br>
data, label = self.data[i], self.label[i]  <span class="hashtag-raw">#here</span> data is actual file name of image<br>
image = self.transform(Image.open(data).convert(‘RGB’))  # here its converted into tensor<br>
return image, label <span class="hashtag-raw">#return</span> image tensor and labels<br>
Below part of code I am using to load data in batch-</p>
</blockquote>
<blockquote>
<p>for i, batch in tqdm(enumerate(self.test_loader, 1)):<br>
if torch.cuda.is_available():<br>
data, _  = [_.cuda() for _ in batch]  <a class="hashtag-cooked" href="/c/data/37" data-type="category" data-slug="data" data-id="37"><span class="hashtag-icon-placeholder"><svg class="fa d-icon d-icon-square-full svg-icon svg-node"><use href="#square-full"></use></svg></span><span>data</span></a> loaded in form of tensor<br>
logits = self.model(data)<br>
loss = F.cross_entropy(logits, label)</p>
</blockquote>
<p>So what needs to be changed so that I can see the data not only in form of tensor but also as name of data(file name)?</p>

username: ptrblck
cooked: <p>You could return the <code>name</code> (<code>data</code> in your example) directly in the <code>Dataset.__getitem__</code> method:</p>
<pre><code class="lang-python">def __getitem___(self, i):
    data, label = self.data[i], self.label[i] #here data is actual file name of image
    image = self.transform(Image.open(data).convert(‘RGB’)) # here its converted into tensor
    return image, label, data
</code></pre>

username: treadstone
cooked: <p>I already tried to do the same suggested by you but then I got below error-</p>
<pre><code class="lang-auto">data, _ = [_.cuda() for _ in batch]
AttributeError: 'list' object has no attribute 'cuda'
</code></pre>
<p>Above error disapperas when I don’t return 3 values from dataloader<br>
test_loader is used as below-</p>
<pre><code class="lang-auto">testset = Dataset('test', args)
test_sampler = CategoriesSampler(testset.label, 1, # args.num_eval_episodes, 
args.eval_way, args.eval_shot + args.eval_query)
test_loader = DataLoader(dataset=testset, batch_sampler=test_sampler, num_workers=args.num_workers)
</code></pre>
<p>Do I have to adapt test_loader so that batch takes image name?</p>

username: ptrblck
cooked: <p>You would have to split the returned batch to the three objects (<code>tensor</code>, <code>tensor</code>, <code>list</code>), as you are currently treating them all as <code>tensors</code>:</p>
<pre><code class="lang-python">data, _ = [_.cuda() for _ in batch]
</code></pre>

username: treadstone
cooked: <p>you mean like -</p>
<pre><code class="lang-auto">for i, batch in tqdm(enumerate(self.test_loader, 1)):
if torch.cuda.is_available():
    file_name=[]
    data, _ , file_name= [_.cuda() for _ in batch]
</code></pre>

username: ptrblck
cooked: <p>No, this should work:</p>
<pre><code class="lang-python">data, _, file_name = batch
data = data.cuda()
</code></pre>
<p>assuming <code>batch</code> contains 3 objects.</p>

username: Samuel_Mensah_Baffoe
cooked: <p>ValueError                                Traceback (most recent call last)<br>
/tmp/ipykernel_689838/1233800762.py in <br>
12 tl=test_loader<br>
13 model_path= ‘best_model.pt’<br>
—&gt; 14 test_model(tl,model_path, save_mod=False)</p>
<p>/tmp/ipykernel_689838/2148491603.py in test_model(test_loader, model_path, save_mod)<br>
270<br>
271<br>
 → 272                 ids=get_ids(test_loader)<br>
273                 img_id=ids<br>
274                 im=transforms.ToPILImage()(t).convert(“RGB”)</p>
<p>/tmp/ipykernel_689838/2148491603.py in get_ids(test_loader)<br>
164         torch.device(“cuda” if torch.cuda.is_available() else “cpu”)<br>
165         ids=[]<br>
 → 166         data,_,ids= batch<br>
167         data=data.to(device)<br>
168         ids=ids.to(device)</p>
<p>ValueError: not enough values to unpack (expected 3, got 2)</p>
<p>I got this Error when I tried . Can you help</p>
<p>This was my code<br>
def <strong>getitem</strong>_(self, i):<br>
data, label = self.data[i], self.label[i] <span class="hashtag">#here</span> data is actual file name of image<br>
#image=self.transforms.ToPILImage()(data).convert(“RGB”)<br>
image = self.transform(Image.open(data).convert(“RGB”)) # here its converted into tensor<br>
return image, label, data<br>
def get_ids(test_loader):<br>
all_ids = torch.tensor([]).to(device)<br>
for i, batch in enumerate(test_loader):<br>
torch.device(“cuda” if torch.cuda.is_available() else “cpu”)</p>
<pre><code>    ids=[]
    data,_,ids= batch
    data=data.to(device)
    ids=ids.to(device)
    ids = ids.float()
    all_ids = torch.cat((all_ids, ids)) 
return all_ids</code></pre>

username: ptrblck
cooked: <p>Your <code>__getitem__</code> method seems to return a single tensor (<code>all_ids</code>) while you are trying to unpack it into 3 return values. Could you explain your use case a bit more?</p>

username: Samuel_Mensah_Baffoe
cooked: <p>This is the first part<br>
<strong>getitem</strong>(self, i):<br>
data, label = self.data[i], self.label[i]<br>
#image=self.transforms.ToPILImage()(data).convert(“RGB”)<br>
image = self.transform(Image.open(data).convert(“RGB”))<br>
return image, label, data</p>
<p>it first returns image, label and data<br>
So i want to extract the filenames(ids) from the inputs (data) and that I tried<br>
for i, batch in enumerate(test_loader):<br>
torch.device(“cuda” if torch.cuda.is_available() else " cpu")<br>
ids=[ ]<br>
data, _, ids= batch<br>
data = data.to (device)<br>
ids= ids.to(device)  # i want to get the ids here<br>
return ids</p>

username: ptrblck
cooked: <p>Ah, I missed the <code>return</code> statement as your code isn’t formatted properly. You can add code snippets by wrapping them into three backticks ``` for easier debugging.<br>
Your code should work, so feel free to post a minimal, executable code snippet reproducing the error.</p>