Index the tuple to extract the tensor

username: phongng20
cooked: <p>I am working to denoising autoencoder. I use the ConcatDataset below to concatenate noisy and original images. I am running into a problem where the ConcatDataset(train_dataset_noisy, train_dataset_original) produces a tuple as a list. How can I index the tuple to extract the tensor?</p>
<p>class ConcatDataset(torch.utils.data.Dataset):<br>
def <strong>init</strong>(self, *datasets):<br>
self.datasets = datasets<br>
def <strong>getitem</strong>(self, i):<br>
return tuple(d[i] for d in self.datasets)<br>
def <strong>len</strong>(self):<br>
return min(len(d) for d in self.datasets)</p>
<h2>Error:</h2>
<p>AttributeError                            Traceback (most recent call last)<br>
 in ()<br>
6<br>
7 for epoch in range(1, max_epoch+1):<br>
----&gt; 8   train(epoch, device=device)<br>
9   test(epoch, device=device)</p>
<p> in train(epoch, device)<br>
6<br>
7     optimizer.zero_grad()<br>
----&gt; 8     images = images.to(device)<br>
9     output = AE(images)<br>
10     loss = loss_fn(output, images) # Here is a typical loss function (Mean square error)</p>
<p>AttributeError: ‘list’ object has no attribute ‘to’</p>

username: RoySadaka
cooked: <p>Does your model accept multiple inputs?<br>
If <code>images</code> are separate inputs, do:</p>
<pre><code class="lang-python">inputs = [x.to(device) for x in images]
outputs = AE(*inputs)
</code></pre>
<p>if images are single inputs, as a batch, use the <code>collate_fn</code> to process the list of samples to form a batch.</p>
<p>Roy</p>

username: phongng20
cooked: <p>Sorry for my lack of understanding. My model accepts single inputs. Since my ConcatDataset return a tuple where it contains the tensor and label, I still confuse whether I need to make changes to the Concat Dataset() or make change in my model to form a batch. I look for collete_fn function but I can’t find it (torch.utils.data).</p>
<p>class our_AE(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(our_AE, self).<strong>init</strong>()<br>
self.encoder = nn.Sequential(<br>
nn.Conv2d(1, 16, 3, stride=2, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(16, 32, 3, stride=2, padding=1),<br>
nn.ReLU(),<br>
nn.Conv2d(32, 64, 7)<br>
)<br>
self.decoder = nn.Sequential(<br>
nn.ConvTranspose2d(64, 32, 7),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(32, 16, 3, stride=2, padding=1, output_padding=1),<br>
nn.ReLU(),<br>
nn.ConvTranspose2d(16, 1, 3, stride=2, padding=1, output_padding=1),<br>
nn.Sigmoid()<br>
)<br>
def forward(self, x):<br>
x = self.encoder(x)<br>
x = self.decoder(x)<br>
return x<br>
AE = our_AE().to(device)<br>
optimizer = optim.Adam(AE.parameters(), lr=1e-4)<br>
loss_fn = nn.MSELoss(reduction=‘sum’)</p>
<p>def train(epoch, device):<br>
AE.train()<br>
for batch_idx, (images, _) in enumerate(train_loader):<br>
optimizer.zero_grad()<br>
images = images.to(device)<br>
output = AE(images)<br>
loss = loss_fn(output, images) # Here is a typical loss function (Mean square error)<br>
loss.backward()<br>
optimizer.step()<br>
if batch_idx % 10 == 0: # We record our output every 10 batches<br>
train_losses.append(loss.item()/batch_size_train) # item() is to get the value of the tensor directly<br>
train_counter.append(<br>
(batch_idx*64) + ((epoch-1)<em>len(train_loader.dataset)))<br>
if batch_idx % 100 == 0: # We visulize our output every 100 batches<br>
print(f’Epoch {epoch}: [{batch_idx</em>len(images)}/{len(train_loader.dataset)}] Loss: {loss.item()/batch_size_train}’)</p>
<p>def test(epoch, device):<br>
AE.eval()<br>
test_loss = 0<br>
correct = 0<br>
with torch.no_grad():<br>
for images, _ in test_loader:<br>
images = images.to(device)<br>
output = AE(images)<br>
test_loss += loss_fn(output, images).item()</p>
<p>test_loss /= len(test_loader.dataset)<br>
test_losses.append(test_loss)<br>
test_counter.append(len(train_loader.dataset)*epoch)<br>
print(f’Test result on epoch {epoch}: Avg loss is {test_loss}’)</p>
<p>train_losses = []<br>
train_counter = []<br>
test_losses = []<br>
test_counter = []<br>
max_epoch = 3<br>
for epoch in range(1, max_epoch+1):<br>
train(epoch, device=device)<br>
test(epoch, device=device)</p>

username: RoySadaka
cooked: <p>Sorry, I’ll try to explain my previous response:</p>
<p>You mentioned your error is</p>
<blockquote>
<p>AttributeError: ‘list’ object has no attribute ‘to’</p>
</blockquote>
<p>From what I see, the only place you have (possibly problematic) “to” is in</p>
<pre data-code-wrap="python"><code class="lang-python">images = images.to(device)
</code></pre>
<p>Can you describe “images”?  it appears to be a list, so that gives 2 possible options:</p>
<ul>
<li>
<p>Does this list represent a batch? aka a list of tensor samples? if so, you need to turn this list into a tensor <strong>before</strong> calling <code>images.to(device)</code></p>
</li>
<li>
<p>Does this list represents multiple inputs to forward function?<br>
a.k.a your forward function looks like: <code>forward(self, x1, x2, x3)</code> then you need to do</p>
</li>
</ul>
<pre data-code-wrap="python"><code class="lang-python">inputs = [x.to(device) for x in images]
outputs = AE(*inputs)
</code></pre>
<p>From the new code you posted, it seems like your model’s forward is <code>forward(self, x)</code> so <strong>no</strong> multiple inputs, then I guess “images” is a list of samples (a mini-batch), and all you need to do it to turn it to a tensor before calling <code>to(device)</code></p>
<p><strong>Side note 1:</strong><br>
The <code>collate_fn</code> is an argument to <code>DataLoader</code> class, in the <a href="https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader" rel="noopener nofollow ugc">doc</a> it says:</p>
<blockquote>
<p><strong>collate_fn</strong>  (<em>callable</em> <em>,</em>  <em>optional</em> ) – merges a list of samples to form a mini-batch of Tensor(s). Used when using batched loading from a map-style dataset.</p>
</blockquote>
<p>It is a function <strong>you</strong> provide to handle the list of samples and turn them into a Tensor batch that your model can get as an input</p>
<p><strong>Side note 2:</strong><br>
The default <code>collate_fn</code>  expects all the images in a batch to have the <strong>same size</strong> because it uses <code>torch.stack()</code>  to pack the images. If the images provided by <code>Dataset</code>  have <strong>variable size</strong>, you <strong>have to provide</strong> your custom <code>collate_fn</code></p>