akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Where to put data transforms? Dataset or collate_fn?

Versus: <p>We have a dataset of texts. One item = 1 text.<br>
And we have 3 transform-pipelines: one for X1, one for X2 and one for Y.</p>
<p>Question: what is the “correct” way to implement transforms?</p>
<ol>
<li>Inside of Dataset? So that each item returns tensors. But then it is not reversible, in case I want to see the original text.</li>
<li>Outside of dataset and before dataloader (my current implementation)</li>
<li>Or inside of <code>collate_fn</code> of Dataloader. Then each item is converted to X1, X2 and Y on batch level. Somehow, <code>collate_fn</code> doesn’t seem to me like the right place for such operations.</li>
</ol>
<p>Ideally, I need transforms to run only when training starts.<br>
I will appreciate your ideas.</p>

ptrblck: <p>The usual workflow would be to add the transformations in the <code>Dataset.__getitem__</code>.<br>
If you need to see the original text (without the transformations), you could return the original as well as the transformed samples from <code>__getitem__</code>.</p>
Inception : Calculated padded input size per channel: (4 x 4). Kernel size: (5 x 5)

Hagar-Usama: <p>Hello,<br>
I am trying to add more layers to inception v3 to fit my dataset. Yet, I don’t figure out the Runtime Error. my input is (3* 256 * 256)</p>
<pre><code class="lang-auto">model =  models.inception_v3(init_weights=True, pretrained=False)

fc_feat = model.fc.in_features
model.fc = nn.Sequential(
    nn.Linear(fc_feat, 512),
    nn.ReLU(),
    nn.Linear(512,3),
)
train_model(model, optimizer, scheduler, 10)
</code></pre>
<pre><code class="lang-auto">
Error:
RuntimeError                              Traceback (most recent call last)

&lt;ipython-input-66-f4a3ac1ce1ac&gt; in &lt;module&gt;()
----&gt; 1 train_model(model, optimizer, scheduler, 10)

11 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
    418                             _pair(0), self.dilation, self.groups)
    419         return F.conv2d(input, weight, self.bias, self.stride,
--&gt; 420                         self.padding, self.dilation, self.groups)
    421 
    422     def forward(self, input: Tensor) -&gt; Tensor:

RuntimeError: Calculated padded input size per channel: (4 x 4). Kernel size: (5 x 5). Kernel size can't be greater than actual input size
</code></pre>
<p>Thanks in advance!</p>

ptrblck: <p>The original Inception model expects an input in the shape <code>[batch_size, 3, 299, 299]</code>, so a spatial size of <code>256x256</code> might be too small for the architecture and an empty activation would be created, which raises the issue.</p>