1D Wasserstein DCGAN size error

fllci: <p>Hi all!<br>
I keep getting this error:</p>
<blockquote>
<p>RuntimeError: Calculated padded input size per channel: (3). Kernel size: (4). Kernel size can’t be greater than actual input size</p>
</blockquote>
<p>I understand the error. I did all the calculations of the convolutions and transpose convolutions. There cannot be any input size channel 3. I don’t understand.</p>
<p>Here is my code:</p>
<blockquote>
<p>class Discriminator(nn.Module):<br>
def <strong>init</strong>(self, channels_img, features_d):<br>
super(Discriminator, self).<strong>init</strong>()<br>
self.disc = nn.Sequential(<br>
# input: 1x1x4096<br>
nn.Conv1d(1, 32, kernel_size=4, stride=2, padding=1), # 1x32x2048<br>
nn.LeakyReLU(0.2),<br>
# _block(in_channels, out_channels, kernel_size, stride, padding)<br>
self._block(32, 64, 4, 2, 1), # 1x64x1024<br>
self._block(64, 128, 4, 2, 1), # 1x128x512<br>
self._block(128, 256, 4, 2, 1), # 1x256x256<br>
nn.Conv1d(256, 1, kernel_size=256, stride=1, padding=0), # 1x1x1<br>
)</p>
<pre><code>def _block(self, in_channels, out_channels, kernel_size, stride, padding):
    return nn.Sequential(
        nn.Conv1d(
            in_channels,
            out_channels,
            kernel_size,
            stride,
            padding,
            bias=False,
        ),
        nn.InstanceNorm1d(out_channels, affine=True),
        nn.LeakyReLU(0.2),  
    )

def forward(self, x):
    return self.disc(x)
</code></pre>
<p>class Generator(nn.Module):<br>
def <strong>init</strong>(self, channels_noise, channels_img, features_g):<br>
super(Generator, self).<strong>init</strong>()<br>
<a href="http://self.net" rel="noopener nofollow ugc">self.net</a> = nn.Sequential(<br>
# Input: 1x1x256<br>
self._block(256, 256, 256, 1, 0),  # 1x256x256<br>
self._block(256, 128, 4, 2, 1),  # 1x128x512<br>
self._block(128, 64, 4, 2, 1),  # 1x64x1024<br>
self._block(64, 32, 4, 2, 1),  # 1x32x2048<br>
nn.ConvTranspose1d(32, 1, kernel_size=4, stride=2, padding=1),<br>
# Output: 1x1x4096<br>
nn.Tanh())</p>
<pre><code>def _block(self, in_channels, out_channels, kernel_size, stride, padding):
    return nn.Sequential(
        nn.ConvTranspose1d(
            in_channels,
            out_channels,
            kernel_size,
            stride,
            padding,
            bias=False,
        ),
        nn.BatchNorm1d(out_channels),
        nn.ReLU(),
    )

def forward(self, x):
    return self.net(x)
</code></pre>
</blockquote>
<p>Thank you!!</p>

ptrblck: <aside class="quote no-group" data-username="fllci" data-post="1" data-topic="133340">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fllci/48/12955_2.png" class="avatar"> fllci:</div>
<blockquote>
<p>There cannot be any input size channel 3.</p>
</blockquote>
</aside>
<p>The error points to the spatial size of an activation, which is 3 and thus smaller than the spatial size of a the conv kernel, which is 4.<br>
Check the shapes of all intermediate activations and make sure the spatial size doesn’t become smaller than the kernels.</p>

fllci: <p>Hi ptrblck!<br>
I noticed the one causing the error in the training section is this line:</p>
<blockquote>
<pre><code>        critic_real = critic(data).reshape(-1)
</code></pre>
</blockquote>
<p>My input from real data is tensor of  [16, 1, 1,] and 16 is the batchsize. The one from generator has [1,1,4096]. So I just had to squeeze and unsqueeze the real data tensor to [1,1,16].<br>
But, now I got a different error:</p>
<blockquote>
<p>ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 128, 1])</p>
</blockquote>
<p>Trying to figure that out right now.</p>

ptrblck: <p>The new error is most likely raised by a batchnorm layer, which cannot calculate the batch statistics from a single sample (or element in this case). Increase the batch size during training and it should work.</p>

fllci: <p>I fixed it. I am also using instancenorm. Now I got another type of error that I’ve never come across before. I am currently working on it. Note that my data is 1D vector.</p>
<blockquote>
<pre><code>save_handler = SAVE[format.upper()]
</code></pre>
<p>KeyError: ‘PNG’</p>
</blockquote>
<p>This function causes the error:</p>
<blockquote>
<p>def save(self, fp, format=None, **params):<br>
“”"<br>
Saves this image under the given filename.  If no format is<br>
specified, the format to use is determined from the filename<br>
extension, if possible.</p>
<pre><code>    Keyword options can be used to provide additional instructions
    to the writer. If a writer doesn't recognise an option, it is
    silently ignored. The available options are described in the
    :doc:`image format documentation
    &lt;../handbook/image-file-formats&gt;` for each writer.

    You can use a file object instead of a filename. In this case,
    you must always specify the format. The file object must
    implement the ``seek``, ``tell``, and ``write``
    methods, and be opened in binary mode.

    :param fp: A filename (string), pathlib.Path object or file object.
    :param format: Optional format override.  If omitted, the
       format to use is determined from the filename extension.
       If a file object was used instead of a filename, this
       parameter should always be used.
    :param params: Extra parameters to the image writer.
    :returns: None
    :exception ValueError: If the output format could not be determined
       from the file name.  Use the format option to solve this.
    :exception OSError: If the file could not be written.  The file
       may have been created, and may contain partial data.
    """

    filename = ""
    open_fp = False
    if isPath(fp):
        filename = fp
        open_fp = True
    elif isinstance(fp, Path):
        filename = str(fp)
        open_fp = True
    if not filename and hasattr(fp, "name") and isPath(fp.name):
        # only set the name for metadata purposes
        filename = fp.name

    # may mutate self!
    self._ensure_mutable()

    save_all = params.pop("save_all", False)
    self.encoderinfo = params
    self.encoderconfig = ()

    preinit()

    ext = os.path.splitext(filename)[1].lower()

    if not format:
        if ext not in EXTENSION:
            init()
        try:
            format = EXTENSION[ext]
        except KeyError as e:
            raise ValueError(f"unknown file extension: {ext}") from e

    if format.upper() not in SAVE:
        init()
    if save_all:
        save_handler = SAVE_ALL[format.upper()]
    else:
        save_handler = SAVE[format.upper()]

    if open_fp:
        if params.get("append", False):
            # Open also for reading ("+"), because TIFF save_all
            # writer needs to go back and edit the written data.
            fp = builtins.open(filename, "r+b")
        else:
            fp = builtins.open(filename, "w+b")

    try:
        save_handler(self, fp, filename)
    finally:
        # do what we can to clean up
        if open_fp:
            fp.close()
</code></pre>
</blockquote>

fllci: <p>Alright. Solved it. For anyone following the WGAN from Aladdin Persson on YouTube, he does the WGAN for images and obviously he adds the results on tensorboard using writer.add_image(). But my data is 1D. That’s why I was getting this weird error. I needed to change that to writer.add_scalar(). For anyone who got this kind of error, most likely this is the source of the error.<br>
Best.</p>