# Generative Model for Skeletal Human Movements Based on Conditional DC-GAN Applied to Pseudo-Images
## 3 Materials and Methods
### Conditional Deep Convolution Generative Adversarial Network (Conditional DC-GAN)  
In this paper, we used a conditional Deep Convolution Generative Adversarial Network (conditional DC-GAN) based on DC-GAN [3] and cGAN [21]. In the original DC-GAN architecture, the discriminator was made up of strided convolution layers, batch norm layers, and LeakyReLU activations. Its input was a \(64\times 64\times 3\) (Height \(\times\) Width \(\times\) Channel) image, and the output was a scalar probability that the input is from real-data distribution. The generator was comprised of convolutional-transpose layers, batch norm layers, and ReLU activations. Its input was a latent vector z, that was drawn from a standard normal distribution, and the output was a \(64\times 64\times 3\) RGB image. The strided conv-transpose layers allowed the latent vector to be transformed into a volume with the same shape as an image. Figure 2 shows the architecture of our conditional DC-GAN. In the generator, we used an upsample layer and a convolutional layer instead of the original convolutional-transpose layer in order to avoid checkerboard artifacts [22]. If a standard deconvolution (based on convolutional-transpose layer) is used to upscale from a lower resolution image to a higher one, it uses every point in the small image to \"paint\" a square in the larger one. Here comes the problem of \"uneven overlap\", that is to say, when these \"squares\" in the larger image overlaps. In particular, deconvolution has uneven overlap when the kernel size is not divisible by the stride. These overlapped pixels create an unwanted checkerboard pattern on generated images. Figure 1: Illustration (adapted from [20]) of Tree Structure Skeleton Image (TSSI): (**a**) skeleton structure and order in NTU_RGB+D, with traversal order that respects spatial relations, and (**b**) joint arrangements of TSSI. The shape is (T, J, d) where T is the sequence duration, J’ the number of joints in traversal order (J joint repeated for the TSSI), and d the dimension (d = 3 for (x, y, z) 3D positions). Thus, we instead used an upsample layer followed by convolutional layers to achieve the \"deconvolution\" operation. We resized the image, using the upsample layer with bilinear interpolation (or nearest-neighbor interpolation) and then using a convolutional layer. For the best result, before the convolutional layer, we added a RelectionPad2d layer of size 1 to avoid boundary artifacts. Details are provided in Appendix A. In order to control the output action, the model receives the class label at input for both the generator and discriminator. The generator receives as input a random noise (\(1\times 1\times Z\)) and a one-hot conditional label (\(1\times 1\times C\)), where Z is the dimension of z and C is the number of classes. By default, we chose \(Z=100\) and C = 60. They were concatenated and passed to a convolutional layer, with a \(1\times 1\) kernel. The output channel was 8192 (=\(4\times 4\times 512\)). Then, the output data were resized to the shape of \(4\times 4\times 512\). Finally, we passed the data to several upscaling blocks (Upsample + Conv2D + BatchNorm, except for the output layer), which is exactly the same as the ones in the original DC-GAN. The reason for not directly resizing the input data (the concatenation of random noise and one-hot label) was that, when the data size is \(1\times 1\), it can only be upsampled to an equal-valued data. Also, we wished to keep the input noise size at \(1\times 1\times Z\). Therefore, we decided to add an extra convolutional layer and the resizing step to make it work. In the discriminator, the label was encoded as an image with C channels (C for class number). The label was one-hot encoded along the dimension of channel. To be more specific, the label of the nth action was an H \(\times\) W \(\times\) C sized data, with all 1s on the nth channel and 0 for the rest.