## 5 Impact Sound Generation for Object Materials
### Proposed GAN Architecture  
In our method, we focused on GAN for image generation and applied it to dynamic impact sound generation. This section outlines the following three points, related to the characteristics of the neural network architecture used in this study. 1. Introduction of residual block, from the SRResNet structure used for super-resolution image generation in Generator
2. Introduction of residual block to Discriminator
3. Introduction of projection to Discriminator, to improve accuracy for multi-category tasks. Residual block is an architecture based on the deep residual network (ResNet-50) [26] used in architectures such as TactGAN [18] and SRGAN [30]. By using a shortcut for this network, gradient disappearance and gradient explosion can be prevented, which enables a deeper network structure. Figure 9 shows the basic architecture of residual block. In ResNet, the purpose is to learn \(H(x)=F(x)+x\) by inserting a network structure, called skip connection, as shown in x: identity, in Figure 9. That is, the difference is trained for the two weight layers in Figure 9, namely, \(F(x)=H(x)-x\). This network structure is called Residual. Here, \(H(x)\) indicates the function that we want to learn. The problem of gradient disappearance can be avoided by learning the difference from the function \(H(x)\) to be learned. Figure 10 shows the architecture constructed by the residual block in this study, which is based on WGAN-GP [31]. In Discriminator, downsampling is performed by average pooling, after the second convolution. Learning can be stabilized by using a normalization function, or activation function, as the processing, after each convolution layer [32]. There are several types of activation functions, but Discriminator uses the Leaky ReLU function. The Leaky ReLU function is an improved version of the ReLU function and, having a small gradient at \(x<0\), can prevent the gradient from disappearing, due to the input of a negative value. In Generator, the basic structure is the same as Discriminator, but the normalization function is included before the activation function, and there is an upsampling layer, instead of average pooling. The ReLU function is used as the activation function. The formulae of the ReLU function and the Leaky ReLU function are shown in Equations (2) and (3), respectively. The downsampling and upsampling procedures are constructed according to the implementation in [33]. We determined the activation function by referring to related studies, such as TactGAN. \[\text{f}(x)=\max(0,\ x) \tag{2}\]  
Figure 10: Residual block architecture for learning. Figure 9: Basic architecture of residual block. \[\mathrm{f(x)}=\left\{ \begin{array}{cc}x&(x>0)\\ ax&(x\leq 0)\end{array} \right. \tag{3}\]  
Figure 11 shows the GAN architecture, constructed using the residual block architecture. Next, we explain the Projection, shown in the Discriminator architecture. In our method, the generation of spectrogram images, based on the material labels of texture images, is required. In AC-GAN [29], this is done by simultaneously discriminating both Real/Fake input images and the class labels. A similar method is utilized in the TactGAN architecture [18], but it has the disadvantage of the learning becoming more difficult, since the loss function term of Discriminator increases. As a countermeasure, we introduce Projection [34], as shown in Figure 11b. In [34], the output of Discriminator is expressed by decomposing it into the sum of two log-likelihoods. Therefore, we derive the output of Discriminator using Equation (4). \[f(x,\ y;\theta)\ :=f_{1}(x,\ y;\theta)+f_{2}(x;\theta)=y^{T}V\phi(x;\theta_{ \phi})+\psi(\phi(x;\theta_{\phi});\theta_{\Psi}) \tag{4}\]  
For the output function \(f\) of Discriminator, \(x\) is the input image, \(y\) is the condition vector, and \(\theta\) is the parameter. The second term on the right side of Equation (4) shows \(\psi\), which is one of the outputs from \(\phi\), and the first term embeds the first output of \(\phi\) and the condition vector \(y\).