Algorithm 3 Pseudocode for optimizing a quantized-latent InfoWGAN-GP. Hyperparameter values are detailed in Appendix C.2.
Require: dataset  $\mathcal{D}$  , batch size  $b$  AdamW hyperparameters  $(\alpha ,\beta_{1},\beta_{2}$  weight decay), encoder-to-generator update ratio  $n_e:g$  loss weights  $\lambda \coloneqq (\lambda_{\mathrm{reconstruct~latent}},\lambda_{\mathrm{quantize}},\lambda_{\mathrm{commit}},\lambda_{\mathrm{value}},\lambda_{\mathrm{gradient~penalty}})$    
1: initialize encoder  $\hat{g}^{-1}:\mathcal{X}\to \mathbb{R}^{n_z}$  , discrete value array  $\mathrm{V}\in \mathbb{R}^{n_z\times n_v}$  , decoder  $\hat{g}:\mathbb{R}^{n_z}\rightarrow \mathcal{X}$    
2: initialize critic  $\hat{g}_c^{-1}:\mathcal{X}\rightarrow \mathbb{R}$    
3: while  $\theta \coloneqq (\hat{g}^{-1},\mathrm{V},\hat{g},\hat{g}_c^{-1})$  has not converged do   
4: for  $t = 1,\dots ,n_{e:g}$  do ▷ encoder and critic updates   
5: for  $i = 1,\ldots ,b$  do   
6:  $x_{\mathrm{real}}\sim \mathcal{D}$ $z_{\mathrm{fake}}\sim Z$ $\epsilon \sim$  Uniform([0,1])   
7:  $x_{\mathrm{fake}}\gets \hat{g} (z_{\mathrm{fake}})$    
8:  $\mathcal{L}_{\mathrm{value}}\leftarrow \hat{g}_c^{-1}(x_{\mathrm{fake}}) - \hat{g}_c^{-1}(x_{\mathrm{real}})$  ▷ from WGAN [1]   
9:  $x_{\mathrm{interpolated}}\gets \epsilon x_{\mathrm{real}} + (1 - \epsilon)x_{\mathrm{fake}}$    
10:  $\mathcal{L}_{\mathrm{gradient~penalty}}\gets (\| \nabla_{x_{\mathrm{interpolated}}} \hat{g}_c^{-1}(x_{\mathrm{interpolated}})\| _2 - 1)^2$  ▷ from WGAN-GP [23]   
11:  $\hat{z}_{\mathrm{fake}},\mathcal{L}_{\mathrm{quantize}},\mathcal{L}_{\mathrm{commit}}\gets$  LatentQuantization(xfake,  $\hat{g}^{-1}$  V) ▷ Algorithm 1   
12:  $\mathcal{L}_{\mathrm{reconstruct~latent}}\gets$  MeanSquaredError(  $\hat{z}_{\mathrm{fake}},z_{\mathrm{fake}}$  ) ▷ from InfoGAN [12]   
13:  $\mathcal{L}^{(i)}\gets \lambda \cdot (\mathcal{L}_{\mathrm{reconstruct~latent}},\mathcal{L}_{\mathrm{quantize}},\mathcal{L}_{\mathrm{commit}},\mathcal{L}_{\mathrm{value}},\mathcal{L}_{\mathrm{gradient~penalty}})$    
14:  $(\hat{g}^{-1},\hat{g}_c^{-1})\gets$  AdamW(  $\nabla_{(\hat{g}^{-1},\hat{g}_c^{-1})}\frac{1}{b}\sum_{i = 1}^b{\mathcal{L}}^{(i)},(\hat{g}^{-1},\hat{g}_c^{-1}),\alpha ,\beta_1,\beta_2$  ,weight decay)   
for  $i = 1,\dots ,b$  do   
16:  $z_{\mathrm{fake}}\sim Z$    
17:  $x_{\mathrm{fake}}\gets \hat{g} (z_{\mathrm{fake}})$    
18:  $\mathcal{L}_{\mathrm{value}}\gets -\hat{g}_c^{-1}(x_{\mathrm{fake}})$  ▷ from WGAN [1]   
19:  $\hat{z}_{\mathrm{fake}},\mathcal{L}_{\mathrm{quantize}},\mathcal{L}_{\mathrm{commit}}\gets$  LatentQuantization(xfake,  $\hat{g}^{-1}$  V) ▷ Algorithm 1   
20:  $\mathcal{L}_{\mathrm{reconstruct~latent}}\gets$  MeanSquaredError(  $\hat{z}_{\mathrm{fake}},z_{\mathrm{fake}}$  ) ▷ from InfoGAN [12]   
21:  $\mathcal{L}^{(i)}\gets \lambda \cdot (\mathcal{L}_{\mathrm{reconstruct~latent}},\mathcal{L}_{\mathrm{quantize}},\mathcal{L}_{\mathrm{commit}},\mathcal{L}_{\mathrm{value}},0)$    
22:  $\hat{g}\gets$  AdamW(  $\nabla_{\hat{g}}\frac {1}{b}\sum_{i = 1}^{b}\mathcal {L}^{(i)},$ $\hat{g},\alpha ,\beta_1,\beta_2$  ,weight decay)   
23: V  $\leftarrow$  Adam(  $\nabla_{V}\frac {1}{b}\sum_{i = 1}^{b}\mathcal {L}^{(i)},V,\alpha ,\beta_1,\beta_2$