CI-VAE: a Class-Informed Deep Variational Autoencoder for Enhanced Class-Specific Data Interpolation
Authors: Mohsen Nabian
Zahra Eftekhari
Alec Wong
Venue: Submitted to ICLR 2023 (2023)
Keywords: Variational Auto Encoder
Supervised
Latent Space Traversal
Data Interpolation
Discriminator

Abstract: We proposed Class-Informed Variational Autoencoder (CI-VAE) to enable interpolation between arbitrary pairs of observations of the same class. CI-VAE combines the general VAE architecture with a linear discriminator layer on the latent space to enforce the construction of a latent space such that observations from different classes are linearly separable. In conventional VAEs, class overlapping on the latent space usually occurs. However, in CI-VAE, the enforced linear separability of classes on the latent space allows for robust latent-space linear traversal and data generation between two arbitrary observations of the same class. Class-specific data interpolation has extensive potential applications in science, particularly in biology, such as uncovering the biological trajectory of diseases or cancer. We used the MNIST dataset of handwritten digits as a case study to compare the performance of CI-VAE and VAE in class-specific data augmentation. We showed that CI-VAE significantly improved class-specific linear traversal and data augmentation compared with VAE while maintaining comparable reconstruction error. In a study of Colon cancer genomics data, we showed that the interpolation between normal cells and tumor cells using CI-VAE may enhance our understanding of cancer development. 

Under review as a conference paper at ICLR 2023
CI-VAE: a Class-Informed Deep Variational Autoen- coder for Enhanced Class-Specific Data Interpolation
Anonymous authors Paper under double-blind review
ABSTRACT
We proposed polation between arbitrary pairs of observations of the same class. CI-VAE com- bines the general VAE architecture with linear discriminator layer on the latent space to enforce the construction of latent space such that observations from dif- ferent classes are linearly separable. In conventional VAES, class overlapping on the latent space usually occurs. However, in CI- VAE, the enforced linear separa- bility of classes on the latent space allows for robust latent-space linear traversal and data generation between two arbitrary observations of the same class. Class- specific data interpolation has extensive potential applications in science, particu- larly in biology, such as uncovering the biological trajectory of diseases or cancer. We used the MNIST dataset of handwritten digits a case study to compare the performance of CI-VAE and VAE in class-specific data augmentation. We showed that CI-VAE significantly improved class-speciff linear traversal and data augmentation compared with VAE while maintaining comparable reconstruction error. In a study of Colon cancer genomics data, we showed that the interpolation between normal cells and tumor cells using CI-VAE may enhance our understand- ing of cancer development.
Variational Autoencoder (CI-VAE) to enable inter-
INTRODUCTION
Variational Autoencoders (VAEs) have emerged popular unsupervised probabilistic neural net-
work models Kingma & Welling (2013) Dilokthanakul et al. 2) Hsu et al. (2017) with a variety
of applicatiinns in computer vision Hsieh et al. 2018] Vahdat Kautz 2020 Tabacof et al. (206));
Huang et al. (2018), natural language processing Wu et al. (2019); Bahuleyan et al. (2017): Seme-
niuta et al. (211)) 7. genomics and precision medicine Grønbech et al. (2020); Minoura et al. (221))
and many other domains. In VAEs, through the encoder, probabilistic latent representation of data in lower dimensional space is inferred. Besides many applications of dimensionality reduction us- ing VAEs, these probabilistic models have proven to be very effective in synthetic data generation. Although VAES are initially designed for unsupervised learning, several variants of VAEs have been designed in other domains such supervised learning and semi-supervised learning Kameoka et al.
2019
et al. (2018); Ehsan Abbasnejad et al. (2017) 701 Xu al. (017)) Wu et al. Sohn al. (2015); Higgins al. (016); Zhao al. (2019).
2019
Suppose we would like to know how two observations transform from one to another. Through linear traversal on the latent space in VAEs, we can generate a trajectory and observe how this transforma- tion may take place. A wide variety of applications can be benefited from this type of solution. For instance, one may attempt to uncover the disease mechanism of Parkinson's through understanding how neuronal cell in healthy brain tissue transformed to a neural cell with Parkinson's 's disease Hook al. 2018); Blauwendraat al. (0200p. Such investigations are often intended for a specific subset/class of the data. In a Parkinson's discase study, for example, the neuronal cells as cell-type/class of the entire population of cells. Therefore, to perform lincar traversal within neuronal cells, we need latent space that is linearly separable among classes to ensure that, during linear traversal, there is no overlapping of classes (cell types in this example).
may intend to investigate
With this motivation, proposed Class Informedd Variational AutoEncoders (CI-VAE), novel deep learning model architecture that includes an additional linear discriminator applied on the latent space, extending the capabilities of VAEs form latent space where observations from different

---

Under review as a conference paper at ICLR 2023
Autoencoder
Veriational Autoencader
Class-lnfrren Informofoddd Variational Autoencoder
Cample from N
Somple ria from sir - NiA Fex
Benoder DenoderNetwoh Network x (C3
Decoder Networt (e(zlt)
Decoder Networt (o((i2)
fromalx
Sample from
Encoder Network = 80)
Mox
p60 - A(03
(00) MOLI
Enceder Network (y(rl))
Eneccer Network
Dota
Inpur Data
leput Data
(a)
(b)
Figure I: a. Autoencoders(AEs) Input data r is deterministically encoded to a lower-dimensions, latent variable = and then mapped via decoder reconstruct the original data é. b. Variational Autoencoder (VAEs): Input data is mapped to the parameters (mean and standard deviation) of the probability function of lower-dimensional latent variable z. is then forming the inferred probability function and the decoder maps the data back to its original form &. Class-Informed Variational Autoencoders (CI-VAEs): trainable weights predicts classes of observations added to the total loss function of the network model.
addition to the VAE network, a linear discriminator network with
the latent space, and its classification loss is
classes are linearly separable. The linear discriminator is simultaneously trained with the entire model. The classification loss of the linear discriminator is included in the total cost function of the model. Using MNIST handwritten digit dataset LeCun et al.] (1998), we investigated class-specific linear traversal and class-specific synthetic data generation and illustrated how it provided robust solution as compared to VAES for data generation during linear latent space traversal. As another use case, we studied Colon Cancer single-cell RNAseq data Qi al. (202)) using CI-VAE enhance our understanding of the development of cancer cells from normal colon cells (See Appendix).
2
AUTOENCODERS
Autoencoders unsupervised neural network models, composed of encoder that provides deterministic mapping of data æ into a lower-dimensional latent space z, followed by a decoder to reconstruct the data into its original form & (see figure 2). Among the applications of autoencoders are dimensionality reduction, data compression as well as data denoising [Simonyan & Zisserman (2014); Kramer (1991). The total cost function in Autoencoders are the reconstruction error, de- scribed by the distance between original input data and the reconstructed observation a.
JAE L(x, 2)
(1)
In Variational Autoencoders (VAEs), however, instead of mapping data æ to latent variable 2, we map the input data z the posterior distribution p(2[z). More specifically, with the assumption of
the Gaussian distribution for p(z[z), ). in VAEs, there is deterministic mapping of the input data to the mean Majæ and standard deviation Engl to construct posterior distribution p(2(æ)) Followed by random sampling from the posterior p(2[z), latent variable constructed. Finally, through the decoder. the latent variable 2 is deterministically mapped to the reconstructed input data x An &
Cho 2015 Kingma Welling (2019).
The total cost function of the variational autoencoder is comprised of 1) The reconstruction error and 2) The regularization term, enforcing the latent space distribution p(z(æ)) to be as close as to the prior p(2) = (ū 7) to enact orthgooalltylnndeeedenee of the z axis as well as to provide regularization for the network weights.
This architecture provides the advantage of 1) projecting data into a lower-dimensine dimensional space and 2) approximating posterior distribution p(=|æz and 3) enabling the generation of new synthetic data
2

---

Under review as a conference paper at ICLR 2023
Define latent state distributions
Sample from distributions
%2
Ps
R2
22
R
X4
32
Variance
22
32
R4
32
%
84
Class
V
Linear Discriminator
Figure 2: General architecture of Class-Informed Variational Autoencoders (CI-VAEs). The linear discriminator is a single linear layer neural network followed by softmax function to predict class probabilities of observations on the latent space.
from the inferred latent space.
JV = L(x, +ax KL[N (* N6)) 7)]
2.1 CLASS-INFORMED VARIATIONAL AUTOENCODERS (CI-VAE)
The proposed EA-V model supervised model based on the VAE architecture with additional components in both the architecture and the total cost function (see figure 2b). To construct CI-VAE, we implemented
current standard VAE architecture with the addition single layer neural network which is used to map latent variable to set of probabilities predict its class y. The linear discriminator is trained with the network with its trainable weights Wixda where h is the latent space dimension and is the number of unique classes y of the data The classification loss (cross-entropy loss) of the linear discriminator is also added to the total loss the standard VAEs. The linear discriminator loss thus enforcing observations from different classes on the latent space to be linearly separable. The aforementioned "Linear" separation is stemmed from the fact that the discriminator network linear. Thus, the linear discriminator is providing two simultaneous impacts: 1) During the backpropagation, the linear discriminator becomes a better classifier. However, since its performance plateaued because of its linearity constraint, 2) the gradient signals are sent to the entire model to infer latent space that is linearly separable based on classes. The entire cost function of CI-VAE therefore described as:
linear discriminator as
JCI-VAB = L(x, a) + a X K KLN ([ 21 NO, 1)]+
(3)
B JLD (v)0)
where Jr.D is the cross-entropy loss between the true class y and the linear class 9 predicted by the linear discriminator for any observation
3

---

Under review as a conference paper at ICLR 2023
2
cp
c1
c4
c2
c3
Z1
Figure 3: Schematic 2D representation of the latent space & after trained on CI-VAE. Latent variables normally distributed around O with the standard deviation of 1 enforced by prior p(2) N T)- In addition, enforced by the linear discriminator in CI-VAE, classes are linearly separated.
2.2 CLASS-SPECIFIC LINEAR TRAVERSAL AND CLASS-SPECIES DATA AUGMENTATION
figure we demonstrated a schematic representation of the latent space generated by CI-VAE. As illustrated in this figure, there a higher probability of data around the origin Ô. This imposed by the KL-divergence loss between the posterior p(2[z) and the prior p(z) = N 7). In addi- tion, observations from different classes are linearly separable network in the CI-VAE model. Linear separability is the desired characteristic when within-class data generation through a linear traversal in the latent space is intended. The linear separability of latent space, theoretically guarantees that any within-class linear traversal would not interfere with the space other classes. While there often the class overlapping problem in VAEs, this geomet- rical property. which is imposed to the latent space in the CI-VAE model, minimizes the chance of interfering with other classes during the within-class data generation through linear traversal.
result of the línear discriminator
Linear traversal can be building block for data augmentation in VAEs. Since CI- CI-VAEs allow for class-specific linear traversal, enables class-specific data augmentation or in other words, it enables generating synthetic labeled data. To this end, we repeatedly randomly select pairs of observations on the latent space from the desired class and perform linear traversal while generating new samples from the same class.
EXPERIMENTS
We empirically assessed proposed CI-VAE MNIST LeCun et al. (1998) handwritten digits dataset. addition to training a CI- CI-VAE model, we trained standard VAE model as a benchmark. For fair comparison, used identical model architecture (except for the linear discriminator in CI- CIE,, the same hyper-parameten as well as using the same training, validation, and test data.
The MNIST dataset consists of 80,000 handwritten digits images each having 28x28x pixels, with the digits between 9. We used the digits as the true class associated with each image, although choosing any other arbitrary class in CI-VAR is possible. The data processing included normalizing

---

Under review as a conference paper at ICLR 2023
VAE
CI-VAE
Taining Total loss
20
Frining
Test Total loss
Test Total loss
10
loss
Test KLD loss
Test KLD loss
8-
15
Test loss
4-
2-
0
100
200
300
400
500
600
700
0
100
200
300
400
500
600
700
800
Figure 4: Average loss components v.s. epoch number for Left: VAE, and Right: CI-VAE
Table 1: Performance comparison between VAE and CI-VAE for reconstruction mean absolute error (MAE) and linear separability of classes on latent space measured by performance of linear logistic regression.
VAE
CI-VAE
Reconstruction Error (MAE)
2.82
2.56
Linear Separability(%)
90.81
99.87
pixel values, flattening features into 28x28=526 dimension, and randomly splitting data into 80% for training-set, 10% validation-set, and the remaining 10% for the test-set.
3.1 MODEL ARCHITECTRRE
The encoder in the CI- VAE model was comprised of seven fully connected layers, each layer is followed by batch-normalization, relu non-linear activation, and dropout to map input data into the mean and standard deviation vectors the latent variable z with 20 dimensions. Random sampling is then taken from Gaussian distribution deseribed by the mean and standard deviation vectors to extract the latent vector 2. Additionally, the linear discriminator, proposed in CI- VAE, consists of single fully connected neural network that takes the = vector input and predicts class probabilities y as output. The decoder also consists of seven fully connected layers, with the same layout as the encoder, making deterministic mapping from reconstructed noted á. The weights in
the model are trained for over 2000 epochs with the batch-size of 512 images until we reached satisfactory convergence (figure
3.2 RESULTS
Once CI-VAE and VAE models are trained, we comparatively investigated the following quantities for each model: 1) Reconstruction Quality, 2) Linear separability of classes of observations in the latent space and 3) Quality of class-specific synthetic data generation through linear traversal.
We measured the reconstruction quality using mean absolute error (MAE) between the input data and the reconstructed data ă. With the measurement of reconstruction eπror (see tablel.). we found out that CI-VAE (with MAE = 2.56) outperformed VAE (with MAE = 2.82). To visualize the CL-VAE and VAE reconstruction performance, in Figure E we illustrated few samples with real digit images, reconstructed images, and the difference-iman real image from the reconstructed one. As shown, the difference-images are very similar between predictions from CI-VAE and VAE models. We also examined how the latent space in each model was linearly separable based on the classes. To this end, after VAE and CI-VAE are trained and the weights locked, trained separate linear logistic regression classifier applied space representation of the test data to predict their respective classes (digits in this case study) for both CI-VAE and VAE. Linear logistic regression represents set of linear hyper-planes that best
defined as the simple subtraction of
the latent
5

---

Under review as a conference paper at ICLR 2023
VAE
CI-VAE
-10
-20
-20
-30
-30
-20
-10
30
-30
-20
2
0
Tamber
number
Figure 5: Representation of the latent space on test data for: Top 2D TSNE plot, Center 2D UMAP plot, Bottom 3D UMAP plot for both VAE model on the Left and CI- CI-VAE model on the Right.
VAE
CI-VAE
VAE
CI-VAE
VAE
CI-VAE
3
3
5
5
8
8
Reconstructed
Reconstructed
Reconstructed
Reconstructed
3
3
5
5
8
8
Diference
Difference
Difference
Difference
Difference
Figure 6: Top: a real observation, Middle: corresponding reconstructed observation, Bottom: error profile. Generated by VAE model (left) and CI-VAE model (right).

---

Under review as a conference paper at ICLR 2023
discriminate classes the latent space. With this regression analysis, we showed that CI-VAE achieved much higher classification accuracy (99.87%) compared to VAE (90.81 %) (see table 1), demonstrating that CI-VAE has greatly separated classes in the latent space compared to VAE by nearly 10%. Enforcement of linear separation of classes in the latent space in CI-VAE enhanced the performance of within-class data generation through linear traversal. In figure 7 we illustrated three examples where we aimed to generate within-class data through latent space linear traversal in both CI-VAE and VAE. We performed this task for digits 3, 5, and 8 respectively. As shown in figure 4 in each traversal, VAE generated figures resembling different digit that was originally intended. For example, VAE generated an image of digit during traversal within two digits 3 and generated images of digit 6 while traversing within two digits 5 and two digits 8. However, CI-VAE is robustly generating samples of digits fully consistent with the intended digit all the experiments. This is due to the fact that classes in the latent space are linearly separated in CI-VAE while having significant overlap in VAE. Due to limited space, we only showed a few examples, but we can draw many more similar examples where CI-VAE robustly generates within-class data where VAE fails to do that.
Within-class linear traversal could be leveraged as the building block for within-class data augmen- tation. We generated synthetic data belonging to class C by randomly selecting two points a E C and b C on the latent space and through linear traversal, we can generate new within-class syn- thetic data. Iterative random selection of pairs of points, linear traversal, and generating synthetic data allows for the generation of an unlimited number of within-class synthetic data. In applications where labeled data is limited, using this method in the CI- model, we are able to generate an extensive amount of new labeled synthetic data. Shown in figure g through latent space linear traversal within digit 3, we generated many synthetic samples for the digit 3. The data points illus- trated in this figure, are presented using a 2D TSNE plot from the test data in its original coordinates.
4 DISCUSSION
We introduced CI-VAE to extend the capabilities of VAEs such that the representation of obser- vations in the lower dimensional latent space becomes linearly separable among classes. We hy- pothesized that by enforeing linear separability of classes using linear discriminator, the class overlapping issues would be addressed and thus it possibly allow "class-specifie" linear traversal and data interpolation.
Shown in figure 2 we purposefully used discriminator with no non-linearity so as to infer a linearly separable latent space. One may suggest using a non-linear discriminator instead, but linear discriminator is particularly chosen I) Linear discriminators on the latent space are equivalent to non-linear high dimensional hyper-planes on the original dimension of the data x. II) Once the linear discriminator is saturated and is unable to linearly separate the data in the latent space, through gradient back-propagation. it sends signals to the entire network to contribute to forming latent space that is more linearly separable while maintaining low reconstruction quality. III) For class-specific data generation, we typically perform linear traversal, thus linear separability of classes allows high-quality within-class data generation.
By visualizing the latent space inferred by VAES and CI-VAEs as in figure 5 we have shown clear separation of classes in CI-VAEs while there exists a significant overlap among classes for the VAE. It is noted that the ability of CI-VAEs to separate classes would be even more apparent when choos- ing more sophisticated classes such as assigning class 1 to female hand- wrrtteni digits and class 2 as otherwise. The point is that CI- CA-S may be applied to any arbitrary class definitions.
In CI-VAE, by adding new cost term from the linear discriminator to the total cost function of the model as shown in equation 3, we may expect the reconstruction quality to be impacted. In table1, we showed that CI-VAE resulted in higher reconstruction quality compared to VAE. This may be a result of providing class information that may have improved learning of the overall model. How- ever, it may be possible to have lower reconstruction error in VAE due to adding linear discriminator loss in CI- VAE. This may occur especially when class definitions are far from the dimees-innii structure of the data. However, by tuning 9 in equation 3, we can control the contribution of the lin- discriminator loss to maintain high reconstruction quality in CI-VAE models.
7

---

Under review as a conference paper at ICLR 2023
A
T1
72
T3
B
3
3
3
3
3
VAE
3
3
3
3
3
CI-VAE
5
5
6
VAE
5
5
5
5
5
5
5
CI-VAE
VAE
8
8
CI-VAE
Figure 7: Linear Traversal three experiments using VAE (top) and CI-VAE (bottom) models. In each experiment, the first and last frames (A and B) are real data are also the same across VAE and CI-VAE models. The three middle frames (71,7', and T3) are synthetically generated frames during linear traversal for each model. It is shown that for VAE, linear traversal occasionally leads to generating data that may belong to a different class. For example digit turned into and 9, digit 5 is turned into 6, and digit is tumed into 6. However, synthetic data generated by CI-VAC CI-VAE are all robustly within the intended class/digit.
8

---

Under review as a conference paper at ICLR 2023
20
10
-10
-20
number
-30
-20
-10
10
20
Figure 8: Data augmentation of digit using CI- VAE model shown in 2D TSNE plot on the original dimensions x.
By fitting logistic regression on the latent space, we have shown Table that the linear sepa- rability in CI-VAE has increased by 10%. Thus, illustrated in figure 7 through linear traversal, VAE is generating images from digits other than intended while CI- CI-VAE robustly generated digits from the intended digit class.
Having linearly separated classes in the latent space, allows to generate class-specific data. In Figure data augmentation of digit 3 is performed using CI-VAE. In VAE however, generate samples from specific class as classes are overlapping on the latent space. Thus, in scenarios where have limited labeled data, can CI- to generate from any class and thus to increase the amount of data to potentially enhance training performance.
unable to
labeled samples
CI-VAE may also be used Labeled portion of the training data can participate in training the entire network, while the unlabeled potion of data may be used to train the model (encoder and decoder) without the participation of the linear discriminator network.
semi-supervised learning model Zhu (0055 Chapelle et al. (2009)
While in CI-VAE, we addressed the issue of overlapping classes latent space to allow class- specific data interpolation, both VAE and CI-VAE may not address the issue of having possible discontinuity (holes) on the latent space. Though, there are many ongoing research works on ad- dressing this issue
5 CONCLUSION
In this paper, we proposed Class-Informed Variational Autoencoder (CI-VAE), generative neural network model comprised of an encoder and decoder similar to the standard architecture of vari- ational autoencoder, with an addition of linear discriminator network which predicts classes of data from its latent representation. This architecture results in the construction of probabilistic latent space that also linearly separable among different classes. This enables within-class data generation through latent space linear traversal between two arbitrary samples of the same class. We showed that the addition of linear discriminator loss into the loss function minimizes the pos- sibility of generating samples from different class during linear traversal. We have conducted comparative study between CI-VAE and VAE using the MNIST dataset for handwritten digits. We showed that CI-VAE provides more robust within-class data generation through linear traversal and improved class-specific data augmentation to compare with VAEs. A wide variety of applications requiring within-class traversal for generating synthetic labeled data in various supervised or semi- supervised problems made possible using Ci-S

---

Under review as a conference paper at ICLR 2023
REFERENCES
Jinwon An and Sungzoon Cho. Variational autoencoder based anomaly detection using reconstruction proba- bility. Special Lecture on IE, 2(1):1-18, 2015.
Hareesh Bahuleyan, Lili Mou, Olga Vechtomova, and Pascal Poupart. Variational attention for sequence-to- sequence models. arXiv preprint urXiv:1112.08207. 2017.
Cornelis Blauwendraat, Mike A Nalls, and Andrew B Singleton. The genetic architecture of parkinson's 's dis- case. The Lancet Neurology, 19(2):170-178, 2020.
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi- Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions Neural Networks, 20(3):542- -5222,54, 2009.
Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimheni, Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint 1oX::::611..0644: 2016.
M Ehsan Abbasnejad, Anthony Dick, and Anton van den Hengel. Infinite variational autoencoder for semi- supervised learning. In Proceedings of the IEEE Conference Computer Vision and Pottern Recognition, pp. 5888 0198-58972220 2017.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-pparaguurre, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik, Automatic chemical design using data-driven continuous representation of molecules. ACS central science, 4(2):268-22) 276, 2018.
Christopher Heje Grønbech, Maximillian Fomitz Vording, Pascal N Timshel, Casper Kaae Sønderby, Tune Pers, and Olc Winther, sevac; Variational auto-encoders for single-cell gene expression data. Bioinformatics, 36(16):4415-4422, 2020.
Irina Higgins, Loic Mathhey Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mo- hamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. 2016.
Paul Hook, Sarah McClymont, Gabrielle Cannon, William D Law, Jennifer Morton, Loyal Goff, and Andrew S McCallion. Single-cell ma-seq of mouse dopaminergic neurons informs candidate gene selection for sporadic parkinson disease. The American Journal of Human Genetics, 102(3):427-4466 2018.
Jun -ing Hsich, Bingbin Liu, De-An Huang, Li Fei-Fei, and Juan Carlos Niebles. Learning to decompose and disentangle representations for video prediction. arXiv preprint 1806.48800.41100 2018
Wei-Ning Hsu, Yu Zhang, and James Glass. Unsupervised domain adaptation for robust speech recognition via variational autoencoder--aan data augmentation. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 16-23. IEEE, 2017.
Huaibo Huang, Zhihang Li, Ran He, Zhenan Sun, and Tieniu Tan. Introvae: Introspective variational autoen- coders for photographic image synthesis. arXiv preprint arXiv:1807.06358. 2018.
Hirokazu Kameoka, Li, Shota Inoue, Shoji Makino. Supervised determined source separation with multichannel variational autoencoder. Neural computation, 31(9):189-11944, 2019.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.61111 2013.
Diederik P Kingma and Max Welling. An introduction variational autoencoders. arXiv: 1906.990669999 2019.
arXiv preprint
Mark A Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE jour- nal, 37(2):233 243, 1991.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11)) 2324, 1998.
Kodai Minoura, Ko Abe, Hyunha Nam, Hiroyoshi Nishikawa, and Teppei Shimamura. A mixture-of-experts deep generative model for integrated analysis of single-cell multionics data. Cell Reports Methods. 1(5): 100071, 2021.
Jingjing Qi. Hongxiang Sun, Yao Zhang, Zhengting Wang, Zhenzhen Xun, Ziyi Li, Xinyu Ding, Rujuan Bao, Liwen Hong, Wenqing Jia, et al. Single-cell and spatial analysis reveal interaction of fap+ fibroblasts and sppl+ + macrophages in colorectal cancer. Nature commanications, 13(1):1-20, 2022.
10

---

Under review as a conference paper at ICLR 2023
Stanislau Semeniuta, Aliaksei Severyn, and Erhardt Barth. hybrid convolutional variational autoencoder for text generation. arXiv preprint arXiv:1702.02390 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409. 2014.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances neural information processing systems, 28, 2015.
Pedro Tabacof, Julia Tavares, and Eduardo Valle. Adversarial images for variational autoencoders. arXiv preprint arXiv:1612.00155, 2016.
Arash Vahdat and Jan Kautz.
Nvae:
deep hierarchical variational autoencoder. arXiv preprint
arXiv:2007.03888, 2020.
Chuhan Wu, Fangzhao Wu, Sixing Wu, Zhigang Yuan, Junxin Liu, and Yongfeng Huang. Semi-supervised dimensional sentiment analysis with variational autoencoder. Knowledge- Based Systems, 165:30-39, 2019.
Broweedge-Based
Weidi Xu, Haoze Sun, Chao Deng. and Ying Tan. Variational autoencoder for semi-supervised text classifica- tion. In Thirty-Fire AAAI Conference on Artificial Intelligence. 2017.
Shengjia Zhao, Jiaming Song, and Stefano Ermon. Infovae: Balancing learning and inference in variational autoencoders. Proceedings of the aaai conference on artificial intelligence, volume 33, pp. 5885-5892, 2019.
Xiaojin Jenyy Zhu, superviee learning literature survey. 2005.


Reviews:
[{"review_id": "gWrD4pVUFjZ", "reviewer": "Reviewer_RrTY", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to apply a logistic regression to the latent space samples of a VAE to enforce linear separability in the latent code. The authors argue that this approach is useful for interpolating between two samples of the sample class without crossing class boundaries. The experiments on MNIST indicate that the regularised latent space fulfils the set-out requirements better than a vanilla VAE.", "summary_of_the_review": "The paper seems generally unpolished with references to missing experiments and incorrect figure references. A lot of relevant background literature is missing which also leads to insufficient experiments and baselines for them.", "strength_and_weaknesses": "Strengths:\n- The paper is clearly written and easy to follow.\n- The proposed method seems to achieve the goal of linear class separability in the latent code.\n\nWeaknesses:\n- The paper is missing important baselines and comparisons to e.g. class-conditional general or GMM-VAEs. Furthermore, it is lacking any discussion of previous work on latent space shaping such as [1]. [1] seems very related and also enforces class separation and is only a single example for missing literature overview.\n- It is not clear whether, linear interpolation is the best or correct interpolation method, instead spherical interpolation could be used. [2] has an overview of different interpolation methods. This choice should be better justified.\n- The paper seems unfinished and unpolished in places - it mentions results on cancer genomics data in the abstract but it seems to only show results on MNIST, which as a dataset is not enough to warrant publication.\n\nMisc:\n- Please correct the usage of `\\citep` and `\\citet` for the ease of reading.\n- Most Figure references seem to be incorrect.\n- Around eq. 1 it is mentioned that AEs optimise the distance between original data and reconstructed data. It would be good to be specific and add example distance metrics.\n- The introduction of VAEs is a bit imprecise when it comes to the likelihood of the output space.\n- Fig. 3 shows a schematic example latent space for the CI-VAE. However, the logistic regression is applied to the samples of z ~ p(z|x) which means that there is no guarantee that z is not sampled outside of the class boundary.\n- Sec 2.2 describes some usage of the model for data augmentation. However, it is very imprecise and should compare to class-conditional VAEs.\n- Fig. 4 shows the losses but it is unclear which loss is which.\n- p 5: 28*28 = 784 instead of 28*28=526\n- Table 1: It would be good to add error bars.\n- Why did you use dropout in this model?\n- Figure 5: How do you interpret these latent space plots?\n- When you perform linear separability testing, how is the new classifier trained? What is the reported value tested on?\n- I do not understand the value of figure 8.\n- What values are chosen for $\\alpha, \\beta$?\n\n[1] Connor, Marissa, Gregory Canal, and Christopher Rozell. \"Variational autoencoder with learned latent structure.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2021.\n[2] Mi, Lu, et al. \"Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework.\" arXiv preprint arXiv:2110.06421 (2021).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CZ-Be1mQiX1", "reviewer": "Reviewer_W4UK", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed Class-Informed Variational Autoencoder (CI-VAE), a generative neural network model comprised of an encoder and decoder similar to the standard architecture of variational autoencoder, with an addition of a linear discriminator network which predicts classes of data from its latent representation. The model was tested with MNIST against vanila VAE.\n", "summary_of_the_review": "The authors proposed Class-Informed Variational Autoencoder (CI-VAE), a generative neural network model comprised of an encoder and decoder similar to the standard architecture of variational autoencoder, with an addition of a linear discriminator network which predicts classes of data from its latent representation. The proposed model was examined only using MNIST and comparison is mostly qualitative.\n", "strength_and_weaknesses": "Strength\n\n- The strategy seems straightforward.\n\nWeaknesses\n\n- In the main text, the model was tested only using MNIST against vanila VAE, and lacking quantitative comparison.\n- In the appenix, the authors applied teh CI-VAE against a particular colon cancer single cell RNA-seq dataset, but the reason why this particular dataset was chosen is unclear, and there is a huge jump from MINST.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9Zf_kGH08xf", "reviewer": "Reviewer_29ie", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes adding a linear discriminator in the training loss of VAE in order to ensure separability of the classes. The goal of such an approach seems to either increase the VAE performance or allow linear interpolation between the data class, which hence provide a quite versatile approach to perform interpolation between object of the same class.\n\n", "summary_of_the_review": "The main drawback of the paper is the lack of discussion and comparison with concurrent approaches, which make the importance of the paper not clear enough.\n", "strength_and_weaknesses": "The paper is very written, clear, and sound. The main weakness is that there is a lack of comparison with other ML approaches for interpolation and a discussion about how the approach differs from the disentangled VAE approaches.\n\nIn particular, one important approach to performing class-specific interpolation can be found in the Optimal Transport ML literature. How does OT interpolation compare to CI-VAE interpolation for MNIST? What would be the benefit/drawback of such an approach versus OT? (The versatility of the CI-VAE is one benefit, but a thorough discussion would be very welcome).\n\nFor biology, there is recent literature about interpolation with OT that the authors should cite and look into. For instance (but not limited to) see and references therein:\n- Fast and Smooth Interpolation on Wasserstein Space\n- TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7okpoYTk32", "reviewer": "Reviewer_ftSd", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce a variational autoencoder with associated loss function that enables to supervision of the latent embedding by class information. In particular, the authors consider linear separability preventing overlapping representations for different classes.", "summary_of_the_review": "- Overall the ideas presented in the paper are useful. The presentation is good, but relatively weak in terms of theory and experiments. The same ideas have been discussed in the literature before and in more detail. ", "strength_and_weaknesses": "_Strengths_:\n\n- The idea is interesting, well motivated and well presented. \n- It makes sense to use label information when it is available. Guiding the latent variable using a weak classifier is a sensible regularization strategy.\n- Using a weak classifier helps impose structure on the latent space.\n\n_Weaknesses_:\n\n- The paper is lacking experiments in several directions. There is only one experiment in the paper and that experiment is with the common MNIST dataset. The approach doesn't compare to any of the previous semi-supervised approaches. \n- While I appreciate the utility of a weak classifier, I am not convinced that linear separability is completely necessary or even desirable. I think a more detailed argument would be necessary. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject