Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders This research attempts to construct a network that can convert online and offline handwritten characters to each other. The proposed network consists of two Variational AutoEncoders (VAEs) with a shared latent space. The VAEs are trained to generate online and offline handwritten Latin characters simultaneously. In this way, we create a crossmodal VAE (CrossVAE). During training, the proposed CrossVAE is trained to minimize the reconstruction loss of the two modalities, the distribution loss of the two VAEs, and a novel third loss called the space sharing loss. This third, space sharing loss is used to encourage the modalities to share the same latent space by calculating the distance between the latent variables. Through the proposed method mutual conversion of online and offline handwritten characters is possible. In this paper, we demonstrate the performance of the CrossVAE through qualitative and quantitative analysis.

Data Augmentation for Historical Documents via Cascade Variational AutoEncoder In this paper, we introduce a novel model based on Variational AutoEncoder (VAE) that is able to find subclasses of categories and generate new samples with fidelity to the subclass in an unsupervised way. In generating characters from historical documents, this model helps generated characters to avoid ambiguity in the case where there are multiple writing styles of one character without being labeled. With this model we augment historical Japanese document dataset to make it more balanced. The model is trained in two steps. In the first step, the model learns the data distribution and learns to map character images into basic shape vectors. In the second step, the model learns to generate new samples conditioned on the basic shape vectors. The generated samples are more robust against intraclass multimodality. With the usage of augmented dataset, the recognition rate is improved. Ablation study is performed to evaluate the effectiveness of data augmentation.