# Unsupervised Cyclic Siamese Networks Automating Cell Imagery Analysis
## 3 Methodology
### Architecture and Learning Scheme
#### 3.3.2 Learning Scheme  
For images \(x\) of either natural or synthetic type \(t\in\{n,s\}\), the VAEs are able to generate reconstruction losses \(\operatorname{Rec}(x,y)\) from reconstructed images \(y\) of their decoder. \(C_{\operatorname{Rec}}^{t}\) are hand-crafted weighting factors to balance the different reconstruction costs. Choosing these weights to be large results in better reconstruction but worse regression. However, proper reconstruction quality is required to fabricate well-trained encoders, thus demanding the factors to not be too low. The loss for the reconstructions is defined as  
\[REC_{loss}(x,t)=C_{\operatorname{Rec}}^{n}\cdot\operatorname{Rec}(x_{n},y_{n}) +C_{\operatorname{Rec}}^{s}\cdot\operatorname{Rec}(x_{s},y_{s}) \tag{1}\]  
For synthetic data with cell counts \(l\) from 1 to 30, we can also generate a regression loss \(\operatorname{Reg}(x_{s},l)\). However, \(\operatorname{Reg}(x_{n},l)\) cannot be calculated usually, since \(l\) is not known for these. In Section 4, our ablation studies show that this is insufficient for effective regression on natural data. The internal representations of the two types of images are naturally being separated in the bottleneck, precisely what VAEs are usually known and used for, resulting in high precision for synthetic data, but nearly arbitrary cell counts for natural data. Figure 4: Visualization of the Siamese-Cycle-VAE (SC-VAE) architecture. The blue elements represent synthetic data handling, yellow elements depict natural data handling. Green elements are shared by the two VAEs and contain the inner representation of the cell imagery; purple elements result in an estimation for the cell count. The example images that are outlined are samples from the data sets on the left, with their respective results shown on the right. The translated images outlined with color transitioning have been generated from the opposite data type and are of particular interest, as well as the blue and yellow arrows pointing from right to left that indicate the reuse of decoded images. The examples at the very top left and bottom right are of the utmost importance, since they show the conversion of a synthetic image to a natural-looking one, which can then be used as a labeled pseudo-natural image for training of the regressor with natural-looking images. The specialized architecture allows an additional learning scheme to generate a loss for pseudo-natural data. This is done by encoding synthetic data in their specialized encoder, but decoding them with the decoder designed for natural data. This translation works both ways and will result in images \(x_{\mathfrak{s}\rightarrow\mathfrak{n}}\) and \(x_{\mathfrak{n}\rightarrow\mathfrak{s}}\). This new type of data can now be used in the natural pipeline, creating new reconstruction losses \(\mathrm{Rec}(x_{\mathfrak{s}\rightarrow\mathfrak{n}},y_{\mathfrak{s} \rightarrow\mathfrak{n}})\), which can be used to train the according encoder and decoder, especially enriching the data available for the natural pipeline immensely. These images will be called translated or cycled images from here on and they expand the usable image types to \(t\in\{\mathfrak{n},\mathfrak{s},\mathfrak{s}\rightarrow\mathfrak{n},\mathfrak{ n}\rightarrow\mathfrak{s}\}\).