# Bootstrapped SSL CycleGAN for Asymmetric Domain Transfer
## 3 Proposed Approach
### Proposed SSL+BTS Training Procedure  
In the following lines are formally summarized the main steps of the introduced SSL and BTS strategies, described in Sections 3.1 and 3.2. Algorithm 2 begins by the order of steps that in the very beginning perform exclusively SSL strategy, during the first \(K_{0}\) out of \(N\) iterations, \(K_{0}\ll N\). Only after the SSL secures that discriminator \(D_{X}\) is not overfitting, the BTS strategy is invoked for the first time in iteration: \(K_{0}+1\). Afterwards, the BTS is periodically invoked every \(K\)-th iterations, and jointly employed with the SSL strategy till the end of the procedure. ```
procedureCycleGAN SSL+BTS (process in the right part of Figure 1) \(N\), number of iterations; \(K_{0}\), number of initial SSL iterations without BTS, \(K_{0}\ll N\); \(K\), period of BTS repetition; \(m\), minibatch size; \(M_{SSL}=|\mathcal{P}_{data}|\), number of paired examples in \(\mathcal{P}_{data}=\{(x_{i}^{p},y_{i}^{p})|i=1,\dots,M_{SSL}\}=X_{p}\times Y_{p}\); \(m_{SSL}\), SSL minibatch size, \(m_{SSL}<M_{SSL}\); \(q\in(0,1)\), fixed percentage of training data \(Y\) that are randomly chosen to produce the bootstrapping samples for the training pool of \(D_{X}\); \(\eta>0\), learning rate; \(X\in\mathcal{X},Y\in\mathcal{Y}\), unpaired or unlabeled training sets, while \(X_{p}\in\mathcal{X},Y_{p}\in\mathcal{Y}\) are paired or labeled training subsets, such that \(|\mathcal{X}|\ll|\mathcal{Y}|\), \(|X|\ll|Y|\), and \(|X_{p}|=|Y_{p}|\), but \(|X_{p}|\ll|X|\) ; Randomly initialize parameters of \(D_{X}\), \(D_{Y}\), and \(G_{X\to Y}\), \(G_{Y\to X^{\prime}}\); \(\theta_{D_{X}}\), \(\theta_{D_{Y}}\), \(\theta_{G_{X\to Y}}\), \(\theta_{G_{Y\to X}}\) for\(k=1\) to \(N\)do  Sample minibatch of unpaired training data \(\{x_{1},\dots,x_{m}\}\subset X\), \(\{y_{1},\dots,y_{m}\}\subset Y\)  Sample minibatch of paired training data \(\{(x_{1}^{p},y_{1}^{p}),\dots,(x_{m_{SSL}}^{p},y_{m_{SSL}}^{p})\}\subset X_{p}\times Y _{p}\) \(\widehat{\mathcal{L}}_{adv}(G_{X\to Y},D_{Y})=\ \frac{1}{m}\sum_{i=1}^{m}\ln D_{Y}(y_{i})\ +\ \frac{1}{m}\sum_{i=1}^{m}\ln\left(1-D_{Y}(G_{X \to Y}(x_{i}))\right)\) \(\widehat{\mathcal{L}}_{adv}(G_{Y\to X},D_{X})=\ \frac{1}{m}\sum_{i=1}^{m}\ln D_{X}(x_{i})\ +\ \frac{1}{m}\sum_{i=1}^{m}\ln\left(1-D_{X}(G_{Y \to X}(y_{i}))\right)\) \(\widehat{\mathcal{L}}_{SSL}=\ \frac{1}{m}\sum_{i=1}^{m}\left[\|G_{X\to Y}(x_{i})-y_{i}\|_{1}\ +\ \|G_{Y\to X}(y_{i})-x_{i}\|_{1}\right]\) \(\mathcal{L}_{1}=\ \widehat{\mathcal{L}}_{adv}(G_{Y\to X},D_{X})\ +\ \lambda_{ cycle}\,\mathcal{L}_{ cycle}(G_{X\to Y},G_{Y\to X})\ +\ \lambda_{SSL}\,\widehat{\mathcal{L}}_{SSL}(G_{X\to Y},G_{Y \to X})\) \(\mathcal{L}_{2}=\ \widehat{\mathcal{L}}_{adv}(G_{X\to Y},D_{Y})\ +\ \lambda_{ cycle}\,\mathcal{L}_{ cycle}(G_{X\to Y},G_{Y\to X})\ +\ \lambda_{SSL}\,\widehat{\mathcal{L}}_{SSL}(G_{X\to Y},G_{Y \to X})\) \(\theta_{D_{X}}^{(k+1)}\ \leftarrow\ \theta_{D_{Y}}^{(k)}\ -\ \eta\ \
abla_{\theta_{D_{X}}}\widehat{ \mathcal{L}}_{adv}(G_{Y\to X},D_{X})\) \(\theta_{D_{Y}}^{(k+1)}\ \leftarrow\ \theta_{D_{Y}}^{(k)}\ -\ \eta\ \
abla_{\theta_{D_{Y}}} \widehat{\mathcal{L}}_{adv}(G_{X\to Y},D_{Y})\) \(\theta_{G_{Y\to X}}^{(k+1)}\ \leftarrow\ \theta_{G_{Y\to X}}^{(k)}\ -\ \eta\ \
abla_{\theta_{G_{Y\to X}}} \mathcal{L}_{1}\) \(\theta_{G_{X\to Y}}^{(k+1)}\ \leftarrow\ \theta_{G_{X\to Y}}^{(k)}\ -\ \eta\ \
abla_{\theta_{K_{X\to Y}}} \mathcal{L}_{2}\) if\((k==K_{0}+1)\ \vee\ (\ (k>K_{0})\ \wedge(((k-K_{0})\mod K)==0))\)then  Bootstrapping the training pool \(X_{D}\) of discriminator \(D_{X}\). \(X_{D}\leftarrow\{X\cup X_{BTS}\}\), \(X_{BTS}=rand_{q}(\mathcal{C}_{Y\to X}^{(k)}(Y))\), where \(rand_{q}(\cdot)\) represents the operator of choosing \(\lceil q\,|Y|\rceil\) samples from the training pool \(Y\in\mathcal{Y}\), and transforming them by \(G_{Y\to X}^{(k)}\) to scarce domain \(\mathcal{X}\), i.e., generating \(X_{BTS}\).