0

.gitignore

@@ -0,0 +1,7 @@
+datasets/
+run/
+checkpoints/
+results/
+!weights/detect/
+start.ipynb
+*.pyc

README.md

@@ -1,12 +1,465 @@
 ---
-title: CycleGAN
-emoji: 🐠
-colorFrom: pink
-colorTo: purple
-sdk: gradio
-sdk_version: 3.23.0
+title: README
+aliases: 
+emoji: 🖼️
+tags: [CycleGAN]
+date: 2023-02-21
 app_file: app.py
 pinned: false
+sdk: streamlit
+sdk_version: 1.17.0
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+>[CycleGAN 论文原文 arXiv](https://arxiv.org/pdf/1703.10593.pdf)
+
+>![动图封面](https://pic4.zhimg.com/v2-7567749554db9eef3451933ae358026f_b.jpg)
+
+>这是文章作者GitHub上的 junyanz
+
+> [CycleGAN junyanz，作者自己用lua 在GitHub 上的实现](https://github.com/junyanz/CycleGAN)
+>
+> ![动图封面](https://pic4.zhimg.com/v2-fd483c63dc186b49c56c5b3e7ac23bc3_b.jpg)
+>
+> 这是GitHub上面其他人实现的 LYnnHo
+
+# 摘要：
+
+图像到**图像**的**翻译 (Image-to-Image translation)** 是一种视觉上和图像上的问题，它的目标是使用成对的图像作为训练集，（让机器）学习从输入图像到输出图像的映射。然而，在很多任务中，成对的训练数据无法得到。
+
+我们提出一种在缺少成对数据的情况下，（让机器）学习从源数据域X到目标数据域Y 的方法。我们的目标是使用一个对抗损失函数，学习映射G：X → Y ，使得判别器难以区分图片 G(X) 与 图片Y。因为这样子的映射受到巨大的限制，所以我们为映射G 添加了一个相反的映射F：Y → X，使他们成对，同时加入一个循环一致性损失函数 (cycle consistency loss)，以确保 F(G(X)) ≈ X（反之亦然）。
+
+在缺少成对训练数据的情况下，我们比较了风格迁移、物体变形、季节转换、照片增强等任务下的定性结果。经过定性比较，我们的方法表现得比先前的方法更好。
+
+# 介绍
+
+在1873年某个明媚的春日，当莫奈(Claude Monet) 在Argenteuil 的塞纳河畔(the bank of Seine) 放置他的画架时，他究竟看到了什么？如果彩色照片在当时就被发明了，那么这个场景就可以被记录下来——碧蓝的天空倒映在波光粼粼的河面上。莫奈通过他细致的笔触与明亮的色板，将这一场景传达出来。
+
+如果莫奈画画的事情发生在 Cassis 小港口的一个凉爽的夏夜，那么会发生什么？
+
+漫步在挂满莫奈画作的画廊里，我们可以想象他会如何在画作上呈现出这样的场景：也许是淡雅的夜色，加上惊艳的几笔，还有变化平缓的光影范围。
+
+我们可以想象所有的这些东西，尽管从未见过莫奈画作与对应场景的真实照片一对一地放在一起。与此不同的是：我们已经见过许多风景照和莫奈的照片。我们可以推断出这两类图片风格的差异，然后想象出“翻译”后的图像。
+
+![](https://pic4.zhimg.com/80/v2-31e1d93b27b4f6e58758b14e5539847b_720w.webp)
+
+photo → Monet
+
+在这篇文章中，我们提出了一个学习做相同事情的方法：在没有成对图像的情况下，刻画一个图像数据集的特征，并弄清楚如何将这些特征转化为另外一个图像数据集的特征。
+
+这个问题可以被描述成概念更加广泛的图像到图像的翻译 (Image-to-Image translation)，从给定的场景x 完成一张图像到另一个场景 y 的转换。举例：从灰度图片到彩色图片、从图像到语义标签(semantic labels) 、从轮廓到图片。发展了多年的计算机视觉、图像处理、计算图像图形学(computational photography, and graphics ?) 学界提出了有力的监督学习翻译系统，它需要成对的数据
+
+（就像那个 pix2pix 模型一样）。
+
+![](https://pic4.zhimg.com/80/v2-a6d175771c6e6c3d6ba8c73e2b34a28b_720w.webp)
+
+需要成对数据的 pix2pix
+
+然而，获取成对的数据比较困难，也耗费资金。例如：只有几个成对的语义分割数据集，并且它们很小。特别是为艺术风格迁移之类的图像任务获取成对的数据就更难了，进行复杂的输出已经很难了，更何况是进行艺术创作。对于许多任务而言，就像物体变形（例如 斑马-马），这一类任务的输出更加不容易定义。
+
+![](https://pic1.zhimg.com/80/v2-2e1402ae3d0267a236aff137394b2010_720w.webp)
+
+wild horse → zebra
+
+- 因此我们寻找一种算法可以学习如何在没有成对数据的情况下，在两个场景之间进行转换。
+  - 我们假设在两个数据域直接存在某种联系——例如：每中场景中的每幅图片在另一个场景中都有它对应的图像，（我们让机器）去学习这个转换关系。尽管缺乏成对的监督学习样本，我们仍然可以在集合层面使用监督学习：我们在数据域X中给出一组图像，在数据域Y 中给出另外一组图像。我们可以训练一种映射G : X → Y 使得 输出，判别器的功能是将生成样本和真实样本 区分开，我们的实验正是要让 和 无法被判别器区分开。理论上，这一项将包括符合 经验分布的的输出���布（通常这要求映射G 是随机的）。从而存在一个最佳的映射将数据域 翻译为数据域 ,使得 数据域（有相同的分布）。然而，这样的翻译不能保证独立分布的输入 和输出 是有意义的一对——因为有无限多种映射 可以由输入的 导出相同的。此外，在实际中我们发现很难单独地优化判别器：当输出图图片从输入映射到输出的时候，标准的程序经常因为一些众所周知的问题而导致奔溃，使得优化无法继续。
+
+为了解决这些问题，我们需要往我们的模型中添加其他结构。因此我们利用 **翻译应该具有“循环稳定性”(translation should be "cycle consistent")** 的这个性质，某种意义上，我们将一个句子从英语翻译到法语，再从法语翻译回英语，那么我们将会得到一相同的句子。从数学上讲，如果我们有一个翻译器 G : X → Y 与另一个翻译器 F : Y → X ，那么 G 与 F 彼此是相反的，这一对映射是双射(bijections) 。
+
+于是我们将这个结构应用到 映射 G 和 F 的同步训练中，并且加入一个 循环稳定损失函数(_cycle consistency loss)_ 以确保到达与。将这个损失函数与判别器在数据域X 与数据域Y 的对抗损失函数结合起来，就可以得到非成对图像到图像的目标转换。
+
+我们将这个方法应用广泛的领域上，包括风格迁移、物体变形、季节转换、照片增强。与以前的方法比较，以前的方法依既赖过多的人工定义与调节，又依赖于共享的内部参数，在比较中也表明我们的方法要优于这些（合格？）基准线(out method outperforms these baselines )。我们提供了这个模型在 [PyTorch](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) 和 [Torch](https://github.com/junyanz/CycleGAN) 上的实现代码，[点击这个网址](https://junyanz.github.io/CycleGAN/)进行访问。
+
+## 相关工作
+
+对抗生成网络 Generative Adversarial Networks (GANs) 在图像生成、图像编辑、表征学习(representation learning) 等领域以及取得了令人瞩目的成就。在最近，条件图像的生成的方法也采用了相同的思路，例如 从文本到图像 (text2image)、图像修复(image inpainting)、视频预测(future prediction)，与其他领域（视频和三维数据）。
+
+对抗生成网络成功的关键是：通过对抗损失（adversarial loss）促使生成器生成的图像在原则上无法与真实图像区分开来。图像生成正是许多计算机图像生成任务的优化目标，这种损失对在这类任务上特别有用。我们采用对抗损失来学习映射，使得翻译得到的图片难以与目标域的图像区分开。
+
+![](https://pic4.zhimg.com/80/v2-a6d175771c6e6c3d6ba8c73e2b34a28b_720w.webp)
+
+需要成对训练图片的 pix2pix
+
+**Image-to-Image Translation 图像到图像的翻译**
+
+这个想法可以追溯到Hertzmann的 图像类比 (Image Analogies)，这个模型在一对输入输出的训练图像上采用了无参数的纹理模型。最近 (2017) 的更多方法使用 输入-输出样例数据集训练卷积神经网络。我们的研究建立在 Isola 的 pix2pix 框架上，这个框架使用了条件对抗生成网络去学习从输入到输出的映射。相似的想法也已经应用在多个不同的任务上，例如：从轮廓、图像属性、布局语义 (semantic layouts) 生成图片。然而，与就如先前的工作不同，我们可以从不成对的训练图片中，学习到这种映射。
+
+**Unpaired Image-to-Image Translation 不成对的图像到图像的翻译**
+
+其他的几个不同的旨在关联两个数据域 X和Y 的方法，也解决了不成对数据的问题。
+
+最近，Rosales 提出了一个贝叶斯框架，通过对原图像以及从多风格图像中得到的似然项 (likelihood term) 进行计算，得到一个基于区块 (patch-based)、基于先验信息的马尔可夫随机场；更近一点的研究，有 CoGAN 和 跨模态场景网络 (corss-modal scene networks) 使用了权重共享策略 去学习跨领域 (across domains) 的共同表示；同时期的研究，有 刘洺堉 用变分自编码器与对抗生成网络结合起来，拓展了原先的网络框架。同时期另一个方向的研究，有 尝试 (encourages) 共享有特定“内容 (content) ”的特征，即便输入和输出的信息有不同的“风格 (style) ”。这些方法也使用了对抗网络，并添加了一些项目，促使输出的内容在预先定义的度量空间内，更加接近于输入，就像 标签分类空间 (class label space) ，图片像素空间 (image pixel space)， 以及图片特征空间 (image feature space) 。
+
+> 不同于其他方法，我们的设计不依赖于 特定任务 以及 预定义输入输出似然函数，我们也不需要要求输入和输出数据处于一个相同的低纬度嵌入空间 (low-dimensional embedding space) 。因此我们的模型是适用于各种图像任务的通用解决方案。我们直接把本文的方案与先前的、现在的几种方案在第五节进行对比。
+
+**Cycle Consitency 循环一致性**
+
+把可传递性 (transitivity) 作为结构数据正则化的手段由来已久。近十年来，在视觉追踪 (visual tracking) 任务里，确保简单的前后向传播一致 (simple forward-backward consistency) 已经成为一个标准。在语言处理领域，通过“反向翻译与核对(back translation and reconcilation) ”验证并提高人工翻译的质量，机器翻译也是如此。更近一些的研究，使用到使用高阶循环一致性的有：动作检测、三维目标匹配，协同分割(co-segmentation) ，稠密语义分割校准(desnse semantic alignment)，景物深度估计(depth estimation) 。
+
+> 下面两篇文章的与我的工作比较相似，他们也使用了循环一致性损失体现传递性，从而监督卷积网络的训练：基于左右眼一致性的单眼景物深度估计 (Unsupervised monocular depth estimation with left-right consistency) —— Godard 通过三维引导的循环一致性学习稠密的对应关系 (Learning dense correspondence via 3d-guided cycle consistency) —— T. Zhou.
+
+我们引入了相似的损失使得两个生成器G与F 保持彼此一致。同时期的研究，Z. Yi. 受到机器翻译对偶学习的启发，独立地使用了一个与我们类型的结构，用于不成对的图像到图像的翻译——Dualgan: Unsupervised dual learning for image-to-image translation.
+
+**神经网络风格迁移 Neural Style Transfer**
+
+神经网络风格迁移是优化 图像到图像翻译 的另外一种方法，通过比较不同风格的两种图像（一张是普通图片，另一张是另外一种风格的图片(一般来讲是绘画作品)）并将一幅图像的内容和另一幅的风格组合起来，基于预训练期间对伽马矩阵进行统计从而得到深层次的特征，再对这些特征进行匹配，最终创造新的图像。
+
+另一方面，我们主要关注的是：通过刻画更高层级外观结构之间的对应关系，学习两个图像集之间的映射，而不仅是两张特定图片之间的映射。因此，我们的方法可以应用在其他任务上，例如从 绘画 → 图片，物体变形(object transfiguration)，等那些单样品转换方法表现不好的地方。我们在 5.2节 比较了这两种方法。
+
+![](https://pic2.zhimg.com/80/v2-6bd4d69c9c31a79b949af61755f1cebd_720w.webp)
+
+## 公式推导
+
+我们的目标是学习两个数据域 X 与 Y 之间的映射函数，定义数据集合与数据分布，与模型的两个映射，其中：
+
+另外，我们引入了两个判别函数：
+
+- 用于区分{x} 与 {F(y)} 的 D_X
+- 用于区分{y} 与 {G(x)} 的 D_Y 。
+
+我们的构建的模型包含两类组件(Our objective contains two types of terms)：
+
+- 对抗损失(adversarial losses)，使生成的图片在分布上更接近于目标图片；
+- 循环一致性损失(cycle consistency losses)，防止学习到的映射 G与F 相互矛盾。
+
+## 对抗损失(adversarial losses)
+
+我们为两个映射函数都设置了对抗损失。对于映射函数G 和它的判别器 D_Y ，我们有如下的表达式：
+
+$$
+\begin{align} \mathcal{L}_{GAN}(G,D_Y,X,Y) &= \mathbb{E}_{y \sim p_{data}(y)} \big[log (D_Y(Y) ) \big] \\ &+ \mathbb{E}_{x \sim p_{data}(x)} \big[log (1-D_Y(G(x))~) \big] \end{align}
+$$
+
+当映射G 试图生成与数据域Y相似的图片 G(x) 的时候，判别器也在试着将生成的图片从原图中区分出来。映射G 希望通过优化减小的项目与 映射F 希望优化增大的项目 相对抗，另一个映射F 也是如此。这两个相互对称的结构用公式表达就是：
+
+$$
+min_G \ max_{D_Y} \ \mathcal{L}_{GAN} (G,D_Y,X,Y) \\ min_F \ max_{D_X} \ \mathcal{L}_{GAN} (F,D_X,Y, X)
+$$
+
+![](https://pic2.zhimg.com/v2-fa1e93662717b22e89d06ccd18425d69_r.jpg)
+
+图3(b), 图3(c)
+
+## 循环一致性损失(cycle consistency loss)
+
+理论上对抗训练可以学习到 映射G与F，并生成与目标域 Y与X 相似的分布的输出（严格地讲，这要求映射G与F 应该是一个随机函数。然而，当一个网络拥有足够大的容量，那么输入任何随机排列的图片，它都可以映射到与目标图片相匹配的输出分布。因此，不能保证单独依靠对抗损失而学习到的映射可以将每个单独输入的 x_i 映射到期望得到的 y_i 。
+
+为了进一步减少函数映射可能的得到的空间大小，我们认为学习的的得到的函数应该具有循环一致性(cycle-consistent)： 如图3(b) 所示，数据域X 中的每一张图片x 在循环翻译中，应该可以让x 回到翻译的原点，反之亦然，即 前向、后向循环一致，换言之： x \rightarrow G(x) \rightarrow F(G(x)) \approx x \\ y \rightarrow F(y) \rightarrow G(F(y)) \approx y
+
+我们使用循环一致性损失作为激励，于是有：
+
+\begin{align} \mathcal{L}_{cyc}(G, F) &= \mathbb{E}_{x \sim p_{data}(x)}[ || F(G(x)) - x ||_1] \\ &+ \mathbb{E}_{y \sim p_{data}(y)}[ || G(F(y)) - y ||_1] \end{align}
+
+初步实验中，我们也尝试用F(G(x)) 与x 之间、G(F(Y)) 与y 之间的对抗损失替代上面的L1 范数，但是没有观察到更好的性能。
+
+如图4 所示，加入循环一致性损失最终使得模型重构的图像F(G(x)) 与输入的图像x 十分匹配。
+
+![](https://pic2.zhimg.com/v2-d3bbe295c729d6ff6963ccefbc9257f9_r.jpg)
+
+图4, cycleGAN 的循环一致性
+
+## 3.3 完整的模型对象(Full Objective)
+
+我们完整的模型对象如下，其中，_λ_ 控制两个对象的相对重要性。
+
+\begin{align} \mathcal{L}(G, F, D_X, D_Y) &= \mathcal{L}_{GAN}(G,D_Y, X, Y) \\ &+ \mathcal{L}_{GAN}(F,D_X, Y, X) \\ &+ \lambda \mathcal{L}_{cyc}(G, F) \\ \end{align}
+
+我们希望解决映射的学习问题：
+
+G^*, \ F^* = \arg \min_{G, F} \max_{D_X, D_Y} \ \mathcal{L}(G, F, D_X, D_Y)
+
+请注意，我们的模型可以视为训练了两个自动编码器(auto-encoder)：
+
+F \circ G: X \rightarrow X \\ G \circ F: Y \rightarrow Y
+
+然而，每一个自动编码器都有它特殊的内部结构：它们通过中间介质将图片映射到自身，并且这个中间介质属于另一个数据域。这样的一种配置可以视为是使用了对抗损失训练瓶颈层(bottle-neck layer) 去匹配任意目标分布的“对抗性自动编码器”(advesarial auto-encoders)。在我们的例子中，目标分布是中间介质Yi 分布于数据域Y 的自动编码器X_i \rightarrow Y_i \rightarrow X_i 。
+
+在 5.1.4节，我们将我们的方法与消去了 完整对象的模型 (ablations of the full objective) 进行比较，包括只包含对抗损失 \mathcal{L}_{GAN} 、只包含循环一致性损失\mathcal{L}_{cyc}，根据我们的经验，在模型中加入这两个对象，对获得高质量的结果而言十分重要。我们也对单向的循环损失模型进行评估，它的结果表明：单向的循环对这个问题的约束不够充分，因而不足以使训练获得足够的正则化。(a single cycle is not sufficient to regularize teh training for this under-constrained problem)
+
+## 4. 实现(Implementation)
+
+**网络结构**
+
+我们采用了**J. Johnson文章**中的生成网络架构。这个网络包含两个步长为2 的卷积层，几个残差模块，两个步长为1/2 的**转置卷积层**(transposed concolution; 原文是fractionally-strided convolutions)。我们使用了6个模块去处理 128x128 的图片，以及9个模块去处理256x256的高分辨率训练图片。与Johnson 的方法类似，我们对每个实例使用了正则化(instance normalization)。我们使用了 70x70 的PatchGANs 作为我的判别器网络，这个网络用来判断图片覆盖的70x70补丁是否来自于原图。比起全图的鉴别器，这样的补丁层级的鉴别器有更少的参数，并且可以以完全卷积的方式处理任意尺寸的图像。
+
+> **J. Johnson文章**：指的是李飞飞他们的那篇文章：感知损失在 实时风格迁移 与 超分辨率 上的应用 (Perceptual losses for real-time style transfer and super-resolution)  
+>
+> 分数步长卷积 (fractionally-strided convolution)：也就是 转置卷积层 Transposed Convolution，也有人叫 反卷积、逆卷积(deconvolution)，不过这个过程不是卷积的逆过程，所以我建议用 **转置卷积层**称呼它。  
+>
+> PatchGANs：使用条件GAN 实现图片到图片的翻译 Image-to-Image Translation with Conditional Adversarial Networks ——本文作者是这篇文章的第二作者
+
+**训练细节**
+
+我们把学界近期的两个技术拿来用做我们的模型里，用于稳定模型的训练。
+
+第一，对于 \mathcal{L}_{GAN} ，我们使用**最小二乘损失**(least squares loss) 取代 原来的负对数似然损失(就是LeCun 的那个)。在训练时，这个损失函数有更好的稳定性，并且可以生成更高质量的结果。实际上，对于GAN的损失函数，我们为两个映射 G(X) 与D(X) 各自训练了一个损失函数。
+
+\mathcal{L}_{GAN}(G,D,X,Y) \\ G.minimize \big( \mathbb{E}_{x \sim p_{data}(x)}[ || G(F(y)) - y ||_1] \big) \\ D.minimize \big( \mathbb{E}_{y \sim p_{data}(y)}[ || F(G(x)) - x ||_1] \big)
+
+第二，为了减小模型训练时候的震荡(oscillation)，我们遵循Shrivastava 的策略——在更新判别器的时候，使用生成的图片历史，而不是生成器最新一次生成的图片。我们把最近50次生成的图片保存为缓存。
+
+在每一次实验中，在计算损失函数的时候，我们都把下面公式内的 \lambda 设置为10，我们使用Adam 进行批次大小为1 的更新。所有的网络都是把学习率设置为0.0002 后，从头开始训练的。在前100次训练中，我们保持相同的学习率，并且在100次训练后，我们保持学习率向0的方向线性减少。第七节（附录7）记录了关于数据集、模型结构、训练程序的更多细节
+
+\begin{align} \mathcal{L}_{GAN}(G, F, D_X, D_Y) &= \mathcal{L}_{GAN}(G,D_Y, X, Y) \\ &+ \mathcal{L}_{GAN}(F,D_X, Y, X) \\ &+ \mathcal{L}_{GAN}(G, F) \cdot \lambda \\ \end{align}
+
+> **最小二乘损失**：来自于——LSGANs, Least squares generative adversarial networks
+
+## 5.结果
+
+首先，我们将我们的方法与最近的 训练图片不成对的图片翻译 方法对比，并且使用的是成对的数据集，评估的时候使用的是 标记正确的成对图片。然后我们将我们的方案与方案的几种变体一同比较，研究了 对抗损失 与 循环一致性损失 的重要性。最后，在只有不成对图片存在的情况下，我们在更大的范围内，展示了我们算法的泛化性能。
+
+为了简洁起见，我们把这个模型称为 CycleGAN。在[我们的网站](https://junyanz.github.io/CycleGAN/)上，你可以找到所有的研究成果，包括[PyTorch](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) 和 [Torch](https://github.com/junyanz/CycleGAN) 的实现代码。
+
+## 5.1 评估
+
+我们使用与pix2pix 相同的评估数据集，与现行的几个基线(baseline) 进行了定量与定性的比较。这些任务包括了 在城市景观数据集(Cityscapes dataset) 上的 语义标签↔照片，在谷歌地图上获取的 地图↔卫星图片。我们在整个损失函数上进行了模型简化测试(ablation study 模型消融研究 )。
+
+![](https://pic4.zhimg.com/v2-73b9abb932ec42dde1efca6173282543_r.jpg)
+
+完整的cycleGAN 与 模型简化测试(ablation study 模型消融研究)
+
+## **5.1.1 评估指标 (Evalution Metrics)**
+
+**AMT感知研究(AMT perceptual studies)** 在 地图↔卫星图片的任务中，我们在亚马逊 **真 · 人工 智能平台**(Amazon Mechanical Turk) 上，建立了“图片真伪判别”的任务，来评估我们的输出的图片的真实性。我们遵循Isola 等人相同的感知研究方法，不同的是：我每个算法收集了25个实验参与者 (participant) 的结果 。会有一系列成对的真伪图片 展示给实验参与者观看，参与者需要点击选择他认为是真实图片的那一张，（另外一张伪造的图片是由我们的算法生成的，或者是其他基线算法(baselines) 生成的）。每一轮的前10个试验用来练习，并将他们答案的正误 反馈给试验参与者。每一轮只会测试一种算法，每一个试验参与者只允许参与测试一轮。
+
+![](data:image/svg+xml;utf8,<svg%20xmlns='http://www.w3.org/2000/svg'%20width='376'%20height='188'></svg></svg>)
+
+亚马逊 真 · 人工 智能平台(Amazon Mechanical Turk)
+
+请注意，我们报告中的数字，不能直接与其他文章中报告的数字进行比较，因为我们的正确标定图片与他们有轻微的不同，并且实验的参与人员也有可能不同（由于实验室在不同的时间进行的）。因此，我们报告中的数字，用来在这篇文章内部进行比较。
+
+**FCN score (全卷积网络得分 full-convolutional network score )** 虽然感知研究可能是评估图像真实性的黄金准则，但是我们也在寻求不需要人类经验的 自动质量检测方法。为此，我们采用**Isola 的 "FCN score"** 来进行 城市景观标签↔照片 转换任务的评估。全卷积网络的根据现成的语义分割模型 来测量评估图片的可解释性。全卷积从一幅图片 预测出整张图片的语义标签地图，然后使用标准的语义分割模型，各自生成图片与输入图片 的标签地图，然后使用它们用来描述的标准语义进行比较。直觉上，我们从标签地图得到“路上的车”的生成图片，然后把生成的图片输入全卷积网络，如果可以从全卷积网络上也得到“路上的车”的标签地图，那么就是我们恢复成功了。
+
+![](https://pic2.zhimg.com/v2-24a992d3c6e5950f1209a8c83f288105_r.jpg)
+
+城市景观语义分割 Cityscapes dataset
+
+**语义分割指标(Semantic segmentation metrics )** 为了评估 图片→标签 的性能，我们使用了城市景观的标准指标(standard metrics from the CItyscapes benchmark )，包括了每一个像素的准确率，每个等级的准确率，类别交并比(Class IoU, class Intersection-Over-Union) 平均值。
+
+> AMT：亚马逊的 **真 · 人工 智能平台**(Amazon Mechanical Turk)，这个平台就是Amazon 在网站上发布任务，由系统分发任务给人类领取，然后按照自己内部的算法支付酬劳，是真的 人工的 智能平台。其实应该叫 人力手动 智能平台。 AMT perceptual studies AMT感知研究，指的就是把自己的结果让人力手动地 去评估。  
+>
+> Isola 等人的研究：用条件对抗网络进行图像到图像的翻译 Imageto-image translation with conditional adversarial networks. In CVPR, 2017. 包括了用 FCN score 对图片进行评估。
+
+## **5.1.2 基线模型(baselines)**
+
+**CoGAN(Coupled GAN)** 这个模型训练一个对抗生成网络，两个生成网络分别生成数据域X 与 数据域Y，前几层都进行权重绑定，并共享对数据潜在的表达。从X 到Y 的翻译可以通过寻找相同的潜在表达 来生成图片X，然后把这个潜在的表达翻译成风格Y 。
+
+**SimGAN** 与我们的方法类似，用对抗损失训练了一个从X 到Y 的翻译，正则项 ||x-G(x)||_1 用来惩罚图片在像素层级上过大的改动。
+
+**Feature loss + GAN** 我们还测试了SimGAN的变体——使用的预训练网络(VGG-16, relu4_2 ) 在图像深度特征(deep image features) 上计算L1 损失，而不是在RGB值上计算。像这样在深度特征空间上计算距离，有时候也被称为是使用“感知损失(perceptual loss )”。
+
+**BiGAN/ ALI** 无条件约束GAN 训练了生成器G: Z→X ，将一个随机噪声映射为图片x 。BiGAN 和ALI 也建议学习逆向的映射F: X→Z 。虽然它们一开始的设计是 学习将潜在的向量z 映射为图片x ，我们实现了相同的组件，将原始图片x 映射到 目标图片y 。
+
+**pix2pix** 我们也比较了在成对数据上训练的pix2pix模型，想看看在不使用成对训练数据的情况下，我们能够如何接近这个天花板。
+
+为了公平起见，我们使用了与我们的方法 相同的架构和细节实现了这些基线模型，除了CoGAN，CoGAN建立在共享 潜在表达 而输出图片的生成器上。因此我们使用了CoGAN实现的公共版本。
+
+> **BiGAN** V. Dumoulin. Adversarially learned inference. In ICLR, 2017.  
+> **ALI** ?? J. Donahue. Adversarial feature learning. In ICLR, 2017  
+> **pix2pix** P.Isola Imageto-image translation with conditional adversarial networks. In CVPR, 2017.
+
+![](https://pic1.zhimg.com/v2-5fe89905cfb11ea864f8163a4ebd43b8_r.jpg)
+
+可以接受不成对训练数据的cycleGAN 和 需要成对数据的pix2pix
+
+5.1.3 与基线模型相比较
+
+5.1.4 对损失函数的分析
+
+5.1.5 图片重构质量
+
+5.1.6 成对数据集的其他结果
+
+## 5.2 应用(Applications)
+
+我们演示了 成对的训练数据不存在时 cycleGAN 的几种应用方法。可以看第七节 附录，以获取更多关于数据集的细节。我们观察到训练集上的翻译 比测试上的更有吸引力，应用的所有训练与测试的数据都可以在[我们项目的网站](https://junyanz.github.io/CycleGAN/)上看到。
+
+**风格迁移 (Collection style transfer)** 我们用Flickr 和 WikiArt 上下载的风景图片，训练了一个模型。注意，与最近的“**神经网络风格迁移(neural style transfer)**” 不同，我们的方法学习的是 对整个艺术作品数据集的仿造。因此，我们可以学习 以梵高的风格生成图片，而不仅是学习 星夜(Starry Night) 这一幅画的风格。我们对 塞尚(Cezanne)，莫奈(Monet)，梵高(Van Gogh) 以及日本浮世绘 这每个艺术风格 都构建了一个数据集，他们的数量大小分别是 526，1073，400，563 张。
+
+**物件变形(Object transfiguration)** 训练这个模型用来将ImageNet 上的一个类别的物件 转化为另外一个类型的物物件（每个类型包含了约1000张的训练图片）。Turmukhambetov 提出了一个子空间的模型将一类物件转变为同一类别的另外一个物件。而我们的方法侧重于两个（来自于不同类别）而视觉上相似的的物件之间的变形。
+
+**季节转换(Season transfer)** 这个模型在 从Flickr上下载的Yosemite风景照上训练，其中包括 854张冬季（冰雪） 和 1273张 夏季的图片。
+
+从画作中生成照片(Photo generation from paintings) 为了建立映射 画作→照片，我们发现：添加一个鼓励颜色成分保留的损失 对映射的学习有帮助。特别是采用了Taigman 的技术后，当提供目标域的真实样本 作为生成器的输入时，对生成器进行正则化以接近恒等映射(identity mapping)
+
+\begin{align} \mathcal{L}_{GAN}(G, F) &= \mathbb{E}_{y \sim p_{data}}[||G(y)-y||_1] \\ &+ \mathbb{E}_{x \sim p_{data}}[||F(x)-x||_1] \end{align}
+
+在不使用 \mathcal{L}_{identity} 时，生成器G与F 可以自由地改变输入图像的色调，而这是不必要的。例如，当学习莫奈的画作和Flicker 的照片时，生成器经常将白天的画作映射到 黄昏时拍的照片，因为这样的映射可能在 对抗损失和循环一致性损失 的启用下同样有效。这种恒等映射在图9 中可以看到。
+
+![](https://pic3.zhimg.com/v2-e065e98788b444cd75a1f62f0fc851a6_r.jpg)
+
+恒等映射(identity mapping)
+
+> **神经网络风格迁移 neural style transfer** L. A. Gatys. Image style transfer using convolutional neural networks. CVPR, 2016  
+>
+> 使用 条件纹理成分分析 为物体的外观建模 D. **Turmukhambetov**. Modeling object appearance using context-conditioned component analysis. In CVPR, 2015.
+
+图12 中，展示的是将莫奈的画作翻译成照片，图12 与 图9 显示的是 包含了训练集的结果，而本文中的其他实验，我们仅显示测试集的结果。因为训练集不包含成对的图片，所以为测试集提供合理的翻译结果是一项非凡的任务（也许可以复活莫奈让他对着相同的景色画一张？——括号内��话是我自己加的）确实，自从莫奈再也不能创作新的绘画作品后，泛化无法看到原画作的测试集并不是一个迫切的问题。（Indeed, since Monet is no longer able to create new paintings, generalization to unseen, “test set”, paintings is not a pressing problem感谢
+
+[@LIEBE](https://www.zhihu.com/people/a078ee299d4a8f2b598c6d1d1e83b4b1)
+
+在评论区给出的翻译）
+
+**图片增强(Photo enhancement)** 图14 中，我们的方法可以用在生成景深较浅的照片上。我们从Flickr下载花朵的照片，用来训练模型。数据源由智能手机拍摄的花朵图片组成，因为光圈小，所以通常具有较浅的景深。目标域包含了由 拥有更大光圈的单反相机拍摄的图片。我们的模型 用智能手机拍摄的浅光圈图片，成功地生成有大光圈拍摄效果的图片。
+
+![](https://pic3.zhimg.com/v2-2cae9f3e20b8e13dfecc26a52adb02de_r.jpg)
+
+景深增强
+
+**与神经风格迁移相比(Comparison with Neural style transfer)** 我们与神经风格迁移在照片风格化任务上相比，我们的图片可以产生出 具备整个数据集风格 的图片。为了在整个风格数据集上，把我们的方法与神经风格迁移相比较，我们在目标域上。计算了平均Gram 矩阵，并且使用这个矩阵转移 神经风格迁移的“平均风格”。
+
+图16 演示了在其他转移任务上 相似的比较，我们发现 神经风格迁移需要寻找一个尽可能与所需输出 相贴近的目标风格图像，但是依然无法产生足够真实的照片，而我们的方法成功地生成与目标域相似，并且比较自然的结果。
+
+![](https://pic4.zhimg.com/v2-a2bf48bd357b48c788927920aafc170b_r.jpg)
+
+cycleGAN 与Gatys 的神经风格迁移相比(Comparison with Neural style transfer)
+
+![](https://pic1.zhimg.com/v2-3dfdcba6fba6680a31e7efc12f436d90_r.jpg)
+
+cycleGAN 与Gatys 的神经风格迁移相比，这个更明显
+
+## 6. 局限与讨论
+
+虽然我们的方法在多种案例下，取得了令人信服的结果，但是这些结果并不都是一直那么好。图17 就展示了几个典型的失败案例。在包括了涉及颜色和纹理变形的任务上，与上面的许多报告提及的一样，我们的方法经常是成功的。我们还探索了需要几何变换的任务，但是收效甚微(limit success)。举例说明：狗→猫 转换的任务，对翻译的学习 退化为对输入的图片进行最小限度的转换。这可能是由于我们对生成器结构的选择造成的，我们生成器的架构是为了在外观更改上的任务上拥有更好的性能而量身定制的。处理更多和更加极端的变化，尤其是几何变换，是未来工作的重点问题。
+
+训练集的分布特征也会造成一些案例的失败。例如，我们的方法在转换 马→斑马 的时候发生了错乱，因为我们的模型只在ImageNet 上训练了 野马和斑马 这两个类别，而没有包括人类骑马的图片。所以普京骑马的那一张，把普京变成斑马人了。
+
+![](https://pic4.zhimg.com/v2-7d175dfa6e52e5c5d70d09b607aa8bab_r.jpg)
+
+把普京变成斑马人，等
+
+![](https://pic1.zhimg.com/v2-aca4565b492355e015bb2464930d7df4_r.jpg)
+
+猫 → 狗 ，苹果 → 橙子 ，形状的变换不足
+
+我们也发现在成对图片训练 和 非成对训练 之间存在无法消弭的差距。在一些案例里面，这个差距似乎特别难以消除，甚至不可能消除。为了消除（模型对数据理解上的）歧义，模型可能需要一些弱语义监督。集成的弱监督或者半监督数据也许能够造就更有力的翻译器，这些数据依然只会占完全监督系统中的一小部分。
+
+尽管如此，在多种情况下，完全使用不成对的数据依然是足够可行的，我们应该使用。这篇论文拓展了“无监督”配置可能使用范围。
+
+> 论文原文中的部分图表，我没有给出  
+> **致谢部分，我用的是谷歌翻译2018-10-25 16:27:43**  
+> **第七节的附录，我用的是谷歌翻译2018-10-25 16:27:43**
+
+**！！！下面都是谷歌翻译！！！**
+
+**致谢：**我们感谢Aaron Hertzmann，Shiry Ginosar，Deepak Pathak，Bryan Russell，Eli Shechtman，Richard Zhang和Tinghui Zhou的许多有益评论。这项工作部分得到了NSF SMA1514512，NSF IIS-1633310，Google Research Award，Intel Corp以及NVIDIA的硬件捐赠的支持。JYZ由Facebook Graduate Fellowship支持，TP由三星奖学金支持。用于风格转移的照片由AE拍摄，主要在法国拍摄。
+
+## **7.附录7.1。**
+
+训练细节所有网络（边缘除外）均从头开始训练，学习率为0.0002。在实践中，我们将目标除以2，同时优化D，这减慢了D学习的速率，相对于G的速率。我们保持前100个时期的相同学习速率并将速率线性衰减到零。接下来的100个时代。权重从高斯分布初始化，均值为0，标准差为0.02。
+
+Cityscapes标签↔Photo2975训练图像来自Cityscapes训练集[4]，��像大小为128×128。我们使用Cityscapes val集进行测试。
+
+Maps↔aerial照片1096个训练图像是从谷歌地图[22]中删除的，图像大小为256×256。图像来自纽约市内及周边地区。然后将数据分成火车并测试采样区域的中位数纬度（添加缓冲区以确保测试集中没有出现训练像素）。建筑立面标签照片来自CMP Facade数据库的400张训练图像[40]。边缘→鞋子来自UT Zappos50K数据集的大约50,000个训练图像[60]。该模型经过5个时期的训练。
+
+Horse↔Zebra和Apple↔Orange我们使用关键字“野马”，“斑马”，“苹果”和“脐橙”从ImageNet [5]下载图像。图像缩放为256×256像素。每个班级的训练集大小为马：939，斑马：1177，苹果：996，橙色：1020.
+
+Summer↔WinterYosemite使用带有标签yosemite和datetaken字段的Flickr API下载图像。修剪了黑白照片。图像缩放为256×256像素。每个班级的培训规模为夏季：1273，冬季：854。照片艺术风格转移艺术图像从[http://Wikiart.org](http://Wikiart.org)下载。一些素描或**过于淫秽的艺术品**都是手工修剪过的。这些照片是使用标签横向和横向摄影的组合从Flickr下载的。黑白照片被修剪。图像缩放为256×256像素。每个班级的训练集大小为Monet：1074，Cezanne：584，Van Gogh：401，Ukiyo-e：1433，照片：6853。Monet数据集被特别修剪为仅包括风景画，而Van Gogh仅包括他的后期作品代表了他最知名的艺术风格。
+
+Photo↔Art风格转移艺术图像从[http://Wikiart.org](http://Wikiart.org)下载。一些素描或过于淫秽的艺术品都是手工修剪过的。这些照片是使用标签横向和横向摄影的组合从Flickr下载的。黑白照片被修剪。图像缩放为256×256像素。每个班级的训练集大小为Monet：1074，Cezanne：584，Van Gogh：401，Ukiyo-e：1433，照片：6853。
+
+Monet数据集被特别修剪为仅包括风景画，而Van Gogh仅包括他的后期作品代表了他最知名的艺术风格。莫奈的画作→照片为了在保存记忆的同时实现高分辨率，我们使用矩形图像的随机方形作物进行训练。为了生成结果，我们将具有正确宽高比的宽度为512像素的图像作为输入传递给生成器网络。身份映射损失的权重为0.5λ，其中λ是周期一致性损失的权重，我们设置λ= 10.
+
+花卉照片增强智能手机拍摄的花卉图像是通过搜索Apple iPhone 5拍摄的照片从Flickr下载的，5s或6，搜索文本花。具有浅DoF的DSLR图像也通过搜索标签flower，dof从Flickr下载。将图像按比例缩放到360像素宽。使用重量0.5λ的同一性映射损失。智能手机和DSLR数据集的训练集大小分别为1813和3326。
+
+7.2。网络架构
+
+我们提供PyTorch和Torch实现。
+
+**发电机架构** 我们采用Johnson等人的架构。[23]。我们使用6个块用于128×128个训练图像，9个块用于256×256或更高分辨率的训练图像。下面，我们遵循Johnson等人的Github存储库中使用的命名约定。
+
+令c7s1-k表示具有k个滤波器和步幅1的7×7Convolution-InstanceNormReLU层.dk表示具有k个滤波器和步幅2的3×3卷积 - 实例范数 - ReLU层。反射填充用于减少伪像。Rk表示包含两个3×3卷积层的残余块，在两个层上具有相同数量的滤波器。uk表示具有k个滤波器和步幅1 2的3×3分数跨度-ConvolutionInstanceNorm-ReLU层。
+
+具有6个块的网络包括：c7s1-32，d64，d128，R128，R128，R128，R128，R128，R128，u64，u32，c7s1-3
+
+具有9个块的网络包括：c7s1-32，d64，d128，R128，R128，R128，R128，R128，R128，R128，R128，R128，u64 u32，c7s1-3
+
+**鉴别器架构** 对于鉴别器网络，我们使用70×70 PatchGAN [22]。设Ck表示具有k个滤波器和步幅2的4×4卷积 - 实例范数 - LeakyReLU层。在最后一层之后，我们应用卷积来产生1维输出。我们不将InstanceNorm用于第一个C64层。我们使用泄漏的ReLU，斜率为0.2。鉴别器架构是：C64-C128-C256-C512
+
+## **欢迎讨论 ^_^**
+
+## 对评论区的回复：
+
+- 几个关于翻译的建议
+- 如猫狗转换这种一对多的转换，有其他模型能学到吗？有，但是受应用场景限制
+- CycleGAN如何保证不发生交叉映射？在满足双射(bijections)的情况下，保持循环一致
+
+> [@LIEBE](https://www.zhihu.com/people/a078ee299d4a8f2b598c6d1d1e83b4b1)
+>
+>
+> 这里有几个小小的（关于翻译的）建议，不知道我理解得正不正确：
+
+采用并在正文改正：upper bound：天花板×，上限√。作者 写**光圈小，景深深（大）**是正确的，谢谢提醒，我自己弄错了。
+
+1.translation可以翻成转化？（“图像间的转化”读起来更直白一点？）我坚持把translation直接翻译成「翻译」，理由如下：论文原文有“a sentence from English to French, and then translate it back from French to English”， CycleGAN 的灵感是从「**语言翻译**」中来的，**英语→法语→英语，图像域A→图像域B→图像域A**。而[Pix2Pix - Image-to-Image Translation with Conditional Adversarial Networks - 2018](https://zhuanlan.zhihu.com/p/45394148/h%3Cb%3Ettps://arxiv.org/pdf/1611.%3C/b%3E07004.pdf)的论文原文也有“. Just as a concept may be expressed in either **English or French**, a scene may be rendered as an RGB image, ... ...”，因此我没有翻成**转化**。
+
+4.第5.1节 的评估那里是还没翻译完吗？考不考虑手动翻译一遍7.附录呢？不打算翻译，我认为这部分内容不看不会影响我们对CycleGAN的理解。
+
+> [@小厮](https://www.zhihu.com/people/071d206346dc5dcf22339f44bfd324c2)
+>
+> 2019-08-14  
+> cyclegan的话，更强调循环一致，猫狗转换这种，对于一只猫转换为一只狗，一只狗转换为一只猫，存在很多**1-多的关系**。因为不存在明显一一对应关系，那么目标就变成学习一个足够真实（和源分布尽可能相似）的分布。**对于猫狗转换这种，别的gan可以学习学到嘛？**
+
+我的回答：有，但是受应用场景限制。例如：
+
+小样本无监督图片翻译 [Few-Shot Unsupervised Image-to-Image Translation - 英伟达 2019-05 论文pdf](https://arxiv.org/pdf/1905.01723.pdf) （从视频上看，做的非常好，但是我还没有机会去复现它）
+
+知乎相关介绍 [英伟达最新图像转换器火了！万物皆可换脸，试玩开放 - 新智元](https://zhuanlan.zhihu.com/p/65297812)
+
+![](https://pic3.zhimg.com/v2-e0decda33e7936b5af15bd617eb2e076_r.jpg)
+
+让各种食物变成炒面，仔细观察可以看到图片翻译前后，其结构是高度对应的
+
+![动图封面](https://pic3.zhimg.com/v2-d02676e327da54db27b6382268b7ebd6_b.jpg)
+
+让狗变成其他动物，上面是以上是张嘴、歪头的动图
+
+另一个，人类的转换，人脸到人脸转换，有 DeepFace等开源工具可以实现。人脸到其他脸，有它可以实现→ ，论文 [Landmark Assisted CycleGAN for Cartoon Face Generation - 香港中文 - 贾佳亚](https://arxiv.org/abs/1907.01424) 。
+
+知乎相关介绍： [用于卡通人脸生成的关键点辅助CycleGAN](https://zhuanlan.zhihu.com/p/73995207)
+
+![](https://pic3.zhimg.com/v2-bd2e0215d84584f36cc175c4e77b1402_r.jpg)
+
+> [@信息门下走狗](https://www.zhihu.com/people/d67ee185c975baea76cf8d401876f9d3)
+>
+> 2018-11-02  
+> 极端的例子，假定就是马和斑马的变换，马与斑马都分两种姿态，一类是站着，一类是趴着，那么对于系统设计的代价函数而言，我完全可以把所有站着的马都映射为趴着的斑马，趴着的马变成站着的斑马，然后逆向映射把趴着的斑马映射回站着的马。这个映射的代价函数和正常保持马的姿态的映射是一样的。那么系统是如何保证**不发生这种交叉映射**的呢
+
+我的回答：在满足双射(bijections)的情况下，保持循环一致性。
+
+文章第二节的**Cycle Consitency 循环一致性**提到：加入循环一致性损失 (cycle consistency loss)，可以保证了映射的正确，在语言处理领域也是如此。  
+文章第一节的介绍部分，提到：从数学上讲，如果我们有一个翻译器 G : X → Y 与另一个翻译器 F : Y → X ，那么 G 与 F 彼此是相反的，这一对映射是双射(bijections) 。系统通过**Cycle Consitency 循环一致性**，保证不发生**趴着的野马**，**站着的斑马**这种交叉映射，从数学上讲，原理是[双射(wikipedia: bijections)](https://en.wikipedia.org/wiki/Bijection)，即参与映射的两个集合，里面的元素必然是一一对应的。
+
+野马集合 斑马集合
+
+- 站姿1的野马 ⇋ 站姿1的斑马
+- 站姿2的野马 ⇋ 站姿2的斑马
+- ...
+- 跪姿1的野马 ⇋ 跪姿1的斑马
+- ...
+
+↑符合循环一致性的双射
+
+假如在某一次随机重启后，cycleGAN恰好学到了 站姿⇋跪姿 的转化，我们**不期望的交叉映射**发生了，有如下：
+
+野马集合 斑马集合
+
+- 站姿1的野马 ⇋ 跪姿1的斑马
+- 站姿2的野马 ⇋ 跪姿2的斑马
+- ...
+- 跪姿1的野马 ⇋ 站姿1的斑马
+- ...
+
+其实经过分析，我们会发现**无法证明 对于站姿n，有稳定的跪姿n 与其一一对应**，**所以这种交叉映射是不符合双射的。**不符合双射的两个映射，不满足**循环一致性**。然而，由于野马斑马外形差别不大，导致站姿n的野马 到 站姿n斑马 总是存在稳定的一一对应的姿态可以相互转换，所以，可以预见到，在执行图片到图片的翻译任务时，当训练到收敛的时候，循环稳定性可以得到正确的结果，避免交叉映射。
+
+根据文章中提到的：使用cycleGAN 可以进行 野马 ⇋斑马 的转换，但是无法转化形态差异比较大的 猫 ⇋ 狗。 因为从 猫 ⇋ 狗 之间的转化，**不能严格满足 双射 的要求**，当存在许多差异明显的猫狗时，因为循环一致性并不能指导模型，导致��多的交叉映射破坏循环一致性，从而无法训练得到满意的模型。
+
+![](data:image/svg+xml;utf8,<svg%20xmlns='http://www.w3.org/2000/svg'%20width='294'%20height='174'></svg></svg>)
+
+文章中提及的 失败案例（形态差异大的） 猫 ⇋ 狗
+
+所以，我的结论：
+
+- 野马 ⇋ 斑马：满足双射，可以避免交叉映射，符合循环一致性，训练效果好
+- 猫 ⇋ 狗：不能严格满足双射，无法完全避免交叉映射，破坏了循环一致性，训练效果不佳
+
+cyclegan的话，更强调循环一致，猫狗转换这种，对于一只猫转换为一只狗，一只狗转换为一只猫，存在很多1-多的关系。因为不存在明显一一对应关系，那么目标就变成学习一个足够真实（和源分布尽可能相似）的分布。对于猫狗转换这种，别的gan可以学习学到嘛？

app.py

@@ -0,0 +1,97 @@
+"""
+实现web界面
+"""
+
+from pathlib import Path
+
+import gradio as gr
+from detect import detect
+from util import get_all_weights
+
+app_introduce = """
+# CycleGAN
+功能：上传本地文件、选择转换风格
+"""
+
+css = """
+# :root{
+# --block-background-fill: #f3f3f3;
+# }
+footer{
+    display:none!important;
+}
+"""
+
+demo = gr.Blocks(
+    # theme=gr.themes.Soft(),
+    css=css
+)
+
+
+def add_img(img):
+    imgs = []
+    for style in get_all_weights():
+        fake_img = detect(img, style=style)
+        imgs.append(fake_img)
+    return imgs
+
+
+def tab1(label="上传单张图片"):
+    default_img_paths = [
+        [Path.cwd().joinpath("./imgs/horse.jpg"), "horse2zebra"],
+        [Path.cwd().joinpath("./imgs/monet.jpg"), "monet2photo"],
+    ]
+
+    with gr.Tab(label):
+        with gr.Row():
+            with gr.Column():
+                img = gr.Image(type="pil", label="选择需要进行风格转换的图片")
+                style = gr.Dropdown(choices=get_all_weights(), label="转换的风格")
+                detect_btn = gr.Button("♻️风格转换")
+            with gr.Column():
+                out_img = gr.Image(label="风格图")
+        detect_btn.click(fn=detect, inputs=[img, style], outputs=[out_img])
+        gr.Examples(default_img_paths, inputs=[img, style])
+
+
+def tab2(label="单图多风格试转换"):
+    with gr.Tab(label):
+        with gr.Row():
+            with gr.Column(scale=1):
+                img = gr.Image(type="pil", label="选择需要进行风格转换的图片")
+                gr.Markdown("上传一张图片，会将所有风格推理一遍。")
+                btn = gr.Button("♻️风格转换")
+            with gr.Column(scale=2):
+                gallery = gr.Gallery(
+                    label="风格图",
+                    elem_id="gallery",
+                ).style(grid=[3], height="auto")
+        btn.click(fn=add_img, inputs=[img], outputs=gallery)
+
+
+def tab3(label="参数设置"):
+    from detect import opt
+
+    with gr.Tab(label):
+        with gr.Column():
+            for k, v in sorted(vars(opt).items()):
+                if type(v) == bool:
+                    gr.Checkbox(label=k, value=v)
+                elif type(v) == (int or float):
+                    gr.Number(label=k, value=v)
+                elif type(v) == list:
+                    gr.CheckboxGroup(label=k, value=v)
+                else:
+                    gr.Textbox(label=k, value=v)
+
+
+with demo:
+    gr.Markdown(app_introduce)
+    tab1()
+    tab2()
+    tab3()
+if __name__ == "__main__":
+    demo.launch(share=True)
+    # 如果不以`demo`命名，`gradio app.py`会报错`Error loading ASGI app. Attribute "demo.app" not found in module "app".`
+    # 注意gradio库的reload.py的头信息 $ gradio app.py my_demo, to use variable names other than "demo"
+    # my_demo 是定义的变量。离谱o(╥﹏╥)o

app1.py

@@ -0,0 +1,87 @@
+# 2023年2月23日
+"""
+实现web界面
+
+>>> streamlit run app.py
+"""
+
+from io import BytesIO
+from pathlib import Path
+
+import streamlit as st
+from detect import detect, opt
+from PIL import Image
+from util import get_all_weights
+
+"""
+# CycleGAN
+功能：上传本地文件、选择转换风格
+"""
+
+
+def load_css(css_path="./util/streamlit/css.css"):
+    """
+    加载CSS文件
+    :param css_path: CSS文件路径
+    """
+    if Path(css_path).exists():
+        with open(css_path) as f:
+            # 将CSS文件内容插入到HTML中
+            st.markdown(
+                f"""<style>{f.read()}</style>""",
+                unsafe_allow_html=True,
+            )
+
+
+def load_img_file(file):
+    """读取图片文件"""
+    img = Image.open(BytesIO(file.read()))
+    st.image(img, use_column_width=True)  # 显示图片
+    return img
+
+
+def set_style_options(label: str, frame=st):
+    """风格选项"""
+    style_options = get_all_weights()
+    options = [None] + style_options  # 默认空
+    style_param = frame.selectbox(label=label, options=options)
+    return style_param
+
+
+# load_css()
+tab_mul2mul, tab_mul2one, tab_set = st.tabs(["多图多风格转换", "多图同风格转换", "参数"])
+
+with tab_mul2mul:
+    uploaded_files = st.file_uploader(label="选择本地图片", accept_multiple_files=True, key=1)
+    if uploaded_files:
+        for idx, uploaded_file in enumerate(uploaded_files):
+            colL, colR = st.columns(2)
+            with colL:
+                img = load_img_file(uploaded_file)
+                style = set_style_options(label=str(uploaded_file), frame=st)
+            with colR:
+                if style:
+                    fake_img = detect(img=img, style=style)
+                    st.image(fake_img, caption="", use_column_width=True)
+
+with tab_set:
+    colL, colR = st.columns([1, 3])
+    for k, v in sorted(vars(opt).items()):
+        st.text_input(label=k, value=v, disabled=True)
+    # st.selectbox("ss", options=opt.parse_args())
+    confidence_threshold = st.slider("Confidence threshold", 0.0, 1.0, 0.5, 0.01)
+    opt.no_dropout = st.radio("no_droput", [True, False])
+
+with tab_mul2one:
+    uploaded_files = st.file_uploader(label="选择本地图片", accept_multiple_files=True, key=2)
+    if uploaded_files:
+        colL, colR = st.columns(2)
+        with colL:
+            imgs = [load_img_file(ii) for ii in uploaded_files]
+        with colR:
+            style = set_style_options(label="选择风格", frame=st)
+            if style:
+                if st.button("♻️风格转换", use_container_width=True):
+                    for img in imgs:
+                        fake_img = detect(img, style)
+                        st.image(fake_img, caption="", use_column_width=True)

data/__init__.py

@@ -0,0 +1,114 @@
+"""不同的模型使用不同的数据集
+
+比如有监督模型使用的都是成对的训练数据、无监督模型使用的数据集不必使用成对的数据
+This package includes all the modules related to data loading and preprocessing
+
+ To add a custom dataset class called 'dummy', you need to add a file called 'dummy_dataset.py' and define a subclass 'DummyDataset' inherited from BaseDataset.
+ You need to implement four functions:
+    -- <__init__>:                      initialize the class, first call BaseDataset.__init__(self, opt).
+    -- <__len__>:                       return the size of dataset.
+    -- <__getitem__>:                   get a data point from data loader.
+    -- <modify_commandline_options>:    (optionally) add dataset-specific options and set default options.
+
+Now you can use the dataset class by specifying a flag '--dataset_mode dummy'.
+See our template dataset class 'template_dataset.py' for more details.
+"""
+
+import pickle
+import importlib
+import torch.utils.data
+from .base_dataset import BaseDataset
+from .one_dataset import *
+
+__all__ = [OneDataset]
+
+
+def find_dataset_by_name(dataset_name: str):
+    """按照数据集名称来寻找所对应的dataset类进行动态导入
+    Import the module "data/[dataset_name]_dataset.py".
+
+    In the file, the class called DatasetNameDataset() will
+    be instantiated. It has to be a subclass of BaseDataset,
+    and it is case-insensitive.
+    """
+    dataset_filename = "data." + dataset_name + "_dataset"
+    datasetlib = importlib.import_module(dataset_filename)
+
+    dataset = None
+    target_dataset_name = dataset_name.replace("_", "") + "dataset"
+    for name, cls in datasetlib.__dict__.items():
+        if name.lower() == target_dataset_name.lower() and issubclass(cls, BaseDataset):
+            dataset = cls
+            
+    if dataset is None:
+        raise NotImplementedError(f"In {dataset_filename}.py, there should be a subclass of BaseDataset with class " f"name that matches {target_dataset_name} in lowercase.")
+    return dataset
+
+
+def get_option_setter(dataset_name):
+    """Return the static method <modify_commandline_options> of the dataset class."""
+    dataset_class = find_dataset_by_name(dataset_name)
+    return dataset_class.modify_commandline_options
+
+
+def create_dataset(opt):
+    """Create a dataset given the option.
+
+    This function wraps the class CustomDatasetDataLoader.
+        This is the main interface between this package and 'train.py'/'test.py'
+
+    Example:
+        >>> from data import create_dataset
+        >>> dataset = create_dataset(opt)
+    """
+    data_loader = CustomDatasetDataLoader(opt)
+    dataset = data_loader.load_data()
+    return dataset
+
+
+class CustomDatasetDataLoader:
+    """Wrapper class of Dataset class that performs multi-threading data loading"""
+
+    def __init__(self, opt):
+        """Initialize this class
+
+        Step 1: create a dataset instance given the name [dataset_mode]
+        Step 2: create a multi-threading data loader.
+        """
+        self.opt = opt
+        dataset_file = f"datasets/{opt.name}.pkl"
+        if not Path(dataset_file).exists():
+            # 判断数据集类型（成对/不成对），得到相应的类包
+            dataset_class = find_dataset_by_name(opt.dataset_mode)
+            # 传入数据集路径到类包中，得到数据集
+            self.dataset = dataset_class(opt)
+            # 打包下次直接使用
+            # 打包后文件也很大，暂时就这样
+            print("pickle dump dataset...")
+            pickle.dump(self.dataset, open(dataset_file, 'wb'))
+        else:
+            print("pickle load dataset...")
+            self.dataset = pickle.load(open(dataset_file, 'rb'))
+        print("dataset [%s] was created" % type(self.dataset).__name__)
+
+        self.dataloader = torch.utils.data.DataLoader(
+            self.dataset,
+            batch_size=opt.batch_size,
+            shuffle=not opt.serial_batches,
+            num_workers=int(opt.num_threads),
+        )
+
+    def load_data(self):
+        print(f"The number of training images = {len(self)}")
+        return self
+
+    def __iter__(self):
+        """Return a batch of data"""
+        for i, data in enumerate(self.dataloader):
+            if i * self.opt.batch_size >= self.opt.max_dataset_size:
+                break
+            yield data
+
+    def __len__(self):
+        """Return the number of data in the dataset"""
+        return min(len(self.dataset), self.opt.max_dataset_size)

data/base_dataset.py

@@ -0,0 +1,231 @@
+"""This module implements an abstract base class (ABC) 'BaseDataset' for datasets.
+
+It also includes common transformation functions (e.g., get_transform, __scale_width), which can be later used in
+subclasses.
+
+"""
+import random
+import numpy as np
+import torch.utils.data as data
+import torch
+from PIL import Image
+import torchvision.transforms as transforms
+from abc import ABC, abstractmethod
+
+
+class BaseDataset(data.Dataset, ABC):
+    """This class is an abstract base class (ABC) for datasets.
+
+    To create a subclass, you need to implement the following four functions:
+    -- <__init__>:                      initialize the class, first call BaseDataset.__init__(self, opt).
+    -- <__len__>:                       return the size of dataset.
+    -- <__getitem__>:                   get a data point.
+    -- <modify_commandline_options>:    (optionally) add dataset-specific options and set default options.
+    """
+
+    def __init__(self, opt):
+        """Initialize the class; save the options in the class
+
+        Parameters:
+            opt (Option class)-- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        self.opt = opt
+        self.root = opt.dataroot
+
+    @staticmethod
+    def modify_commandline_options(parser, is_train):
+        """用于添加针对这个数据集特定的选项，这个脚本里头只是一个样例。
+
+        Parameters:
+            parser          -- original option parser
+            parser:
+            is_train (bool) -- whether training phase or test phase.
+
+        Returns:
+            the modified parser.
+        """
+        return parser
+
+    @abstractmethod
+    def __len__(self):
+        """Return the total number of images in the dataset."""
+        return 0
+
+    @abstractmethod
+    def __getitem__(self, index):
+        """Return a data point and its metadata information.
+
+        Parameters:
+            index - - a random integer for data indexing
+
+        Returns:
+            a dictionary of data with their names. It usually contains the data itself and its metadata information.
+        """
+        pass
+
+
+def get_params(opt, size):
+    w, h = size
+    new_h = h
+    new_w = w
+    if opt.preprocess == "resize_and_crop":
+        new_h = new_w = opt.load_size
+    elif opt.preprocess == "scale_width_and_crop":
+        new_w = opt.load_size
+        new_h = opt.load_size * h // w
+
+    x = random.randint(0, np.maximum(0, new_w - opt.crop_size))
+    y = random.randint(0, np.maximum(0, new_h - opt.crop_size))
+
+    flip = random.random() > 0.5
+
+    return {"crop_pos": (x, y), "flip": flip}
+
+
+def get_transform(
+    opt,
+    params=None,
+    grayscale=False,
+    convert=True,
+    method=transforms.InterpolationMode.BICUBIC,
+):
+    """数据预处理"""
+    transform_list = []
+
+    # 灰度化
+    if grayscale:
+        transform_list.append(transforms.Grayscale(1))
+
+    # 图片大小调整
+    # 默认：双三次插值
+    if "resize" in opt.preprocess:
+        osize = [opt.load_size, opt.load_size]
+        transform_list.append(transforms.Resize(osize, method))
+    elif "scale_width" in opt.preprocess:
+        transform_list.append(
+            transforms.Lambda(
+                lambda img: __scale_width(img, opt.load_size, opt.crop_size, method)
+            )
+        )
+
+    # 裁剪
+    if "crop" in opt.preprocess:
+        if params is None:
+            transform_list.append(transforms.RandomCrop(opt.crop_size))
+        else:
+            transform_list.append(
+                transforms.Lambda(
+                    lambda img: __crop(img, params["crop_pos"], opt.crop_size)
+                )
+            )
+    if opt.preprocess == "none":
+        transform_list.append(
+            transforms.Lambda(lambda img: __make_power_2(img, base=4, method=method))
+        )
+
+    # 图片左右翻转
+    if not opt.no_flip:
+        if params is None:
+            transform_list.append(transforms.RandomHorizontalFlip())
+        elif params["flip"]:
+            transform_list.append(
+                transforms.Lambda(lambda img: __flip(img, params["flip"]))
+            )
+
+    # convert
+    if convert:
+        transform_list += [transforms.ToTensor()]
+        transform_list += [GaussionNoise()] if opt.isTrain else []
+        if grayscale:
+            transform_list += [transforms.Normalize((0.5,), (0.5,))]
+        else:
+            transform_list += [transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]
+    return transforms.Compose(transform_list)
+
+
+def __transforms2pil_resize(method):
+    mapper = {
+        transforms.InterpolationMode.BILINEAR: Image.BILINEAR,
+        transforms.InterpolationMode.BICUBIC: Image.BICUBIC,
+        transforms.InterpolationMode.NEAREST: Image.NEAREST,
+        transforms.InterpolationMode.LANCZOS: Image.LANCZOS,
+    }
+    return mapper[method]
+
+
+def __make_power_2(img, base, method=transforms.InterpolationMode.BICUBIC):
+    """根据给定的方法（例如：双三次插值���，将图片变成指定的大小。
+    其中的round函数是一种四舍五入的方法。
+    """
+    method = __transforms2pil_resize(method)
+    ow, oh = img.size
+    h = int(round(oh / base) * base)
+    w = int(round(ow / base) * base)
+    if h == oh and w == ow:
+        return img
+
+    __print_size_warning(ow, oh, w, h)
+    return img.resize((w, h), method)
+
+
+def __scale_width(
+    img, target_size, crop_size, method=transforms.InterpolationMode.BICUBIC
+):
+    """调整大小"""
+    method = __transforms2pil_resize(method)
+    ow, oh = img.size
+    if ow == target_size and oh >= crop_size:
+        return img
+    w = target_size
+    h = int(max(target_size * oh / ow, crop_size))
+    return img.resize((w, h), method)
+
+
+def __crop(img, pos, size):
+    """图片裁剪"""
+    ow, oh = img.size
+    x1, y1 = pos
+    tw = th = size
+    if ow > tw or oh > th:
+        return img.crop((x1, y1, x1 + tw, y1 + th))
+    return img
+
+
+def __flip(img, flip):
+    """图片左右翻转"""
+    if flip:
+        return img.transpose(Image.FLIP_LEFT_RIGHT)
+    return img
+
+
+def _gaussion_noise(img):
+    noise = torch.randn(img.shape)
+    img = img + noise * 0.1
+    return img
+
+
+def __print_size_warning(ow, oh, w, h):
+    """Print warning information about image size(only print once)"""
+    if not hasattr(__print_size_warning, "has_printed"):
+        print(
+            "The image size needs to be a multiple of 4. "
+            "The loaded image size was (%d, %d), so it was adjusted to "
+            "(%d, %d). This adjustment will be done to all images "
+            "whose sizes are not multiples of 4" % (ow, oh, w, h)
+        )
+        __print_size_warning.has_printed = True
+
+
+class GaussionNoise:
+    """添加高斯噪声"""
+
+    def __init__(self) -> None:
+        pass
+
+    def __call__(self, img):
+        noise = torch.randn(img.shape)
+        img_mix_noise = img + noise * 0.1
+        return img_mix_noise
+
+    def __repr__(self) -> str:
+        return f"{self.__class__.__name__}()"

data/image_folder.py

@@ -0,0 +1,62 @@
+"""A modified image folder class
+
+We modify the official PyTorch image folder (https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py)
+so that this class can load images from both current directory and its subdirectories.
+"""
+
+import torch.utils.data as data
+
+from PIL import Image
+import os
+from pathlib import Path
+
+
+IMG_EXTENSIONS = [".jpg", ".JPG", ".jpeg", ".JPEG", ".png", ".PNG", ".ppm", ".PPM", ".bmp", ".BMP", ".tif", ".TIF", ".tiff", ".TIFF"]
+
+
+def is_image_file(filename):
+    return any(filename.endswith(extension) for extension in IMG_EXTENSIONS)
+
+
+def make_dataset(dir, max_dataset_size=float("inf")):
+    images = []
+    assert os.path.isdir(dir), "%s is not a valid directory" % dir
+
+    for root, _, fnames in sorted(os.walk(dir)):
+        for fname in fnames:
+            if is_image_file(fname):
+                path = os.path.join(root, fname)
+                images.append(path)
+    return images[: min(max_dataset_size, len(images))]
+
+
+def default_loader(path):
+    return Image.open(path).convert("RGB")
+
+
+class ImageFolder(data.Dataset):
+    """根据文件夹制作数据集"""
+
+    def __init__(self, root, transform=None, return_paths=False, loader=default_loader):
+        imgs = make_dataset(root)
+        if len(imgs) == 0:
+            raise (RuntimeError("Found 0 images in: " + root + "\n" "Supported image extensions are: " + ",".join(IMG_EXTENSIONS)))
+
+        self.root = root
+        self.imgs = imgs
+        self.transform = transform
+        self.return_paths = return_paths
+        self.loader = loader
+
+    def __getitem__(self, index):
+        path = self.imgs[index]
+        img = self.loader(path)
+        if self.transform is not None:
+            img = self.transform(img)
+        if self.return_paths:
+            return img, path
+        else:
+            return img
+
+    def __len__(self):
+        return len(self.imgs)

data/one_dataset.py

@@ -0,0 +1,40 @@
+from .base_dataset import BaseDataset, get_transform
+from PIL import Image
+from pathlib import Path
+
+
+class OneDataset(BaseDataset):
+    """
+    加载数据
+
+    加载文件夹中所有图片或直接加载指定文件
+    """
+
+    def __init__(self, img, opt):
+        BaseDataset.__init__(self, opt)
+        # self.opt = opt
+        dataroot = img
+        if type(dataroot) == str:
+            dataroot = Path(dataroot)
+            if dataroot.is_file():
+                self.A_path = [str(dataroot)]
+            if dataroot.is_dir():
+                self.A_path = [str(i) for i in list(dataroot.iterdir())]
+            self.A_img = [Image.open(path).convert("RGB") for path in self.A_path]
+        else:  # dataroot 传入的直接是PIL格式图片
+            self.A_path = [None]
+            self.A_img = [dataroot]
+
+    def __getitem__(self, idx:int):
+        A_path = self.A_path[idx]
+        A_img = self.A_img[idx]
+        A = transform(A_img, self.opt)
+        return {"A": A, "A_paths": A_path}
+
+    def __len__(self):
+        return 1
+
+
+def transform(img, opt):
+    fn_transform = get_transform(opt, grayscale=(opt.input_nc == 1))
+    return fn_transform(img)

data/single_dataset.py

@@ -0,0 +1,40 @@
+from data.base_dataset import BaseDataset, get_transform
+from data.image_folder import make_dataset
+from PIL import Image
+
+
+class SingleDataset(BaseDataset):
+    """This dataset class can load a set of images specified by the path --dataroot /path/to/data.
+
+    It can be used for generating CycleGAN results only for one side with the model option '-model test'.
+    """
+
+    def __init__(self, opt):
+        """Initialize this dataset class.
+
+        Parameters:
+            opt (Option class) -- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        BaseDataset.__init__(self, opt)
+        self.A_paths = sorted(make_dataset(opt.dataroot, opt.max_dataset_size))
+        input_nc = self.opt.output_nc if self.opt.direction == 'BtoA' else self.opt.input_nc
+        self.transform = get_transform(opt, grayscale=(input_nc == 1))
+
+    def __getitem__(self, index):
+        """Return a data point and its metadata information.
+
+        Parameters:
+            index - - a random integer for data indexing
+
+        Returns a dictionary that contains A and A_paths
+            A(tensor) - - an image in one domain
+            A_paths(str) - - the path of the image
+        """
+        A_path = self.A_paths[index]
+        A_img = Image.open(A_path).convert('RGB')
+        A = self.transform(A_img)
+        return {'A': A, 'A_paths': A_path}
+
+    def __len__(self):
+        """Return the total number of images in the dataset."""
+        return len(self.A_paths)

data/unaligned_dataset.py

@@ -0,0 +1,72 @@
+import os
+from pathlib import Path
+from data.base_dataset import BaseDataset, get_transform
+from data.image_folder import make_dataset
+from PIL import Image
+import random
+
+
+class UnalignedDataset(BaseDataset):
+    """
+    This dataset class can load unaligned/unpaired datasets.
+
+    It requires two directories to host training images from domain A '/path/to/data/trainA'
+    and from domain B '/path/to/data/trainB' respectively.
+    You can train the model with the dataset flag '--dataroot /path/to/data'.
+    Similarly, you need to prepare two directories:
+    '/path/to/data/testA' and '/path/to/data/testB' during test time.
+    """
+
+    def __init__(self, opt):
+        """Initialize this dataset class.
+
+        Parameters:
+            opt (Option class) -- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        BaseDataset.__init__(self, opt)
+        self.dir_A = os.path.join(opt.dataroot, opt.phase + 'A')  # create a path '/path/to/data/trainA'
+        self.dir_B = os.path.join(opt.dataroot, opt.phase + 'B')  # create a path '/path/to/data/trainB'
+
+        self.A_paths = sorted(make_dataset(self.dir_A, opt.max_dataset_size))   # load images from '/path/to/data/trainA'
+        self.B_paths = sorted(make_dataset(self.dir_B, opt.max_dataset_size))    # load images from '/path/to/data/trainB'
+        self.A_size = len(self.A_paths)  # get the size of dataset A
+        self.B_size = len(self.B_paths)  # get the size of dataset B
+        btoA = self.opt.direction == 'BtoA'
+        input_nc = self.opt.output_nc if btoA else self.opt.input_nc       # get the number of channels of input image
+        output_nc = self.opt.input_nc if btoA else self.opt.output_nc      # get the number of channels of output image
+        self.transform_A = get_transform(self.opt, grayscale=(input_nc == 1))
+        self.transform_B = get_transform(self.opt, grayscale=(output_nc == 1))
+
+    def __getitem__(self, index):
+        """Return a data point and its metadata information.
+
+        Parameters:
+            index (int)      -- a random integer for data indexing
+
+        Returns a dictionary that contains A, B, A_paths and B_paths
+            A (tensor)       -- an image in the input domain
+            B (tensor)       -- its corresponding image in the target domain
+            A_paths (str)    -- image paths
+            B_paths (str)    -- image paths
+        """
+        A_path = self.A_paths[index % self.A_size]  # make sure index is within then range
+        if self.opt.serial_batches:   # make sure index is within then range
+            index_B = index % self.B_size
+        else:   # randomize the index for domain B to avoid fixed pairs.
+            index_B = random.randint(0, self.B_size - 1)
+        B_path = self.B_paths[index_B]
+        A_img = Image.open(A_path).convert('RGB')
+        B_img = Image.open(B_path).convert('RGB')
+        # apply image transformation
+        A = self.transform_A(A_img)
+        B = self.transform_B(B_img)
+
+        return {'A': A, 'B': B, 'A_paths': A_path, 'B_paths': B_path}
+
+    def __len__(self):
+        """Return the total number of images in the dataset.
+
+        As we have two datasets with potentially different number of images,
+        we take a maximum of them
+        """
+        return max(self.A_size, self.B_size)

detect.py

@@ -0,0 +1,66 @@
+"""
+>>> python detect.py --dataroot ./imgs/horse.jpg --style horse2zebra
+"""
+
+import os
+import sys
+
+BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+sys.path.append(BASE_DIR)
+from pathlib import Path
+
+from data import OneDataset
+from models import create_model
+from options import DetectOptions
+from util import tensor2im, save_image, show_image, now_time, print_info
+
+# 定义参数
+opt = DetectOptions().parse()
+# 硬编码一些测试参数
+opt.num_threads = 0  # 测试代码仅支持num_threads = 0
+opt.batch_size = 1  # 测试代码仅支持batch_size = 1
+opt.serial_batches = True  # 禁用数据混洗；如果需要在随机选择的图像上得到结果，请取消对此行的注释。
+opt.no_flip = True  # 不翻转；如果需要在翻转的图像上得到结果，请取消对此行的注释。
+opt.display_id = -1  # 没有visdom显示；测试代码将结果保存到HTML文件中。
+
+#  加载模型
+model = create_model(opt)
+
+
+def detect(img=opt.dataroot, style=opt.style):
+    result = None
+    time_info = "-" * 30 + f"\n{now_time()}:start"
+    dataset = OneDataset(img, opt)  # 加载数据
+    model.setup(opt, style)  # 设置模型
+    model.eval()  # 切换到评估模式
+    print_info(time_info)
+    for _, data in enumerate(dataset):
+        model.set_input(data)  # 从数据加载器中解包数据
+        model.test()  # 推理
+        visuals = model.get_current_visuals()  # 获取结果图像
+        result = tensor2im(visuals["fake"])
+        time_info = f"{now_time()}:done\n" + "-" * 30
+        print_info(time_info)
+    return result
+
+
+def save_detect_img(results: list, no_save_img=False):
+    """保存或展示检测结果"""
+    # 保存 or 展示
+    if no_save_img:
+        for _, img_fake in results:
+            show_image(img_fake)
+    else:
+        for img_path, img_fake in results:
+            save_dir = Path.cwd().joinpath("results")  # fake图片保存路径
+            img_path = Path(img_path)
+            img_name = img_path.stem + "_fake" + img_path.suffix
+            Path.mkdir(save_dir, exist_ok=True)
+            save_path = save_dir.joinpath(img_name)
+            save_image(img_fake, save_path)
+            print("results_path: ", save_path)
+
+
+if __name__ == "__main__":
+    results = detect()  # 推理图片
+    save_detect_img(results, opt.no_save_img)  # 保存检测结果

models/__init__.py

@@ -0,0 +1,66 @@
+"""This package contains modules related to objective functions, optimizations, and network architectures.
+
+To add a custom model class called 'dummy', you need to add a file called 'dummy_model.py' and define a subclass DummyModel inherited from BaseModel.
+You need to implement the following five functions:
+    -- <__init__>:                      initialize the class; first call BaseModel.__init__(self, opt).
+    -- <set_input>:                     unpack data from dataset and apply preprocessing.
+    -- <forward>:                       produce intermediate results.
+    -- <optimize_parameters>:           calculate loss, gradients, and update network weights.
+    -- <modify_commandline_options>:    (optionally) add model-specific options and set default options.
+
+In the function <__init__>, you need to define four lists:
+    -- self.loss_names (str list):          specify the training losses that you want to plot and save.
+    -- self.net_names (str list):         define networks used in our training.
+    -- self.visual_names (str list):        specify the images that you want to display and save.
+    -- self.optimizers (optimizer list):    define and initialize optimizers. You can define one optimizer for each network. If two networks are updated at the same time, you can use itertools.chain to group them. See cycle_gan_model.py for an usage.
+
+Now you can use the model class by specifying flag '--model dummy'.
+See our template model class 'template_model.py' for more details.
+"""
+
+import importlib
+from .base_model import BaseModel
+
+
+def find_model_using_name(model_name):
+    """Import the module "models/[model_name]_model.py".
+
+    In the file, the class called DatasetNameModel() will
+    be instantiated. It has to be a subclass of BaseModel,
+    and it is case-insensitive.
+    """
+    model_filename = "models." + model_name + "_model"
+    modellib = importlib.import_module(model_filename)
+    model = None
+    target_model_name = model_name.replace("_", "") + "model"
+    for name, cls in modellib.__dict__.items():
+        if name.lower() == target_model_name.lower() and issubclass(cls, BaseModel):
+            model = cls
+
+    if model is None:
+        print("In %s.py, there should be a subclass of BaseModel with class name that matches %s in lowercase." % (model_filename, target_model_name))
+        exit(0)
+
+    return model
+
+
+def get_option_setter(model_name):
+    """Return the static method <modify_commandline_options> of the model class."""
+    model_class = find_model_using_name(model_name)
+    return model_class.modify_commandline_options
+
+
+def create_model(opt):
+    """Create a model given the option.
+
+    This function warps the class CustomDatasetDataLoader.
+    This is the main interface between this package and 'train.py'/'test.py'
+
+    Example:
+        >>> from models import create_model
+        >>> model = create_model(opt)
+    """
+    model = find_model_using_name(opt.model)
+    instance = model(opt)
+    print("model [%s] was created" % type(instance).__name__)
+    return instance

models/base_model.py

@@ -0,0 +1,213 @@
+import os
+from abc import ABC, abstractmethod
+from collections import OrderedDict
+from pathlib import Path
+
+import torch
+
+from . import networks
+
+
+class BaseModel(ABC):
+    """This class is an abstract base class (ABC) for models.
+    To create a subclass, you need to implement the following five functions:
+        -- <__init__>:                      initialize the class; first call BaseModel.__init__(self, opt).
+        -- <set_input>:                     unpack data from dataset and apply preprocessing.
+        -- <forward>:                       produce intermediate results.
+        -- <optimize_parameters>:           calculate losses, gradients, and update network weights.
+        -- <modify_commandline_options>:    (optionally) add model-specific options and set default options.
+    """
+
+    def __init__(self, opt):
+        """Initialize the BaseModel class.
+
+        Parameters:
+            opt (Option class)-- stores all the experiment flags; needs to be a subclass of BaseOptions
+
+        When creating your custom class, you need to implement your own initialization.
+        In this function, you should first call <BaseModel.__init__(self, opt)>
+        Then, you need to define four lists:
+            -- self.loss_names (str list): specify the training losses that you want to plot and save.
+            -- self.net_names (str list): define networks used in our training.
+            -- self.visual_names (str list): specify the images that you want to display and save.
+            -- self.optimizers (optimizer list): define and initialize optimizers.
+            You can define one optimizer for each network.
+            If two networks are updated at the same time, you can use itertools. Chain to group them.
+            See cycle_gan_model.py for an example.
+        """
+        self.opt = opt
+        self.gpu_ids = opt.gpu_ids
+        self.isTrain = opt.isTrain
+        self.device = (
+            torch.device("cuda:{}".format(self.gpu_ids[0]))
+            if self.gpu_ids
+            else torch.device("cpu")
+        )
+        print(self.device)
+        self.save_dir = Path(opt.checkpoints_dir).joinpath(
+            opt.name
+        )  # save all the checkpoints to save_dir
+        # with [scale_width], input images might have different sizes, which hurts the performance of cudnn.benchmark.
+        if opt.preprocess != "scale_width":
+            torch.backends.cudnn.benchmark = True
+        self.loss_names = []
+        self.net_names = []
+        self.visual_names = []
+        self.optimizers = []
+        self.image_paths = []
+        self.metric = 0  # used for learning rate policy 'plateau'
+
+    @staticmethod
+    def modify_commandline_options(parser, is_train):
+        """Add new model-specific options, and rewrite default values for existing options.
+
+        Parameters:
+            parser          -- original option parser
+            parser:
+            is_train (bool) -- whether training phase or test phase. You can use this flag to add training-specific or test-specific options.
+
+        Returns:
+            the modified parser.
+        """
+        return parser
+
+    def setup(self, opt, load_weight=None):
+        """加载和打印网络；创建调度程序
+
+        Parameters:
+            load_weight:
+            opt (Option class) -- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        if self.isTrain:
+            self.schedulers = [
+                networks.get_scheduler(optimizer, opt) for optimizer in self.optimizers
+            ]
+        if not self.isTrain or opt.continue_train:
+            load_suffix = "iter_%d" % opt.load_iter if opt.load_iter > 0 else opt.epoch
+            self.load_networks(load_suffix, load_weight)
+        self.print_networks(opt.verbose)
+
+    def eval(self):
+        """Make models eval mode during test time"""
+        for name in self.net_names:
+            if isinstance(name, str):
+                net = getattr(self, "net_" + name)
+                net.eval()
+
+    def test(self):
+        """Forward function used in test time.
+
+        This function wraps <forward> function in no_grad() so we don't save intermediate steps for backprop
+        It also calls <compute_visuals> to produce additional visualization results
+        """
+        with torch.no_grad():
+            self.forward()
+            self.compute_visuals()
+
+    def compute_visuals(self):
+        """Calculate additional output images for visdom and HTML visualization"""
+        pass
+
+    def get_current_visuals(self):
+        """Return visualization images. train.py will display these images with visdom, and save the images to an HTML"""
+        visual_ret = OrderedDict()
+        for name in self.visual_names:
+            if isinstance(name, str):
+                visual_ret[name] = getattr(self, name)
+        return visual_ret
+
+    def get_image_paths(self):
+        """Return image paths that are used to load current data"""
+        return self.image_paths
+
+    def save_networks(self, epoch):
+        """Save all the networks to the disk.
+
+        Parameters:
+            epoch (int) -- current epoch; used in the file name '%s_net_%s.pth' % (epoch, name)
+        """
+        for name in self.net_names:
+            if isinstance(name, str):
+                save_filename = "%s_net_%s.pth" % (epoch, name)
+                save_path = Path(self.save_dir, save_filename)
+                net = getattr(self, "net_" + name)
+
+                if len(self.gpu_ids) > 0 and torch.cuda.is_available():
+                    torch.save(net.module.cpu().state_dict(), save_path)
+                    net.cuda(self.gpu_ids[0])
+                else:
+                    torch.save(net.cpu().state_dict(), save_path)
+
+    def __patch_instance_norm_state_dict(self, state_dict, module, keys, i=0):
+        """Fix InstanceNorm checkpoints incompatibility (prior to 0.4)"""
+        key = keys[i]
+        if i + 1 == len(keys):  # at the end, pointing to a parameter/buffer
+            if module.__class__.__name__.startswith("InstanceNorm") and (
+                key == "running_mean" or key == "running_var"
+            ):
+                if getattr(module, key) is None:
+                    state_dict.pop(".".join(keys))
+            if module.__class__.__name__.startswith("InstanceNorm") and (
+                key == "num_batches_tracked"
+            ):
+                state_dict.pop(".".join(keys))
+        else:
+            self.__patch_instance_norm_state_dict(
+                state_dict, getattr(module, key), keys, i + 1
+            )
+
+    def load_networks(self, epoch: int, load_weight=None):
+        """Load all the networks from the disk.
+
+        Parameters:
+            load_weight:
+            epoch (int) -- current epoch; used in the file name '%s_net_%s.pth' % (epoch, name)
+        """
+        for name in self.net_names:
+            if isinstance(name, str):
+                if not load_weight:
+                    load_filename = "%s_net_%s.pth" % (epoch, name)
+                else:
+                    load_filename = f"{load_weight}.pth"
+                load_path = self.save_dir.joinpath(load_filename)
+                # if not load_path.exists():
+                #     load_path = "./weights/pre/latest_net_G.pth"
+                net = getattr(self, "net_" + name)
+                if isinstance(net, torch.nn.DataParallel):
+                    net = net.module
+                print("loading the model from %s" % load_path)
+                # if you are using PyTorch newer than 0.4 (e.g., built from
+                # GitHub source), you can remove str() on self.device
+                state_dict = torch.load(load_path, map_location=self.device)
+                if hasattr(state_dict, "_metadata"):
+                    del state_dict._metadata
+                # patch InstanceNorm checkpoints prior to 0.4
+                # need to copy keys here because we mutate in the loop
+                for key in list(state_dict.keys()):
+                    self.__patch_instance_norm_state_dict(
+                        state_dict, net, key.split(".")
+                    )
+                net.load_state_dict(state_dict)
+                # net.half()
+                # 已经是float16了 /(ㄒoㄒ)/~~，没有float8
+
+    def print_networks(self, verbose):
+        """Print the total number of parameters in the network and (if verbose) network architecture
+
+        Parameters:
+            verbose (bool) -- if verbose: print the network architecture
+        """
+        print("---------- Networks initialized -------------")
+        for name in self.net_names:
+            if isinstance(name, str):
+                net = getattr(self, "net_" + name)
+                num_params = 0
+                for param in net.parameters():
+                    num_params += param.numel()
+                if verbose:
+                    print(net)
+                print(
+                    "[Network %s] Total number of parameters : %.3f M"
+                    % (name, num_params / 1e6)
+                )
+        print("-----------------------------------------------")

models/cycle_gan_model.py

@@ -0,0 +1,170 @@
+import itertools
+
+import torch
+from util.image_pool import ImagePool
+
+from . import networks
+from .base_model import BaseModel
+
+
+class CycleGANModel(BaseModel):
+    """
+    This class implements the CycleGAN model, for learning image-to-image translation without paired data.
+
+    The model training requires '--dataset_mode unaligned' dataset.
+    By default, it uses a '--netG resnet_9blocks' ResNet generator,
+    a '--netD basic' discriminator (PatchGAN introduced by pix2pix),
+    and a least-square GANs objective ('--gan_mode lsgan').
+
+    CycleGAN paper: https://arxiv.org/pdf/1703.10593.pdf
+    """
+
+    @staticmethod
+    def modify_commandline_options(parser, is_train=True):
+        """Add new dataset-specific options, and rewrite default values for existing options.
+
+        Parameters:
+            parser          -- original option parser
+            parser:
+            is_train (bool) -- whether training phase or test phase. You can use this flag to add training-specific or test-specific options.
+
+        Returns:
+            the modified parser.
+
+        For CycleGAN, in addition to GAN losses, we introduce lambda_A, lambda_B, and lambda_identity for the following losses.
+        A (source domain), B (target domain).
+        Generators: G_A: A -> B; G_B: B -> A.
+        Discriminators: D_A: G_A(A) vs. B; D_B: G_B(B) vs. A.
+        Forward cycle loss:  lambda_A * ||G_B(G_A(A)) - A|| (Eqn. (2) in the paper)
+        Backward cycle loss: lambda_B * ||G_A(G_B(B)) - B|| (Eqn. (2) in the paper)
+        Identity loss (optional): lambda_identity * (||G_A(B) - B|| * lambda_B + ||G_B(A) - A|| * lambda_A) (Sec 5.2 "Photo generation from paintings" in the paper)
+        Dropout is not used in the original CycleGAN paper.
+        """
+        parser.set_defaults(no_dropout=True)  # default CycleGAN did not use dropout
+        if is_train:
+            parser.add_argument(
+                "--lambda_A",
+                type=float,
+                default=10.0,
+                help="weight for cycle loss (A -> B -> A)",
+            )
+            parser.add_argument(
+                "--lambda_B",
+                type=float,
+                default=10.0,
+                help="weight for cycle loss (B -> A -> B)",
+            )
+            parser.add_argument(
+                "--lambda_identity",
+                type=float,
+                default=0.5,
+                help="use identity mapping. Setting lambda_identity other than 0 has an effect of scaling the weight of the identity mapping loss. For example, if the weight of the identity loss should be 10 times smaller than the weight of the reconstruction loss, please set lambda_identity = 0.1",
+            )
+        return parser
+
+    def __init__(self, opt):
+        """Initialize the CycleGAN class.
+
+        Parameters:
+            opt (Option class)-- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        BaseModel.__init__(self, opt)
+        # specify the images you want to save/display. The training/test scripts will call <BaseModel.get_current_visuals>
+        visual_names_A = ["real_A", "fake_B", "rec_A"]
+        visual_names_B = ["real_B", "fake_A", "rec_B"]
+
+        if (
+            self.isTrain and self.opt.lambda_identity > 0.0
+        ):  # if identity loss is used, we also visualize idt_B=G_A(B) ad idt_A=G_A(B)
+            visual_names_A.append("idt_B")
+            visual_names_B.append("idt_A")
+        self.visual_names = (
+            visual_names_A + visual_names_B
+        )  # combine visualizations for A and B
+        # specify the models you want to save to the disk. The training/test scripts will call <BaseModel.save_networks> and <BaseModel.load_networks>.
+        self.net_names = ["G_A", "G_B"]
+        if self.isTrain:
+            self.net_names.extend(["D_A", "D_B"])
+        # 下面会根据 self.loss_names self.visual_names net_names 中定义的字符串创建对应的变量名
+        # 这样把变量名写在一个列表中而不用字典，通过字典取得变量的写法可能是为了避免代码写的太长？
+        # 关键字 exec 可以根据字符串新建变量
+        # 用法：
+        # varlist = ["a"]
+        # exec
+
+        # define networks (both Generators and discriminators)
+        # The naming is different from those used in the paper.
+        # Code (vs. paper): G_A (G), G_B (F), D_A (D_Y), D_B (D_X)
+        self.net_G_A = networks.define_G(
+            opt.input_nc,
+            opt.output_nc,
+            opt.ngf,
+            opt.netG,
+            opt.norm,
+            not opt.no_dropout,
+            opt.init_type,
+            opt.init_gain,
+            self.gpu_ids,
+        )
+        self.net_G_B = networks.define_G(
+            opt.output_nc,
+            opt.input_nc,
+            opt.ngf,
+            opt.netG,
+            opt.norm,
+            not opt.no_dropout,
+            opt.init_type,
+            opt.init_gain,
+            self.gpu_ids,
+        )
+
+        if self.isTrain:  # define discriminators
+            self.net_D_A = networks.define_D(
+                opt.output_nc,
+                opt.ndf,
+                opt.netD,
+                opt.n_layers_D,
+                opt.norm,
+                opt.init_type,
+                opt.init_gain,
+                self.gpu_ids,
+            )
+            self.net_D_B = networks.define_D(
+                opt.input_nc,
+                opt.ndf,
+                opt.netD,
+                opt.n_layers_D,
+                opt.norm,
+                opt.init_type,
+                opt.init_gain,
+                self.gpu_ids,
+            )
+        if self.isTrain:
+            if (
+                opt.lambda_identity > 0.0
+            ):  # only works when input and output images have the same number of channels
+                assert opt.input_nc == opt.output_nc
+            self.fake_A_pool = ImagePool(
+                opt.pool_size
+            )  # create image buffer to store previously generated images
+            self.fake_B_pool = ImagePool(
+                opt.pool_size
+            )  # create image buffer to store previously generated images
+
+    def set_input(self, input):
+        """Unpack input data from the dataloader and perform necessary pre-processing steps.
+        Parameters:
+            input (dict): include the data itself and its metadata information.
+        The option 'direction' can be used to swap domain A and domain B.
+        """
+        AtoB = self.opt.direction == "AtoB"
+        self.real_A = input["A" if AtoB else "B"].to(self.device)
+        self.real_B = input["B" if AtoB else "A"].to(self.device)
+        self.image_paths = input["A_paths" if AtoB else "B_paths"]
+
+    def forward(self):
+        """Run forward pass; called by both functions <optimize_parameters> and <test>."""
+        self.fake_B = self.net_G_A(self.real_A)  # G_A(A)
+        self.rec_A = self.net_G_B(self.fake_B)  # G_B(G_A(A))
+        self.fake_A = self.net_G_B(self.real_B)  # G_B(B)
+        self.rec_B = self.net_G_A(self.fake_A)  # G_A(G_B(B))

models/networks.py

@@ -0,0 +1,767 @@
+import functools
+
+import torch
+import torch.nn as nn
+from torch.nn import init
+from torch.optim import lr_scheduler
+
+# 2023年2月23日 模型：卷积，残差
+###############################################################################
+
+
+class Identity(nn.Module):
+    def forward(self, x):
+        return x
+
+
+def get_norm_layer(norm_type="instance"):
+    """Return a normalization layer
+
+    Parameters:
+        norm_type (str) -- the name of the normalization layer: batch | instance | none
+
+    For BatchNorm, we use learnable affine parameters and track running statistics (mean/stddev).
+    For InstanceNorm, we do not use learnable affine parameters. We do not track running statistics.
+    """
+    dict_ = {
+        "batch": functools.partial(
+            nn.BatchNorm2d, affine=True, track_running_stats=True
+        ),
+        "instance": functools.partial(
+            nn.InstanceNorm2d, affine=False, track_running_stats=False
+        ),
+        "none": lambda x: Identity(),
+    }
+    # if norm_type == 'batch':
+    #     norm_layer = functools.partial(nn.BatchNorm2d, affine=True, track_running_stats=True)
+    # elif norm_type == 'instance':
+    #     norm_layer = functools.partial(nn.InstanceNorm2d, affine=False, track_running_stats=False)
+    # elif norm_type == 'none':
+    #     def norm_layer(x):
+    #         return Identity()
+    if norm_type in dict_:
+        norm_layer = dict_[norm_type]
+    else:
+        raise NotImplementedError("normalization layer [%s] is not found" % norm_type)
+    return norm_layer
+
+
+def get_scheduler(optimizer, opt):
+    """Return a learning rate scheduler
+
+    Parameters:
+        optimizer          -- the optimizer of the network
+        optimizer:
+        opt (option class) -- stores all the experiment flags; needs to be a subclass of BaseOptions．
+                              opt.lr_policy is the name of learning rate policy: linear | step | plateau | cosine
+
+    For 'linear', we keep the same learning rate for the first <opt.n_epochs> epochs
+    and linearly decay the rate to zero over the next <opt.n_epochs_decay> epochs.
+    For other schedulers (step, plateau, and cosine), we use the default PyTorch schedulers.
+    See https://pytorch.org/docs/stable/optim.html for more details.
+    """
+    if opt.lr_policy == "linear":
+
+        def lambda_rule(epoch):
+            lr_l = 1.0 - max(0, epoch + opt.epoch_count - opt.n_epochs) / float(
+                opt.n_epochs_decay + 1
+            )
+            return lr_l
+
+        scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lambda_rule)
+    elif opt.lr_policy == "step":
+        scheduler = lr_scheduler.StepLR(
+            optimizer, step_size=opt.lr_decay_iters, gamma=0.1
+        )
+    elif opt.lr_policy == "plateau":
+        scheduler = lr_scheduler.ReduceLROnPlateau(
+            optimizer, mode="min", factor=0.2, threshold=0.01, patience=5
+        )
+    elif opt.lr_policy == "cosine":
+        scheduler = lr_scheduler.CosineAnnealingLR(
+            optimizer, T_max=opt.n_epochs, eta_min=0
+        )
+    else:
+        return NotImplementedError(
+            "learning rate policy [%s] is not implemented", opt.lr_policy
+        )
+    return scheduler
+
+
+def init_weights(net, init_type="normal", init_gain=0.02):
+    """Initialize network weights.
+
+    Parameters:
+        net (network)   -- network to be initialized
+        init_type (str) -- the name of an initialization method: normal | xavier | kaiming | orthogonal
+        init_gain (float)    -- scaling factor for normal, xavier and orthogonal.
+
+    We use 'normal' in the original pix2pix and CycleGAN paper. But xavier and kaiming might
+    work better for some applications. Feel free to try yourself.
+    """
+
+    def init_func(m):  # define the initialization function
+        classname = m.__class__.__name__
+        if hasattr(m, "weight") and (
+            classname.find("Conv") != -1 or classname.find("Linear") != -1
+        ):
+            if init_type == "normal":
+                init.normal_(m.weight.data, 0.0, init_gain)
+            elif init_type == "xavier":
+                init.xavier_normal_(m.weight.data, gain=init_gain)
+            elif init_type == "kaiming":
+                init.kaiming_normal_(m.weight.data, a=0, mode="fan_in")
+            elif init_type == "orthogonal":
+                init.orthogonal_(m.weight.data, gain=init_gain)
+            else:
+                raise NotImplementedError(
+                    "initialization method [%s] is not implemented" % init_type
+                )
+            if hasattr(m, "bias") and m.bias is not None:
+                init.constant_(m.bias.data, 0.0)
+        elif (
+            classname.find("BatchNorm2d") != -1
+        ):  # BatchNorm Layer's weight is not a matrix; only normal distribution applies.
+            init.normal_(m.weight.data, 1.0, init_gain)
+            init.constant_(m.bias.data, 0.0)
+
+    print("initialize network with %s" % init_type)
+    net.apply(init_func)  # apply the initialization function <init_func>
+
+
+def init_net(net, init_type="normal", init_gain=0.02, gpu_ids=None):
+    """Initialize a network: 1. register CPU/GPU device (with multi-GPU support); 2. initialize the network weights
+    Parameters:
+        net (network)      -- the network to be initialized
+        init_type (str)    -- the name of an initialization method: normal | xavier | kaiming | orthogonal
+        init_gain (float)       -- scaling factor for normal, xavier and orthogonal.
+        gpu_ids (int list) -- which GPUs the network runs on: e.g., 0,1,2
+
+    Return an initialized network.
+    """
+    if gpu_ids is None:
+        gpu_ids = []
+    if len(gpu_ids) > 0:
+        assert torch.cuda.is_available()
+        net.to(gpu_ids[0])
+        net = torch.nn.DataParallel(net, gpu_ids)  # multi-GPUs
+    init_weights(net, init_type, init_gain=init_gain)
+    return net
+
+
+def define_G(
+    input_nc,
+    output_nc,
+    ngf,
+    netG,
+    norm="batch",
+    use_dropout=False,
+    init_type="normal",
+    init_gain=0.02,
+    gpu_ids=None,
+):
+    """Create a generator
+
+    Parameters:
+        input_nc (int) -- the number of channels in input images
+        output_nc (int) -- the number of channels in output images
+        ngf (int) -- the number of filters in the last conv layer
+        netG (str) -- the architecture's name: resnet_9blocks | resnet_6blocks | unet_256 | unet_128
+        norm (str) -- the name of normalization layers used in the network: batch | instance | none
+        use_dropout (bool) -- if use dropout layers.
+        init_type (str)    -- the name of our initialization method.
+        init_gain (float)  -- scaling factor for normal, xavier and orthogonal.
+        gpu_ids (int list) -- which GPUs the network runs on: e.g., 0,1,2
+
+    Returns a generator
+
+    Our current implementation provides two types of generators:
+        U-Net: [unet_128] (for 128x128 input images) and [unet_256] (for 256x256 input images)
+        The original U-Net paper: https://arxiv.org/abs/1505.04597
+
+        Resnet-based generator: [resnet_6blocks] (with 6 Resnet blocks) and [resnet_9blocks] (with 9 Resnet blocks)
+        Resnet-based generator consists of several Resnet blocks between a few downsampling/upsampling operations.
+        We adapt Torch code from Justin Johnson's neural style transfer project (https://github.com/jcjohnson/fast-neural-style).
+
+
+    The generator has been initialized by <init_net>. It uses RELU for non-linearity.
+    """
+    if gpu_ids is None:
+        gpu_ids = []
+    net = None
+    norm_layer = get_norm_layer(norm_type=norm)
+    net_dict = {
+        "resnet_9blocks": ResnetGenerator(
+            input_nc,
+            output_nc,
+            ngf,
+            norm_layer=norm_layer,
+            use_dropout=use_dropout,
+            n_blocks=9,
+        ),
+        "resnet_6blocks": ResnetGenerator(
+            input_nc,
+            output_nc,
+            ngf,
+            norm_layer=norm_layer,
+            use_dropout=use_dropout,
+            n_blocks=6,
+        ),
+        "unet_128": UnetGenerator(
+            input_nc, output_nc, 7, ngf, norm_layer=norm_layer, use_dropout=use_dropout
+        ),
+        "unet_256": UnetGenerator(
+            input_nc, output_nc, 8, ngf, norm_layer=norm_layer, use_dropout=use_dropout
+        ),
+    }
+
+    # if netG == "resnet_9blocks":
+    #     net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=9)
+    # elif netG == "resnet_6blocks":
+    #     net = ResnetGenerator(input_nc, output_nc, ngf, norm_layer=norm_layer, use_dropout=use_dropout, n_blocks=6)
+    # elif netG == "unet_128":
+    #     net = UnetGenerator(input_nc, output_nc, 7, ngf, norm_layer=norm_layer, use_dropout=use_dropout)
+    # elif netG == "unet_256":
+    #     net = UnetGenerator(input_nc, output_nc, 8, ngf, norm_layer=norm_layer, use_dropout=use_dropout)
+    if netG in net_dict:
+        net = net_dict[netG]
+    else:
+        raise NotImplementedError("Generator model name [%s] is not recognized" % netG)
+    return init_net(net, init_type, init_gain, gpu_ids)
+
+
+def define_D(
+    input_nc,
+    ndf,
+    netD,
+    n_layers_D=3,
+    norm="batch",
+    init_type="normal",
+    init_gain=0.02,
+    gpu_ids=None,
+):
+    """Create a discriminator
+
+    Parameters:
+        input_nc (int)     -- the number of channels in input images
+        ndf (int)          -- the number of filters in the first conv layer
+        netD (str)         -- the architecture's name: basic | n_layers | pixel
+        n_layers_D (int)   -- the number of conv layers in the discriminator; effective when netD=='n_layers'
+        norm (str)         -- the type of normalization layers used in the network.
+        init_type (str)    -- the name of the initialization method.
+        init_gain (float)  -- scaling factor for normal, xavier and orthogonal.
+        gpu_ids (int list) -- which GPUs the network runs on: e.g., 0,1,2
+
+    Returns a discriminator
+
+    Our current implementation provides three types of discriminators:
+        [basic]: 'PatchGAN' classifier described in the original pix2pix paper.
+        It can classify whether 70×70 overlapping patches are real or fake.
+        Such a patch-level discriminator architecture has fewer parameters
+        than a full-image discriminator and can work on arbitrarily-sized images
+        in a fully convolutional fashion.
+
+        [n_layers]: With this mode, you can specify the number of conv layers in the discriminator
+        with the parameter <n_layers_D> (default=3 as used in [basic] (PatchGAN).)
+
+        [pixel]: 1x1 PixelGAN discriminator can classify whether a pixel is real or not.
+        It encourages greater color diversity but has no effect on spatial statistics.
+
+    The discriminator has been initialized by <init_net>. It uses Leakly RELU for non-linearity.
+    """
+    if gpu_ids is None:
+        gpu_ids = []
+    net = None
+    norm_layer = get_norm_layer(norm_type=norm)
+    net_dict = {
+        "basic": NLayerDiscriminator(input_nc, ndf, n_layers=3, norm_layer=norm_layer),
+        "n_layers": NLayerDiscriminator(
+            input_nc, ndf, n_layers_D, norm_layer=norm_layer
+        ),
+        "pixel": PixelDiscriminator(input_nc, ndf, norm_layer=norm_layer),
+    }
+
+    # if netD == "basic":  # default PatchGAN classifier
+    #     net = NLayerDiscriminator(input_nc, ndf, n_layers=3, norm_layer=norm_layer)
+    # elif netD == "n_layers":  # more options
+    #     net = NLayerDiscriminator(input_nc, ndf, n_layers_D, norm_layer=norm_layer)
+    # elif netD == "pixel":  # classify if each pixel is real or fake
+    #     net = PixelDiscriminator(input_nc, ndf, norm_layer=norm_layer)
+    if netD in net_dict:
+        net = net_dict[netD]
+    else:
+        raise NotImplementedError(
+            "Discriminator model name [%s] is not recognized" % netD
+        )
+    return init_net(net, init_type, init_gain, gpu_ids)
+
+
+##############################################################################
+# Classes
+##############################################################################
+
+
+def cal_gradient_penalty(
+    netD, real_data, fake_data, device, type="mixed", constant=1.0, lambda_gp=10.0
+):
+    """Calculate the gradient penalty loss, used in WGAN-GP paper https://arxiv.org/abs/1704.00028
+
+    Arguments:
+        netD (network)              -- discriminator network
+        real_data (tensor array)    -- real images
+        fake_data (tensor array)    -- generated images from the generator
+        device (str)                -- GPU / CPU: from torch.device('cuda:{}'.format(self.gpu_ids[0])) if self.gpu_ids else torch.device('cpu')
+        type (str)                  -- if we mix real and fake data or not [real | fake | mixed].
+        constant (float)            -- the constant used in formula ( ||gradient||_2 - constant)^2
+        lambda_gp (float)           -- weight for this loss
+
+    Returns the gradient penalty loss
+    """
+    if lambda_gp > 0.0:
+        if (
+            type == "real"
+        ):  # either use real images, fake images, or a linear interpolation of two.
+            interpolatesv = real_data
+        elif type == "fake":
+            interpolatesv = fake_data
+        elif type == "mixed":
+            alpha = torch.rand(real_data.shape[0], 1, device=device)
+            alpha = (
+                alpha.expand(
+                    real_data.shape[0], real_data.nelement() // real_data.shape[0]
+                )
+                .contiguous()
+                .view(*real_data.shape)
+            )
+            interpolatesv = alpha * real_data + ((1 - alpha) * fake_data)
+        else:
+            raise NotImplementedError("{} not implemented".format(type))
+        interpolatesv.requires_grad_(True)
+        disc_interpolates = netD(interpolatesv)
+        gradients = torch.autograd.grad(
+            outputs=disc_interpolates,
+            inputs=interpolatesv,
+            grad_outputs=torch.ones(disc_interpolates.size()).to(device),
+            create_graph=True,
+            retain_graph=True,
+            only_inputs=True,
+        )
+        gradients = gradients[0].view(real_data.size(0), -1)  # flat the data
+        gradient_penalty = (
+            ((gradients + 1e-16).norm(2, dim=1) - constant) ** 2
+        ).mean() * lambda_gp  # added eps
+        return gradient_penalty, gradients
+    else:
+        return 0.0, None
+
+
+class ResnetGenerator(nn.Module):
+    """Resnet-based generator that consists of Resnet blocks between a few downsampling/upsampling operations.
+
+    We adapt Torch code and idea from Justin Johnson's neural style transfer project(https://github.com/jcjohnson/fast-neural-style)
+    """
+
+    def __init__(
+        self,
+        input_nc,
+        output_nc,
+        ngf=64,
+        norm_layer=nn.BatchNorm2d,
+        use_dropout=False,
+        n_blocks=6,
+        padding_type="reflect",
+    ):
+        """Construct a Resnet-based generator
+
+        Parameters:
+            input_nc (int)      -- the number of channels in input images
+            output_nc (int)     -- the number of channels in output images
+            ngf (int)           -- the number of filters in the last conv layer
+            norm_layer          -- normalization layer
+            use_dropout (bool)  -- if use dropout layers
+            n_blocks (int)      -- the number of ResNet blocks
+            padding_type (str)  -- the name of padding layer in conv layers: reflect | replicate | zero
+        """
+        assert n_blocks >= 0
+        super(ResnetGenerator, self).__init__()
+        if type(norm_layer) == functools.partial:
+            use_bias = norm_layer.func == nn.InstanceNorm2d
+        else:
+            use_bias = norm_layer == nn.InstanceNorm2d
+        model = [
+            nn.ReflectionPad2d(3),
+            nn.Conv2d(input_nc, ngf, kernel_size=7, padding=0, bias=use_bias),
+            norm_layer(ngf),
+            nn.ReLU(True),
+        ]
+
+        n_downsampling = 2
+        for i in range(n_downsampling):  # add downsampling layers
+            mult = 2**i
+            model += [
+                nn.Conv2d(
+                    ngf * mult,
+                    ngf * mult * 2,
+                    kernel_size=3,
+                    stride=2,
+                    padding=1,
+                    bias=use_bias,
+                ),
+                norm_layer(ngf * mult * 2),
+                nn.ReLU(True),
+            ]
+        mult = 2**n_downsampling
+        for i in range(n_blocks):  # add ResNet blocks
+
+            model += [
+                ResnetBlock(
+                    ngf * mult,
+                    padding_type=padding_type,
+                    norm_layer=norm_layer,
+                    use_dropout=use_dropout,
+                    use_bias=use_bias,
+                )
+            ]
+        for i in range(n_downsampling):  # add upsampling layers
+            mult = 2 ** (n_downsampling - i)
+            model += [
+                nn.ConvTranspose2d(
+                    ngf * mult,
+                    int(ngf * mult / 2),
+                    kernel_size=3,
+                    stride=2,
+                    padding=1,
+                    output_padding=1,
+                    bias=use_bias,
+                ),
+                norm_layer(int(ngf * mult / 2)),
+                nn.ReLU(True),
+            ]
+        model += [nn.ReflectionPad2d(3)]
+        model += [nn.Conv2d(ngf, output_nc, kernel_size=7, padding=0)]
+        model += [nn.Tanh()]
+
+        self.model = nn.Sequential(*model)
+
+    def forward(self, input):
+        """Standard forward"""
+        return self.model(input)
+
+
+class ResnetBlock(nn.Module):
+    """Define a Resnet block"""
+
+    def __init__(self, dim, padding_type, norm_layer, use_dropout, use_bias):
+        """Initialize the Resnet block
+
+        A resnet block is a conv block with skip connections
+        We construct a conv block with build_conv_block function,
+        and implement skip connections in <forward> function.
+        Original Resnet paper: https://arxiv.org/pdf/1512.03385.pdf
+        """
+        super(ResnetBlock, self).__init__()
+        self.conv_block = self.build_conv_block(
+            dim, padding_type, norm_layer, use_dropout, use_bias
+        )
+
+    def build_conv_block(self, dim, padding_type, norm_layer, use_dropout, use_bias):
+        """Construct a convolutional block.
+
+        Parameters:
+            norm_layer:
+            dim (int)           -- the number of channels in the conv layer.
+            padding_type (str)  -- the name of padding layer: reflect | replicate | zero
+            norm_layer          -- normalization layer
+            use_dropout (bool)  -- if use dropout layers.
+            use_bias (bool)     -- if the conv layer uses bias or not
+
+        Returns a conv block (with a conv layer, a normalization layer, and a non-linearity layer (ReLU))
+        """
+        conv_block = []
+        p = 0
+
+        if padding_type == "reflect":
+            conv_block += [nn.ReflectionPad2d(1)]
+        elif padding_type == "replicate":
+            conv_block += [nn.ReplicationPad2d(1)]
+        elif padding_type == "zero":
+            p = 1
+        else:
+            raise NotImplementedError("padding [%s] is not implemented" % padding_type)
+        conv_block += [
+            nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias),
+            norm_layer(dim),
+            nn.ReLU(True),
+        ]
+        if use_dropout:
+            conv_block += [nn.Dropout(0.5)]
+        p = 0
+        if padding_type == "reflect":
+            conv_block += [nn.ReflectionPad2d(1)]
+        elif padding_type == "replicate":
+            conv_block += [nn.ReplicationPad2d(1)]
+        elif padding_type == "zero":
+            p = 1
+        else:
+            raise NotImplementedError("padding [%s] is not implemented" % padding_type)
+        conv_block += [
+            nn.Conv2d(dim, dim, kernel_size=3, padding=p, bias=use_bias),
+            norm_layer(dim),
+        ]
+
+        return nn.Sequential(*conv_block)
+
+    def forward(self, x):
+        """Forward function (with skip connections)"""
+        out = x + self.conv_block(x)  # add skip connections
+        return out
+
+
+class UnetGenerator(nn.Module):
+    """Create a Unet-based generator"""
+
+    def __init__(
+        self,
+        input_nc,
+        output_nc,
+        num_downs,
+        ngf=64,
+        norm_layer=nn.BatchNorm2d,
+        use_dropout=False,
+    ):
+        """Construct a Unet generator
+        Parameters:
+            input_nc (int)  -- the number of channels in input images
+            output_nc (int) -- the number of channels in output images
+            num_downs (int) -- the number of downsamplings in UNet. For example, # if |num_downs| == 7,
+                                image of size 128x128 will become of size 1x1 # at the bottleneck
+            ngf (int)       -- the number of filters in the last conv layer
+            norm_layer      -- normalization layer
+
+        We construct the U-Net from the innermost layer to the outermost layer.
+        It is a recursive process.
+        """
+        super(UnetGenerator, self).__init__()
+        # construct unet structure
+        unet_block = UnetSkipConnectionBlock(
+            ngf * 8,
+            ngf * 8,
+            input_nc=None,
+            submodule=None,
+            norm_layer=norm_layer,
+            innermost=True,
+        )  # add the innermost layer
+        for i in range(num_downs - 5):  # add intermediate layers with ngf * 8 filters
+            unet_block = UnetSkipConnectionBlock(
+                ngf * 8,
+                ngf * 8,
+                input_nc=None,
+                submodule=unet_block,
+                norm_layer=norm_layer,
+                use_dropout=use_dropout,
+            )
+        # gradually reduce the number of filters from ngf * 8 to ngf
+        unet_block = UnetSkipConnectionBlock(
+            ngf * 4, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer
+        )
+        unet_block = UnetSkipConnectionBlock(
+            ngf * 2, ngf * 4, input_nc=None, submodule=unet_block, norm_layer=norm_layer
+        )
+        unet_block = UnetSkipConnectionBlock(
+            ngf, ngf * 2, input_nc=None, submodule=unet_block, norm_layer=norm_layer
+        )
+        self.model = UnetSkipConnectionBlock(
+            output_nc,
+            ngf,
+            input_nc=input_nc,
+            submodule=unet_block,
+            outermost=True,
+            norm_layer=norm_layer,
+        )  # add the outermost layer
+
+    def forward(self, input):
+        """Standard forward"""
+        return self.model(input)
+
+
+class UnetSkipConnectionBlock(nn.Module):
+    """Defines the Unet submodule with skip connection.
+    X -------------------identity----------------------
+    |-- downsampling -- |submodule| -- upsampling --|
+    """
+
+    def __init__(
+        self,
+        outer_nc,
+        inner_nc,
+        input_nc=None,
+        submodule=None,
+        outermost=False,
+        innermost=False,
+        norm_layer=nn.BatchNorm2d,
+        use_dropout=False,
+    ):
+        """Construct a Unet submodule with skip connections.
+
+        Parameters:
+            outer_nc (int) -- the number of filters in the outer conv layer
+            inner_nc (int) -- the number of filters in the inner conv layer
+            input_nc (int) -- the number of channels in input images/features
+            submodule (UnetSkipConnectionBlock) -- previously defined submodules
+            outermost (bool)    -- if this module is the outermost module
+            innermost (bool)    -- if this module is the innermost module
+            norm_layer          -- normalization layer
+            use_dropout (bool)  -- if use dropout layers.
+        """
+        super(UnetSkipConnectionBlock, self).__init__()
+        self.outermost = outermost
+        if type(norm_layer) == functools.partial:
+            use_bias = norm_layer.func == nn.InstanceNorm2d
+        else:
+            use_bias = norm_layer == nn.InstanceNorm2d
+        if input_nc is None:
+            input_nc = outer_nc
+        downconv = nn.Conv2d(
+            input_nc, inner_nc, kernel_size=4, stride=2, padding=1, bias=use_bias
+        )
+        downrelu = nn.LeakyReLU(0.2, True)
+        downnorm = norm_layer(inner_nc)
+        uprelu = nn.ReLU(True)
+        upnorm = norm_layer(outer_nc)
+
+        if outermost:
+            upconv = nn.ConvTranspose2d(
+                inner_nc * 2, outer_nc, kernel_size=4, stride=2, padding=1
+            )
+            down = [downconv]
+            up = [uprelu, upconv, nn.Tanh()]
+            model = down + [submodule] + up
+        elif innermost:
+            upconv = nn.ConvTranspose2d(
+                inner_nc, outer_nc, kernel_size=4, stride=2, padding=1, bias=use_bias
+            )
+            down = [downrelu, downconv]
+            up = [uprelu, upconv, upnorm]
+            model = down + up
+        else:
+            upconv = nn.ConvTranspose2d(
+                inner_nc * 2,
+                outer_nc,
+                kernel_size=4,
+                stride=2,
+                padding=1,
+                bias=use_bias,
+            )
+            down = [downrelu, downconv, downnorm]
+            up = [uprelu, upconv, upnorm]
+
+            if use_dropout:
+                model = down + [submodule] + up + [nn.Dropout(0.5)]
+            else:
+                model = down + [submodule] + up
+        self.model = nn.Sequential(*model)
+
+    def forward(self, x):
+        if self.outermost:
+            return self.model(x)
+        else:  # add skip connections
+            return torch.cat([x, self.model(x)], 1)
+
+
+class NLayerDiscriminator(nn.Module):
+    """Defines a PatchGAN discriminator"""
+
+    def __init__(self, input_nc, ndf=64, n_layers=3, norm_layer=nn.BatchNorm2d):
+        """Construct a PatchGAN discriminator
+
+        Parameters:
+            input_nc (int)  -- the number of channels in input images
+            ndf (int)       -- the number of filters in the last conv layer
+            n_layers (int)  -- the number of conv layers in the discriminator
+            norm_layer      -- normalization layer
+        """
+        super(NLayerDiscriminator, self).__init__()
+        if (
+            type(norm_layer) == functools.partial
+        ):  # no need to use bias as BatchNorm2d has affine parameters
+            use_bias = norm_layer.func == nn.InstanceNorm2d
+        else:
+            use_bias = norm_layer == nn.InstanceNorm2d
+        kw = 4
+        padw = 1
+        sequence = [
+            nn.Conv2d(input_nc, ndf, kernel_size=kw, stride=2, padding=padw),
+            nn.LeakyReLU(0.2, True),
+        ]
+        nf_mult = 1
+        nf_mult_prev = 1
+        for n in range(1, n_layers):  # gradually increase the number of filters
+            nf_mult_prev = nf_mult
+            nf_mult = min(2**n, 8)
+            sequence += [
+                nn.Conv2d(
+                    ndf * nf_mult_prev,
+                    ndf * nf_mult,
+                    kernel_size=kw,
+                    stride=2,
+                    padding=padw,
+                    bias=use_bias,
+                ),
+                norm_layer(ndf * nf_mult),
+                nn.LeakyReLU(0.2, True),
+            ]
+        nf_mult_prev = nf_mult
+        nf_mult = min(2**n_layers, 8)
+        sequence += [
+            nn.Conv2d(
+                ndf * nf_mult_prev,
+                ndf * nf_mult,
+                kernel_size=kw,
+                stride=1,
+                padding=padw,
+                bias=use_bias,
+            ),
+            norm_layer(ndf * nf_mult),
+            nn.LeakyReLU(0.2, True),
+        ]
+
+        sequence += [
+            nn.Conv2d(ndf * nf_mult, 1, kernel_size=kw, stride=1, padding=padw)
+        ]  # output 1 channel prediction map
+        self.model = nn.Sequential(*sequence)
+
+    def forward(self, input):
+        return self.model(input)
+
+
+class PixelDiscriminator(nn.Module):
+    """Defines a 1x1 PatchGAN discriminator (pixelGAN)"""
+
+    def __init__(self, input_nc, ndf=64, norm_layer=nn.BatchNorm2d):
+        """Construct a 1x1 PatchGAN discriminator
+
+        Parameters:
+            input_nc (int)  -- the number of channels in input images
+            ndf (int)       -- the number of filters in the last conv layer
+            norm_layer      -- normalization layer
+        """
+        super(PixelDiscriminator, self).__init__()
+        if (
+            type(norm_layer) == functools.partial
+        ):  # no need to use bias as BatchNorm2d has affine parameters
+            use_bias = norm_layer.func == nn.InstanceNorm2d
+        else:
+            use_bias = norm_layer == nn.InstanceNorm2d
+        self.net = [
+            nn.Conv2d(input_nc, ndf, kernel_size=1, stride=1, padding=0),
+            nn.LeakyReLU(0.2, True),
+            nn.Conv2d(ndf, ndf * 2, kernel_size=1, stride=1, padding=0, bias=use_bias),
+            norm_layer(ndf * 2),
+            nn.LeakyReLU(0.2, True),
+            nn.Conv2d(ndf * 2, 1, kernel_size=1, stride=1, padding=0, bias=use_bias),
+        ]
+
+        self.net = nn.Sequential(*self.net)
+
+    def forward(self, input):
+        return self.net(input)

models/test_model.py

@@ -0,0 +1,86 @@
+from . import networks
+from .base_model import BaseModel
+
+
+class TestModel(BaseModel):
+    """This TestModel can be used to generate CycleGAN results for only one direction.
+    This model will automatically set '--dataset_mode single', which only loads the images from one collection.
+
+    See the test instruction for more details.
+    """
+
+    @staticmethod
+    def modify_commandline_options(parser, is_train=True):
+        """Add new dataset-specific options, and rewrite default values for existing options.
+
+        Parameters:
+            parser          -- original option parser
+            parser:
+            is_train (bool) -- whether training phase or test phase. You can use this flag to add training-specific or test-specific options.
+
+        Returns:
+            the modified parser.
+
+        The model can only be used during test time. It requires '--dataset_mode single'.
+        You need to specify the network using the option '--model_suffix'.
+        """
+        assert not is_train, "TestModel cannot be used during training time"
+        parser.set_defaults(dataset_mode="single")
+        parser.add_argument(
+            "--model_suffix",
+            type=str,
+            default="",
+            help="In checkpoints_dir, [epoch]_net_G[model_suffix].pth will be loaded as the generator.",
+        )
+
+        return parser
+
+    def __init__(self, opt):
+        """Initialize the pix2pix class.
+
+        Parameters:
+            opt (Option class)-- stores all the experiment flags; needs to be a subclass of BaseOptions
+        """
+        assert not opt.isTrain
+        BaseModel.__init__(self, opt)
+        # specify the training losses you want to print out. The training/test scripts  will call <BaseModel.get_current_losses>
+        self.loss_names = []
+        # specify the images you want to save/display. The training/test scripts  will call <BaseModel.get_current_visuals>
+        self.visual_names = ["real", "fake"]
+        # specify the models you want to save to the disk. The training/test scripts will call <BaseModel.save_networks> and <BaseModel.load_networks>
+        self.net_names = ["G" + opt.model_suffix]  # only generator is needed.
+        self.net_G = networks.define_G(
+            opt.input_nc,
+            opt.output_nc,
+            opt.ngf,
+            opt.netG,
+            opt.norm,
+            not opt.no_dropout,
+            opt.init_type,
+            opt.init_gain,
+            self.gpu_ids,
+        )
+
+        # assigns the model to self.netG_[suffix] so that it can be loaded
+        # please see <BaseModel.load_networks>
+        setattr(self, "net_G" + opt.model_suffix, self.net_G)  # store netG in self.
+
+    def set_input(self, input):
+        """Unpack input data from the dataLoader and perform necessary pre-processing steps.
+
+        Parameters:
+            input: a dictionary that contains the data itself and its metadata information.
+
+        We need to use 'single_dataset' a dataset mode.
+        It only loads images from one domain.
+        """
+        self.real = input["A"].to(self.device)
+        self.image_paths = input["A_paths"]
+
+    def forward(self):
+        """Run forward pass."""
+        self.fake = self.net_G(self.real)  # G(real)
+
+    def optimize_parameters(self):
+        """No optimization for test model."""
+        pass

options/__init__.py

@@ -0,0 +1,4 @@
+from .base_options import BaseOptions
+from .detect_options import DetectOptions
+from .test_options import TestOptions
+from .train_options import TrainOptions

options/base_options.py

@@ -0,0 +1,149 @@
+import argparse
+from pathlib import Path
+
+import torch
+
+import data
+import models
+from util import util
+
+
+class BaseOptions:
+    """模型超参数设置
+    """
+    def __init__(self):
+        """重置该类：表示该类尚未被初始化
+        训练模型、测试模型、推理模型的参数全继承该类
+        """
+        self.initialized = False
+
+    def initialize(self, parser):
+        """共有参数"""
+        # basic parameters
+        parser.add_argument("--dataroot", type=str, default="./datasets/horse2zebra/", help="path to images (should have subfolders trainA, trainB, valA, valB, etc)")
+        parser.add_argument("--name", type=str, default="horse2zebra", help="name of the experiment.")
+        parser.add_argument("--gpu_ids", type=str, default="0", help="gpu ids: e.g. 0  0,1,2 -1 for CPU")
+        parser.add_argument("--checkpoints_dir", type=str, default="./checkpoints", help="models_saved")
+        # model parameters
+        parser.add_argument("--model", type=str, default="cycle_gan")
+        parser.add_argument("--input_nc", type=int, default=3, help="# input image channels: 3 for RGB and 1 for grayscale")
+        parser.add_argument("--output_nc", type=int, default=3, help="# output image channels: 3 for RGB and 1 for grayscale")
+        parser.add_argument("--ngf", type=int, default=64, help="# of gen filters in the last conv layer")
+        parser.add_argument("--ndf", type=int, default=64, help="# of discrim filters in the first conv layer")
+        parser.add_argument(
+            "--netD",
+            type=str,
+            default="basic",
+            help="[basic | n_layers | pixel]. basic: a 70x70 PatchGAN. n_layers: allows you to specify the layers in the discriminator",
+        )
+        parser.add_argument("--netG", type=str, default="resnet_9blocks", help="[resnet_9blocks | resnet_6blocks | unet_256 | unet_128]")
+        parser.add_argument("--n_layers_D", type=int, default=3, help="only used if netD==n_layers")
+        parser.add_argument("--norm", type=str, default="instance", help="instance normalization or batch normalization [instance | batch | none]")
+        parser.add_argument("--init_type", type=str, default="normal", help="network initialization [normal | xavier | kaiming | orthogonal]")
+        parser.add_argument("--init_gain", type=float, default=0.02, help="scaling factor for normal, xavier and orthogonal.")
+        parser.add_argument("--no_dropout", type=bool, default=True, help="no dropout for the generator")
+        # dataset parameters
+        parser.add_argument("--dataset_mode", type=str, default="unaligned")
+        parser.add_argument("--direction", type=str, default="AtoB", help="AtoB or BtoA")
+        parser.add_argument("--serial_batches", action="store_true", help="if true, takes images in order to make batches, otherwise takes them randomly")
+        parser.add_argument("--num_threads", default=8, type=int, help="# threads for loading data")
+        parser.add_argument("--batch_size", type=int, default=1)
+        parser.add_argument("--load_size", type=int, default=286, help="scale images to this size")
+        parser.add_argument("--crop_size", type=int, default=256, help="then crop to this size")
+        parser.add_argument(
+            "--max_dataset_size",
+            type=int,
+            default=float("inf"),
+            help="Maximum number of samples allowed per dataset. If the dataset directory contains more than max_dataset_size, only a subset is loaded.",
+        )
+        parser.add_argument("--preprocess", type=str, default="resize_and_crop", help="[resize_and_crop | crop | scale_width | scale_width_and_crop | none] img preprocess")
+        parser.add_argument("--no_flip", action="store_true", help="if specified, do not flip the images for data augmentation")
+        parser.add_argument("--display_winsize", type=int, default=256, help="display window size for both visdom and HTML")
+        # additional parameters
+        parser.add_argument("--epoch", type=str, default="latest", help="which epoch to load? set to latest to use latest cached model")
+        parser.add_argument(
+            "--load_iter", type=int, default="0", help="which iteration to load? if load_iter > 0, the code will load models by iter_[load_iter]; otherwise, the code will load models by [epoch]"
+        )
+        parser.add_argument("--verbose", action="store_true", help="if specified, print more debugging information")
+        parser.add_argument("--suffix", default="", type=str, help="customized suffix: opt.name = opt.name + suffix: e.g., {model}_{netG}_size{load_size}")
+        # wandb parameters
+        parser.add_argument("--use_wandb", action="store_true", help="wandb logging")
+        parser.add_argument("--wandb_project_name", type=str, default="CycleGAN", help="wandb project name")
+        self.initialized = True
+        return parser
+
+    def gather_options(self):
+        """Initialize our parser with basic options(only once).
+        Add additional model-specific and dataset-specific options.
+        These options are defined in the <modify_commandline_options> function
+        in model and dataset classes.
+        """
+        if not self.initialized:  # check if it has been initialized
+            parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+            parser = self.initialize(parser)
+
+        # get the basic options
+        opt, _ = parser.parse_known_args()
+
+        # modify model-related parser options
+        model_name = opt.model
+        model_option_setter = models.get_option_setter(model_name)
+        parser = model_option_setter(parser, self.isTrain)
+        opt, _ = parser.parse_known_args()  # parse again with new defaults
+
+        # modify dataset-related parser options
+        dataset_name = opt.dataset_mode
+        dataset_option_setter = data.get_option_setter(dataset_name)
+        parser = dataset_option_setter(parser, self.isTrain)
+
+        # save and return the parser
+        self.parser = parser
+        # return parser.parse_args()
+        # 用上面的会报以下错误，命令行参数不能合并，gradio运行时会调用命令行参数，而我们的没有
+        # uvicorn: error: unrecognized arguments: app:demo.app --reload --port 7860 --log-level warning --reload-dir E:\miniconda3\envs\yanguan\lib\site-packages\gradio --reload-dir D:\projects\CycleGAN
+        return parser.parse_known_args()[0]
+
+    def print_options(self, opt):
+        """
+        1. 同时打印当前选项和默认值（如果不同）。
+        2. 将选项保存到一个文本文件 /[checkpoints_dir]/opt.txt 中
+        """
+        message = ""
+        message += "----------------- Options ---------------\n"
+        for k, v in sorted(vars(opt).items()):
+            comment = ""
+            default = self.parser.get_default(k)
+            if v != default:
+                comment = "\t[default: %s]" % str(default)
+            message += "{:>25}: {:<30}{}\n".format(str(k), str(v), comment)
+        message += "----------------- End -------------------"
+        print(message)
+
+        # save it to the disk
+        expr_dir = Path(opt.checkpoints_dir, opt.name)
+        util.mkdirs(expr_dir)
+        file_name = Path(expr_dir, "{}_opt.txt".format(opt.phase))
+        with open(file_name, "wt") as opt_file:
+            opt_file.write(message)
+            opt_file.write("\n")
+
+    def parse(self):
+        """解析选项，创建检查点目录后缀，并设置GPU设备。"""
+        opt = self.gather_options()
+        opt.isTrain = self.isTrain  # train or test
+
+        # process opt.suffix
+        if opt.suffix:
+            suffix = ("_" + opt.suffix.format(**vars(opt))) if opt.suffix != "" else ""
+            opt.name = opt.name + suffix
+
+        self.print_options(opt)
+
+        # set gpu ids
+        _ids = [int(str_id) for str_id in opt.gpu_ids.split(",")]
+        opt.gpu_ids = [_id for _id in _ids if _id >= 0]
+        if len(opt.gpu_ids) > 0:
+            torch.cuda.set_device(opt.gpu_ids[0])
+
+        self.opt = opt
+        return self.opt

options/detect_options.py

@@ -0,0 +1,25 @@
+from .test_options import TestOptions
+
+
+class DetectOptions(TestOptions):
+    """
+    继承TestOptions
+
+    新增推理图片路径、推理风格参数
+    """
+
+    def initialize(self, parser):
+        parser = TestOptions.initialize(self, parser)
+        parser.set_defaults(name="detect")
+        parser.set_defaults(dataroot="imgs/", help="folder: /imgs/ or file: xx.jpg")
+        parser.set_defaults(checkpoints_dir="./weights")
+        parser.set_defaults(gpu_ids="-1")
+        # 新增
+        parser.add_argument("--no_save_img", action="store_true", help="no save fake_img")
+        parser.add_argument("--style", type=str, default="horse2zebra")
+        #
+        # parser.add_argument("--reload", action="store_true")
+        # parser.add_argument("--port")
+        # parser.add_argument("--reload-dir")
+        # parser.add_argument("--log-level")
+        return parser

options/test_options.py

@@ -0,0 +1,28 @@
+from .base_options import BaseOptions
+
+
+class TestOptions(BaseOptions):
+    """
+    继承BaseOptions
+    """
+
+    def initialize(self, parser):
+        parser = BaseOptions.initialize(self, parser)  # define shared options
+        # 重写参数
+        self.isTrain = False
+        # To avoid cropping, the load_size should be the same as crop_size
+        parser.set_defaults(load_size=parser.get_default("crop_size"))
+        parser.set_defaults(model="test")
+        parser.set_defaults(no_dropout=True)
+        # 自定义部分
+        parser.set_defaults(name="horse2zebra")
+        parser.set_defaults(dataroot=r"datasets\horse2zebra\testA")
+
+        # 新增参数
+        parser.add_argument("--results_dir", type=str, default="./results/", help="saves results here.")
+        parser.add_argument("--aspect_ratio", type=float, default=1.0, help="aspect ratio of result images")
+        parser.add_argument("--phase", type=str, default="test", help="train, val, test, etc")
+        # Dropout and batch norm have different behaviors during training and test.
+        parser.add_argument("--eval", action="store_true", help="use eval mode during test time.")
+        parser.add_argument("--num_test", type=int, default=50, help="how many test images to run")
+        return parser

options/train_options.py

@@ -0,0 +1,42 @@
+from pathlib import Path
+
+from .base_options import BaseOptions
+
+
+class TrainOptions(BaseOptions):
+    """继承BaseOptions，补充训练参数
+    """
+
+    def initialize(self, parser):
+        self.isTrain = True
+        parser = BaseOptions.initialize(self, parser)
+        # 训练过程可视化参数
+        parser.add_argument("--display_freq", type=int, default=400, help="frequency of showing training results on screen")
+        parser.add_argument("--display_ncols", type=int, default=4, help="if positive, display all images in a single visdom web panel with certain number of images per row.")
+        parser.add_argument("--display_id", type=int, default=1, help="window id of the web display")
+        parser.add_argument("--display_server", type=str, default="http://localhost", help="visdom server of the web display")
+        parser.add_argument("--display_env", type=str, default="main", help="visdom display environment name")
+        parser.add_argument("--display_port", type=int, default=8097, help="visdom port of the web display")
+        parser.add_argument("--update_html_freq", type=int, default=1000, help="frequency of saving training results to html")
+        parser.add_argument("--print_freq", type=int, default=100, help="frequency of showing training results on console")
+        parser.add_argument("--no_html", action="store_true", help="do not save intermediate training results to [opt.checkpoints_dir]/[opt.name]/web/")
+        # 网络保存和加载参数
+        parser.add_argument("--save_latest_freq", type=int, default=5000, help="frequency of saving the latest results")
+        parser.add_argument("--save_epoch_freq", type=int, default=1, help="frequency of saving checkpoints at the end of epochs")
+        parser.add_argument("--save_by_iter", action="store_true", help="whether saves model by iteration")
+        parser.add_argument("--continue_train", action="store_true", help="continue training: load the latest model")
+        parser.add_argument("--epoch_count", type=int, default=1, help="the starting epoch count, we save the model by <epoch_count>, <epoch_count>+<save_latest_freq>, ...")
+        parser.add_argument("--phase", type=str, default="train", help="train, val, test, etc")
+        # 训练参数
+        parser.add_argument("--n_epochs", type=int, default=200, help="number of epochs with the initial learning rate")
+        parser.add_argument("--n_epochs_decay", type=int, default=100, help="number of epochs to linearly decay learning rate to zero")
+        parser.add_argument("--beta1", type=float, default=0.5, help="momentum term of adam")
+        parser.add_argument("--lr", type=float, default=0.0002, help="initial learning rate for adam")
+        parser.add_argument("--gan_mode", type=str, default="lsgan", help="the type of GAN objective. [vanilla| lsgan | wgangp]. vanilla GAN loss used in the original GAN")
+        parser.add_argument("--pool_size", type=int, default=50, help="the size of image buffer that stores previously generated images")
+        parser.add_argument("--lr_policy", type=str, default="linear", help="learning rate policy. [linear | step | plateau | cosine]")
+        parser.add_argument("--lr_decay_iters", type=int, default=50, help="multiply by a gamma every lr_decay_iters iterations")
+        #
+        parser.set_defaults(dataroot="./datasets/horse2zebra/")
+        parser.set_defaults(name="horse2zebra")
+        return parser

requirements.txt

@@ -0,0 +1,12 @@
+beautifulsoup4==4.12.0
+dominate==2.7.0
+gradio==3.24.0
+matplotlib==3.7.1
+numpy==1.23.3
+Pillow==9.4.0
+requests==2.25.1
+torch==2.0.0
+torchvision==0.15.1
+tqdm==4.65.0
+visdom==0.2.4
+wandb==0.13.11

scripts/download_cyclegan_model.sh

@@ -0,0 +1,11 @@
+FILE=$1
+
+echo "Note: available models are apple2orange, orange2apple, summer2winter_yosemite, winter2summer_yosemite, horse2zebra, zebra2horse, monet2photo, style_monet, style_cezanne, style_ukiyoe, style_vangogh, sat2map, map2sat, cityscapes_photo2label, cityscapes_label2photo, facades_photo2label, facades_label2photo, iphone2dslr_flower"
+
+echo "Specified [$FILE]"
+
+mkdir -p ./checkpoints/${FILE}_pretrained
+MODEL_FILE=./checkpoints/${FILE}_pretrained/latest_net_G.pth
+URL=http://efrosgans.eecs.berkeley.edu/cyclegan/pretrained_models/$FILE.pth
+
+wget -N $URL -O $MODEL_FILE

scripts/test_before_push.py

@@ -0,0 +1,51 @@
+# Simple script to make sure basic usage
+# such as training, testing, saving and loading
+# runs without errors.
+import os
+
+
+def run(command):
+    print(command)
+    exit_status = os.system(command)
+    if exit_status > 0:
+        exit(1)
+
+
+if __name__ == '__main__':
+    # download mini datasets
+    if not os.path.exists('./datasets/mini'):
+        run('bash ./datasets/download_cyclegan_dataset.sh mini')
+
+    if not os.path.exists('./datasets/mini_pix2pix'):
+        run('bash ./datasets/download_cyclegan_dataset.sh mini_pix2pix')
+
+    # pretrained cyclegan model
+    if not os.path.exists('./checkpoints/horse2zebra_pretrained/latest_net_G.pth'):
+        run('bash ./scripts/download_cyclegan_model.sh horse2zebra')
+    run('python test.py --model test --dataroot ./datasets/mini --name horse2zebra_pretrained --no_dropout --num_test 1 --no_dropout')
+
+    # pretrained pix2pix model
+    if not os.path.exists('./checkpoints/facades_label2photo_pretrained/latest_net_G.pth'):
+        run('bash ./scripts/download_pix2pix_model.sh facades_label2photo')
+    if not os.path.exists('./datasets/facades'):
+        run('bash ./datasets/download_pix2pix_dataset.sh facades')
+    run('python test.py --dataroot ./datasets/facades/ --direction BtoA --model pix2pix --name facades_label2photo_pretrained --num_test 1')
+
+    # cyclegan train/test
+    run('python train.py --model cycle_gan --name temp_cyclegan --dataroot ./datasets/mini --n_epochs 1 --n_epochs_decay 0 --save_latest_freq 10  --print_freq 1 --display_id -1')
+    run('python test.py --model test --name temp_cyclegan --dataroot ./datasets/mini --num_test 1 --model_suffix "_A" --no_dropout')
+
+    # pix2pix train/test
+    run('python train.py --model pix2pix --name temp_pix2pix --dataroot ./datasets/mini_pix2pix --n_epochs 1 --n_epochs_decay 5 --save_latest_freq 10 --display_id -1')
+    run('python test.py --model pix2pix --name temp_pix2pix --dataroot ./datasets/mini_pix2pix --num_test 1')
+
+    # template train/test
+    run('python train.py --model template --name temp2 --dataroot ./datasets/mini_pix2pix --n_epochs 1 --n_epochs_decay 0 --save_latest_freq 10 --display_id -1')
+    run('python test.py --model template --name temp2 --dataroot ./datasets/mini_pix2pix --num_test 1')
+
+    # colorization train/test (optional)
+    if not os.path.exists('./datasets/mini_colorization'):
+        run('bash ./datasets/download_cyclegan_dataset.sh mini_colorization')
+
+    run('python train.py --model colorization --name temp_color --dataroot ./datasets/mini_colorization --n_epochs 1 --n_epochs_decay 0 --save_latest_freq 5 --display_id -1')
+    run('python test.py --model colorization --name temp_color --dataroot ./datasets/mini_colorization --num_test 1')

scripts/test_cyclegan.sh

@@ -0,0 +1,2 @@
+set -ex
+python test.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --phase test --no_dropout

scripts/test_single.sh

@@ -0,0 +1,2 @@
+set -ex
+python test.py --dataroot ./datasets/facades/testB/ --name facades_pix2pix --model test --netG unet_256 --direction BtoA --dataset_mode single --norm batch

scripts/train_cyclegan.sh

@@ -0,0 +1,2 @@
+set -ex
+python train.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --pool_size 50 --no_dropout

util/__init__.py

@@ -0,0 +1,2 @@
+from .tools import *
+from .util import *

util/get_data.py

@@ -0,0 +1,97 @@
+# from __future__ import print_function
+
+import tarfile
+from pathlib import Path
+from warnings import warn
+from zipfile import ZipFile
+
+import requests
+from bs4 import BeautifulSoup
+
+
+class GetData(object):
+    """A Python script for downloading CycleGAN or pix2pix datasets.
+
+    Parameters:
+        technique (str) -- One of: 'cyclegan' or 'pix2pix'.
+        verbose (bool)  -- If True, print additional information.
+
+    Examples:
+        >>> from util.get_data import GetData
+        >>> gd = GetData(technique='cyclegan', save_path='./datasets')# options will be displayed.
+
+    Alternatively, You can use bash scripts: 'scripts/download_pix2pix_model.sh'
+    and 'scripts/download_cyclegan_model.sh'.
+    """
+
+    def __init__(self, technique="CycleGAN", save_path="./datasets", verbose=True):
+        url_dict = {
+            "cyclegan": "http://efrosgans.eecs.berkeley.edu/cyclegan/datasets/",
+        }
+        self.url = url_dict.get(technique.lower())
+        self._verbose = verbose
+        self.get(save_path=save_path)
+
+    def _print(self, text: str):
+        if self._verbose:
+            print(text)
+
+    @staticmethod
+    def _get_options(r):
+        soup = BeautifulSoup(r.text, "lxml")
+        options = [
+            h.text
+            for h in soup.find_all("a", href=True)
+            if h.text.endswith((".zip", "tar.gz"))
+        ]
+        return options
+
+    def _present_options(self):
+        print(self.url)
+        r = requests.get(self.url)
+        options = self._get_options(r)
+        print("Options:\n")
+        for i, o in enumerate(options):
+            print("{0}: {1}".format(i, o))
+        choice = input(
+            "\nPlease enter the number of the " "dataset above you wish to download:"
+        )
+        return options[int(choice)]
+
+    def _download_data(self, dataset_url: str, dataset_path: Path):
+        dataset_path.mkdir(exist_ok=True)
+
+        save_path = Path(dataset_path).joinpath(Path(dataset_url).name)
+
+        print(dataset_url)
+        import urllib.request
+
+        urllib.request.urlretrieve(dataset_url, save_path)
+        print("--> 下载完成 ")
+
+        if save_path.endswith(".tar.gz"):
+            obj = tarfile.open(save_path)
+        elif save_path.endswith(".zip"):
+            obj = ZipFile(save_path, "r")
+        else:
+            raise ValueError("Unknown File Type: {0}.".format(save_path))
+        self._print("Unpacking Data...")
+        obj.extractall(save_path)
+        obj.close()
+
+    def get(self, save_path: str, dataset=None):
+        save_path_ = Path(save_path)
+        if dataset is None:
+            selected_dataset = self._present_options()
+        else:
+            selected_dataset = dataset
+        save_path_full = save_path_.joinpath(selected_dataset.split(".")[0])
+        print(save_path_full)
+
+        if save_path_full.is_dir():
+            warn("\n'{0}' already exists.".format(save_path_full))
+        else:
+            self._print("Downloading Data...")
+            url = "{0}/{1}".format(self.url, selected_dataset)
+            self._download_data(url, save_path=save_path)
+        return Path(save_path_full)

util/html.py

@@ -0,0 +1,95 @@
+import os
+from pathlib import Path
+
+import dominate
+from dominate.tags import a, br, h3, img, meta, p, table, td, tr
+
+
+class HTML:
+    """This HTML class allows us to save images and write texts into a single HTML file.
+
+    It consists of functions such as <add_header> (add a text header to the HTML file),
+    <add_images> (add a row of images to the HTML file), and <save> (save the HTML to the disk).
+    It is based on the Python library 'dominate', a Python library for creating and manipulating HTML documents using a DOM API.
+    """
+
+    def __init__(self, web_dir, title, refresh=0):
+        """Initialize the HTML classes
+
+        Parameters:
+            web_dir (str) -- a directory that stores the webpage. HTML file will be created at <web_dir>/index.html; images will be saved at <web_dir/images/
+            title (str)   -- the webpage name
+            refresh (int) -- how often the website refreshes itself; if 0; no refreshing
+        """
+        self.title = title
+        self.web_dir = web_dir
+        self.img_dir = Path(self.web_dir, "images")
+        if not os.path.exists(self.web_dir):
+            os.makedirs(self.web_dir)
+        if not os.path.exists(self.img_dir):
+            os.makedirs(self.img_dir)
+        self.doc = dominate.document(title=title)
+        if refresh > 0:
+            with self.doc.head:
+                meta(http_equiv="refresh", content=str(refresh))
+
+    def get_image_dir(self):
+        """Return the directory that stores images"""
+        return self.img_dir
+
+    def add_header(self, text):
+        """Insert a header to the HTML file
+
+        Parameters:
+            text (str) -- the header text
+        """
+        with self.doc:
+            h3(text)
+
+    def add_images(self, ims, txts, links, width=400):
+        """add images to the HTML file
+
+        Parameters:
+            width:
+            ims (str list)   -- a list of image paths
+            txts (str list) -- a list of image names shown on the website
+            links (str list) -- a list of hyperref links; when you click an image, it will redirect you to a new page
+        """
+        self.t = table(border=1, style="table-layout: fixed;")  # Insert a table
+        self.doc.add(self.t)
+        with self.t:
+            with tr():
+                for im, txt, link in zip(ims, txts, links):
+                    with td(
+                        style="word-wrap: break-word;", halign="center", valign="top"
+                    ):
+                        with p():
+                            with a(href=Path("images", link)):
+                                img(style="width:%dpx" % width, src=Path("images", im))
+                            br()
+                            p(txt)
+
+    def save(self):
+        """将内容保存到HMTL文件"""
+        html_file = f"{self.web_dir}/index.html"
+        with open(html_file, "wt") as f:
+            html = (
+                self.doc.render()
+                .replace("</body>", "</center></body>")
+                .replace("<body>", "<body><center>")
+            )
+            f.write(html)  # 添加body居中
+        print("--> " + html_file)
+
+
+if __name__ == "__main__":  # show an example usage here.
+    html = HTML("web/", "test_html")
+
+    html.add_header("hello world")
+    ims, txts, links = [], [], []
+    for n in range(4):
+        ims.append("image_%d.png" % n)
+        txts.append("text_%d" % n)
+        links.append("image_%d.png" % n)
+    html.add_images(ims, txts, links)
+    html.save()

util/image_pool.py

@@ -0,0 +1,60 @@
+import random
+
+import torch
+
+
+class ImagePool:
+    """This class implements an image buffer that stores previously generated images.
+
+    This buffer enables us to update discriminators using a history of generated images
+    rather than the ones produced by the latest generators.
+    """
+
+    def __init__(self, pool_size):
+        """Initialize the ImagePool class
+
+        Parameters:
+            pool_size (int) -- the size of the image buffer, if pool_size=0, no buffer will be created
+        """
+        self.pool_size = pool_size
+        if self.pool_size > 0:  # create an empty pool
+            self.num_imgs = 0
+            self.images = []
+
+    def query(self, images):
+        """Return an image from the pool.
+
+        Parameters:
+            images: the latest generated images from the generator
+
+        Returns images from the buffer.
+
+        By 50/100, the buffer will return input images.
+        By 50/100, the buffer will return images previously stored in the buffer,
+        and insert the current images to the buffer.
+        """
+        if self.pool_size == 0:  # if the buffer size is 0, do nothing
+            return images
+        return_images = []
+        for image in images:
+            image = torch.unsqueeze(image.data, 0)
+            if self.num_imgs < self.pool_size:
+                # if the buffer is not full; keep inserting current images to the buffer
+                self.num_imgs = self.num_imgs + 1
+                self.images.append(image)
+                return_images.append(image)
+            else:
+                p = random.uniform(0, 1)
+                if p > 0.5:
+                    # by 50% chance, the buffer will return a previously stored image, and insert the current image
+                    # into the buffer
+                    random_id = random.randint(
+                        0, self.pool_size - 1
+                    )  # randint is inclusive
+                    tmp = self.images[random_id].clone()
+                    self.images[random_id] = image
+                    return_images.append(tmp)
+                else:  # by another 50% chance, the buffer will return the current image
+                    return_images.append(image)
+        return_images = torch.cat(return_images, 0)  # collect all the images and return
+        return return_images

util/streamlit/css.css

@@ -0,0 +1,3 @@
+label {
+    display: none !important;
+}

util/tools.py

@@ -0,0 +1,23 @@
+def get_all_weights(weights_path="./weights/detect"):
+    from pathlib import Path
+
+    # 获取文件夹下所有文件（不包括文件夹）
+    all_weights = [
+        str(f.stem)
+        for f in Path(weights_path).iterdir()
+        if Path(f).is_file() and Path(f).suffix == ".pth"
+    ]
+    # print_info(all_weights)
+    return all_weights
+
+
+def now_time():
+    from datetime import datetime
+
+    return datetime.now().strftime(r"%Y-%m-%dT%H%M%S")
+
+
+def print_info(text: str):
+    BEGIN_COLOR = "\033[32m"
+    END_COLOR = "\033[0m"
+    print(f"{BEGIN_COLOR}{text}{END_COLOR}")

util/util.py

@@ -0,0 +1,135 @@
+"""功能函数
+"""
+from __future__ import print_function
+
+import os
+
+import numpy as np
+import torch
+from PIL import Image
+from torch import tensor
+
+
+def tensor2im(input_image: tensor, imtype=np.uint8):
+    """ "Converts a Tensor array into a numpy image array.
+
+    Parameters:
+        input_image (tensor) --  the input image tensor array
+        imtype (type)        --  the desired type of the converted numpy array
+    """
+    if len(input_image.size()) == 3:
+        input_image = input_image.unsqueeze(0)
+    if not isinstance(input_image, np.ndarray):
+        if isinstance(input_image, torch.Tensor):  # get the data from a variable
+            image_tensor = input_image.data
+        else:
+            return input_image
+        # convert it into a numpy array
+        image_numpy = image_tensor[0].cpu().float().numpy()
+        if image_numpy.shape[0] == 1:  # grayscale to RGB
+            image_numpy = np.tile(image_numpy, (3, 1, 1))
+        image_numpy = (
+            (np.transpose(image_numpy, (1, 2, 0)) + 1) / 2.0 * 255.0
+        )  # post-processing: tranpose and scaling
+    else:  # if it is a numpy array, do nothing
+        image_numpy = input_image
+    return image_numpy.astype(imtype)
+
+
+def diagnose_network(net, name="network"):
+    """Calculate and print the mean of average absolute(gradients)
+
+    Parameters:
+        net (torch network) -- Torch network
+        name (str) -- the name of the network
+    """
+    mean = 0.0
+    count = 0
+    for param in net.parameters():
+        if param.grad is not None:
+            mean += torch.mean(torch.abs(param.grad.data))
+            count += 1
+    if count > 0:
+        mean = mean / count
+    print(name)
+    print(mean)
+
+
+def save_image(image_numpy, image_path, aspect_ratio=1.0):
+    """Save a numpy image to the disk
+
+    Parameters:
+        aspect_ratio:
+        image_numpy (numpy array) -- input numpy array
+        image_path (str)          -- the path of the image
+    """
+
+    image_pil = Image.fromarray(image_numpy)
+    h, w, _ = image_numpy.shape
+
+    if aspect_ratio > 1.0:
+        image_pil = image_pil.resize((h, int(w * aspect_ratio)), Image.BICUBIC)
+    if aspect_ratio < 1.0:
+        image_pil = image_pil.resize((int(h / aspect_ratio), w), Image.BICUBIC)
+    image_pil.save(image_path)
+
+
+def print_numpy(x, val=True, shp=False):
+    """Print the mean, min, max, median, std, and size of a numpy array
+
+    Parameters:
+        x: (np.array)
+        val (bool) -- if print the values of the numpy array
+        shp (bool) -- if print the shape of the numpy array
+    """
+    x = x.astype(np.float64)
+    if shp:
+        print("shape,", x.shape)
+    if val:
+        x = x.flatten()
+        print(
+            "mean = %3.3f, min = %3.3f, max = %3.3f, median = %3.3f, std=%3.3f"
+            % (np.mean(x), np.min(x), np.max(x), np.median(x), np.std(x))
+        )
+
+
+def mkdirs(paths):
+    """create empty directories if they don't exist
+
+    Parameters:
+        paths (str list) -- a list of directory paths
+    """
+    if isinstance(paths, list) and not isinstance(paths, str):
+        for path in paths:
+            mkdir(path)
+    else:
+        mkdir(paths)
+
+
+def mkdir(path):
+    """create a single empty directory if it didn't exist
+
+    Parameters:
+        path (str) -- a single directory path
+    """
+    if not os.path.exists(path):
+        os.makedirs(path)
+
+
+def show_image(image_numpy, aspect_ratio=1.0):
+    """Save a numpy image to the disk
+
+    Parameters:
+        aspect_ratio:
+        image_numpy (numpy array) -- input numpy array
+    """
+
+    image_pil = Image.fromarray(image_numpy)
+    h, w, _ = image_numpy.shape
+
+    if aspect_ratio > 1.0:
+        image_pil = image_pil.resize((h, int(w * aspect_ratio)), Image.BICUBIC)
+    if aspect_ratio < 1.0:
+        image_pil = image_pil.resize((int(h / aspect_ratio), w), Image.BICUBIC)
+    # image_pil.save(image_path)
+    image_pil.show()

util/visualizer.py

@@ -0,0 +1,297 @@
+import ntpath
+import os
+import sys
+import time
+from pathlib import Path
+from subprocess import PIPE, Popen
+
+import numpy as np
+
+from util import now_time
+from . import html, util
+
+
+try:
+    import wandb
+except ImportError:
+    print(
+        'Warning: wandb package cannot be found. The option "--use_wandb" will result in error.'
+    )
+if sys.version_info[0] == 2:
+    VisdomExceptionBase = Exception
+else:
+    VisdomExceptionBase = ConnectionError
+
+
+def save_images(
+    webpage, visuals, image_path, aspect_ratio=1.0, width=256, use_wandb=False
+):
+    """Save images to the disk.
+
+    Parameters:
+        use_wandb:
+        webpage (the HTML class) -- the HTML webpage class that stores these imaegs (see html.py for more details)
+        visuals (OrderedDict)    -- an ordered dictionary that stores (name, images (either tensor or numpy) ) pairs
+        image_path (str)         -- the string is used to create image paths
+        aspect_ratio (float)     -- the aspect ratio of saved images
+        width (int)              -- the images will be resized to width x width
+
+    This function will save images stored in 'visuals' to the HTML file specified by 'webpage'.
+    """
+    image_dir = webpage.get_image_dir()
+    short_path = ntpath.basename(image_path[0])
+    name = os.path.splitext(short_path)[0]
+
+    webpage.add_header(name)
+    ims, txts, links = [], [], []
+    ims_dict = {}
+    for label, im_data in visuals.items():
+        im = util.tensor2im(im_data)
+        image_name = "%s_%s.png" % (name, label)
+        save_path = Path(image_dir, image_name)
+        util.save_image(im, save_path, aspect_ratio=aspect_ratio)
+        ims.append(image_name)
+        txts.append(label)
+        links.append(image_name)
+        if use_wandb:
+            ims_dict[label] = wandb.Image(im)
+    webpage.add_images(ims, txts, links, width=width)
+    if use_wandb:
+        wandb.log(ims_dict)
+
+
+class Visualizer:
+    """This class includes several functions that can display/save images and print/save logging information.
+
+    It uses a Python library 'visdom' for display, and a Python library 'dominate' (wrapped in 'HTML') for creating HTML files with images.
+    """
+
+    def __init__(self, opt):
+        """Initialize the Visualizer class
+
+        Parameters:
+            opt -- stores all the experiment flags; needs to be a subclass of BaseOptions
+        Step 1: Cache the training/test options
+        Step 2: connect to a visdom server
+        Step 3: create an HTML object for saveing HTML filters
+        Step 4: create a logging file to store training losses
+        """
+        self.opt = opt  # cache the option
+        self.display_id = opt.display_id
+        self.use_html = opt.isTrain and not opt.no_html
+        self.win_size = opt.display_winsize
+        self.name = opt.name
+        self.port = opt.display_port
+        self.saved = False
+        self.use_wandb = opt.use_wandb
+        self.wandb_project_name = opt.wandb_project_name
+        self.current_epoch = 0
+        self.ncols = opt.display_ncols
+
+        if (
+            self.display_id > 0
+        ):  # connect to a visdom server given <display_port> and <display_server>
+            import visdom
+
+            self.vis = visdom.Visdom(
+                server=opt.display_server, port=opt.display_port, env=opt.display_env
+            )
+            if not self.vis.check_connection():
+                self.create_visdom_connections()
+        if self.use_wandb:
+            self.wandb_run = (
+                wandb.init(project=self.wandb_project_name, name=opt.name, config=opt)
+                if not wandb.run
+                else wandb.run
+            )
+            self.wandb_run._label(repo="CycleGAN")
+        # create an HTML object at <checkpoints_dir>/web/; images will be saved under <checkpoints_dir>/web/images/
+        if self.use_html:
+            self.web_dir = Path(opt.checkpoints_dir, opt.name, "web" + now_time())
+            self.img_dir = self.web_dir.joinpath("images")
+            print("Create web directory %s..." % self.web_dir)
+            util.mkdirs([self.web_dir, self.img_dir])
+        # create a logging file to store training losses
+        self.log_name = Path(opt.checkpoints_dir, opt.name, "loss_log.txt")
+        with open(self.log_name, "a") as log_file:
+            now = time.strftime("%c")
+            log_file.write(
+                "================ Training Loss (%s) ================\n" % now
+            )
+
+    def reset(self):
+        """Reset the self.saved status"""
+        self.saved = False
+
+    def create_visdom_connections(self):
+        """If the program could not connect to Visdom server, this function will start a new server at port < self.port >"""
+        cmd = sys.executable + " -m visdom.server -p %d &>/dev/null &" % self.port
+        print("\n\nCould not connect to Visdom server. \n Trying to start a server....")
+        print("Command: %s" % cmd)
+        Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE)
+
+    def display_current_results(self, visuals, epoch, save_result):
+        """Display current results on visdom; save current results to an HTML file.
+
+        Parameters:
+            visuals (OrderedDict) - - dictionary of images to display or save
+            epoch (int) - - the current epoch
+            save_result (bool) - - if save the current results to an HTML file
+        """
+        if self.display_id > 0:  # show images in the browser using visdom
+            ncols = self.ncols
+            if ncols > 0:  # show all the images in one visdom panel
+                ncols = min(ncols, len(visuals))
+                h, w = next(iter(visuals.values())).shape[:2]
+                table_css = """<style>
+
+                        table {border-collapse: separate; border-spacing: 4px; white-space: nowrap; text-align: center}
+
+                        table td {width: % dpx; height: % dpx; padding: 4px; outline: 4px solid black}
+
+                        </style>""" % (
+                    w,
+                    h,
+                )  # create a table css
+                # create a table of images.
+                title = self.name
+                label_html = ""
+                label_html_row = ""
+                images = []
+                image_numpy = None
+                idx = 0
+                for label, image in visuals.items():
+                    image_numpy = util.tensor2im(image)
+                    label_html_row += "<td>%s</td>" % label
+                    images.append(image_numpy.transpose([2, 0, 1]))
+                    idx += 1
+                    if idx % ncols == 0:
+                        label_html += "<tr>%s</tr>" % label_html_row
+                        label_html_row = ""
+                white_image = np.ones_like(image_numpy.transpose([2, 0, 1])) * 255
+                while idx % ncols != 0:
+                    images.append(white_image)
+                    label_html_row += "<td></td>"
+                    idx += 1
+                if label_html_row != "":
+                    label_html += "<tr>%s</tr>" % label_html_row
+                try:
+                    self.vis.images(
+                        images,
+                        nrow=ncols,
+                        win=self.display_id + 1,
+                        padding=2,
+                        opts=dict(title=title + " images"),
+                    )
+                    label_html = "<table>%s</table>" % label_html
+                    self.vis.text(
+                        table_css + label_html,
+                        win=self.display_id + 2,
+                        opts=dict(title=title + " labels"),
+                    )
+                except VisdomExceptionBase:
+                    self.create_visdom_connections()
+            else:  # show each image in a separate visdom panel;
+                idx = 1
+                try:
+                    for label, image in visuals.items():
+                        image_numpy = util.tensor2im(image)
+                        self.vis.image(
+                            image_numpy.transpose([2, 0, 1]),
+                            opts=dict(title=label),
+                            win=self.display_id + idx,
+                        )
+                        idx += 1
+                except VisdomExceptionBase:
+                    self.create_visdom_connections()
+        if self.use_wandb:
+            columns = [key for key, _ in visuals.items()]
+            columns.insert(0, "epoch")
+            result_table = wandb.Table(columns=columns)
+            table_row = [epoch]
+            ims_dict = {}
+            for label, image in visuals.items():
+                image_numpy = util.tensor2im(image)
+                wandb_image = wandb.Image(image_numpy)
+                table_row.append(wandb_image)
+                ims_dict[label] = wandb_image
+            self.wandb_run.log(ims_dict)
+            if epoch != self.current_epoch:
+                self.current_epoch = epoch
+                result_table.add_data(*table_row)
+                self.wandb_run.log({"Result": result_table})
+        if self.use_html and (
+            save_result or not self.saved
+        ):  # save images to an HTML file if they haven't been saved.
+            self.saved = True
+            # save images to the disk
+            for label, image in visuals.items():
+                image_numpy = util.tensor2im(image)
+                img_path = Path(self.img_dir, "epoch%.3d_%s.png" % (epoch, label))
+                util.save_image(image_numpy, img_path)
+            # update website
+            webpage = html.HTML(
+                self.web_dir, "Experiment name = %s" % self.name, refresh=1
+            )
+            for n in range(epoch, 0, -1):
+                webpage.add_header("epoch [%d]" % n)
+                ims, txts, links = [], [], []
+
+                for label, image_numpy in visuals.items():
+                    # image_numpy = util.tensor2im(image)
+                    img_path = "epoch%.3d_%s.png" % (n, label)
+                    ims.append(img_path)
+                    txts.append(label)
+                    links.append(img_path)
+                webpage.add_images(ims, txts, links, width=self.win_size)
+            webpage.save()
+
+    def plot_current_losses(self, epoch, counter_ratio, losses):
+        """display the current losses on visdom display: dictionary of error labels and values
+
+        Parameters:
+            epoch (int)           -- current epoch
+            counter_ratio (float) -- progress (percentage) in the current epoch, between 0 to 1
+            losses (OrderedDict)  -- training losses stored in the format of (name, float) pairs
+        """
+        if not hasattr(self, "plot_data"):
+            self.plot_data = {"X": [], "Y": [], "legend": list(losses.keys())}
+        self.plot_data["X"].append(epoch + counter_ratio)
+        self.plot_data["Y"].append([losses[k] for k in self.plot_data["legend"]])
+        try:
+            self.vis.line(
+                X=np.stack(
+                    [np.array(self.plot_data["X"])] * len(self.plot_data["legend"]), 1
+                ),
+                Y=np.array(self.plot_data["Y"]),
+                opts={
+                    "title": f"{self.name} loss over time",
+                    "legend": self.plot_data["legend"],
+                    "xlabel": "epoch",
+                    "ylabel": "loss",
+                },
+                win=self.display_id,
+            )
+        except VisdomExceptionBase:
+            self.create_visdom_connections()
+        if self.use_wandb:
+            self.wandb_run.log(losses)
+
+    # losses: same format as |losses| of plot_current_losses
+    def print_current_losses(self, epoch, iters, losses, t_comp, t_data):
+        """print current losses on console; also save the losses to the disk
+
+        Parameters:
+            epoch (int) -- current epoch
+            iters (int) -- current training iteration during this epoch (reset to 0 at the end of every epoch)
+            losses (OrderedDict) -- training losses stored in the format of (name, float) pairs
+            t_comp (float) -- computational time per data point (normalized by batch_size)
+            t_data (float) -- data loading time per data point (normalized by batch_size)
+        """
+        message = f"(epoch: {epoch:>2d}, iters: {iters:>4d}, time: {t_comp:.3f}, data: {t_data:.3f})"
+        for k, v in losses.items():
+            message += f" {k:s}: {v:.3f}"
+        print(message)
+
+        with open(self.log_name, "a") as log_file:
+            log_file.write(f"{message:s}\n")

weights/detect/apple2orange.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff989c98f3a077cd11c2623ed8742e2ef147e81d10b4e4c2ee6935796bda957a
+size 45575747

weights/detect/cityscapes_label2photo.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad637d74a1fb752aa4d2f427452954e153a5bedf5342492de7b53bffb04e5081
+size 45575747

weights/detect/cityscapes_photo2label.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7649da0ec6f393efe171f333fb17508e1e1d0176191496fba2a0d37c90db82d0
+size 45575747

weights/detect/facades_label2photo.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:184711ff5ccae4e8aab7e39d72eb7fb481b52952b04cfe4d47cdd75317e77d7d
+size 45575747

weights/detect/facades_photo2label.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1714709fd7fa496a5300c0c32a7078653ba5a9530a81b30339a86337610f420b
+size 45575747

weights/detect/horse2zebra.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bfc34b154035be1cd592cd273fdf328810bee4756044c9cf7deb8d4b6d2fadb
+size 45575747

weights/detect/iphone2dslr_flower_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c376197c87302f2b1af52c2e1f4dbd033d57babc7ae1eff1d66a81c6b2109bd
+size 45575747

weights/detect/latest_net_G_A.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:858eb597ea7335db5a230555418fdc5be198807600b67f18701ec0b5090c6006
+size 45531635

weights/detect/latest_net_G_A0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13950e21593133d93d2907bf4e0a0a3e9ca47664ebb2c74209113446f95c5997
+size 45531635

weights/detect/latest_net_G_A1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2607bdb799223f80c34c1de0fba301c294118d8d3a1a804e7add145bd4e546f5
+size 45531635

weights/detect/latest_net_G_B.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:768139e9d7d01ab20c82384311e2db934fc9bd16122fbbc84c36557396e1e6b8
+size 45531635

weights/detect/latest_net_G_B0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd5f04202bd0028475ada0cfe2a1321d91533df213b01c2e976557c196362a60
+size 45531635

weights/detect/latest_net_G_B1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfa9858158beeca602182c0956b42a0fb91bb9bc195cf923a1c9bb707035e87a
+size 45531635