Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary: 
The paper addresses the ill-posedness of the unsupervised domain translation (UDT) problem. It provides a more structured and rigorous problem definition than previous works (mainly CycleGAN-based), and proposes the theory of optimal transport (OT) as a better framework for solving UDT. The paper provides an interesting link between a dynamical formulation of OT and residual networks, which leads to a practical algorithm for solving OT/UDT. Experiments highlight two main points: 1) CycleGAN are biased towards learning nearly identity mappings, and 2) the OT formulation allows for modelling explicit biases in the learned solution through the design of the cost function.

Strengths & Weaknesses:
  +  The paper addresses an important problem, which as far as I know, is widely known but not properly or explicitly addressed in prior work. 
  -  While most definitions are rather intuitive, some are still vague so they cannot be constructive. For example, a UDT task is a subset of all possible mappings which are *desirable* for the given task, but it is not clear how we can exactly define *desirable* mappings.
  -  In addition, it is not clear why the set of all mappings X_{alpha,beta} needs to be constrained to invertible mappings. I see invertibility as only a constraint added by CycleGAN to limit the set of possible learned mappings.

  +  The paper makes an interesting observation that CycleGAN is biased towards simple, and nearly identity mappings (which I believe is the main consequence of small initialization values), which could explain its practical success.
  -  However, the paper needs to emphasize that this is particularly tied to the choice of resnet architectures that is commonly used.

  +  I like the proposed dynamical formulation for solving OT and the link to resnets, which provides an interesting practical algorithm.
  -  The main problem that remains unsolved is how to choose the cost function $c$. The paper acknowledges that, and proposes a specific cost functions for the specific tasks of the experimental section.

  -  While experiments support the main claims of the paper, they are still quite limited and do not really have a clear practical significance. The paper would have been much stronger if the proposed approach solves a more practical problem. 

In conclusion, while I think that the practical significance of the proposed approach is rather limited, I think that overall it makes an interesting contribution to the domain of UDT which can be useful for future work.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper revisits unsupervised domain translation (UDT) in light of optimal transport. The paper shows that CycleGAN-like models are ill-posed. It redefines UDT tasks using an additional set of suitable mappings. Then the paper redefines UDT problems in the optimal transport framework. Last it proposes an approach to solve UDT problems based on the dynamical formulation of optimal transport. Experiments support the proposed approach.

UDT is a relevant and up-to date problem. The paper helps to clarify some shortcomings of previous approaches and proposes a new solution. The paper is well written. Therefore, in my opinion, the paper should be accepted to ICLR. But, as I am not expert in optimal transport, I would like to have the exact reference of Theorem 1 because I would like to be sure that, in the proof of Proposition 3, the optimum of (A_c) is unique and therefore also satisfies the first item of Theorem 1.

Detailed comments.
* It should be fair to say somewhere that in Zhu et al. (2017a) limits of the approach were already mentioned
* As said before, you should give the exact reference of Theorem 1: which Theorem in Santambrogio (2015). In the proof of proposition 3, you should explain why the minimum of (A_c) is unique and thus corresponds to the minimum in Theorem 1.
* End of Section 3.1. The design of the cost function is left open. This should be made explicit and be discussed somewhere, perhaps in the conclusion. 
* I think that there should be a paragraph for the computation of the inverse. This question is considered in different parts of the paper. See for instance the caption of Figure 5. What is the meaning of "inverting the forward network" and to which part of the text does it refer?
* End of Section 4.1. As said before, the design of the cost function is sensitive. Did you have any idea of other cost who would allow to learn the targeted translation without using internal representations?

Typos. 
* The notation $T^{\alpha-a.s.} is difficult to read and should be explained
* Beginning of Section 3. "based the dynamical formulation of PT"
* Please check references in texts.  such as "from OT theory Santambrogio (2015)"
* Beginning of Section 3.2., "calculate the retrieve the OT mappings"
* Please give a reference for the dynamical formulation of OT. 
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The paper proposes an analysis of CycleGAN methods. Observing that the class of solutions (mappings) covered by those methods is very large, they propose to restrict the set of solutions by looking at low energy mappings (energy being defined wrt. a dedicated cost). A natural formulation of the associated problem is found in optimal transport (OT) theory. They examine the underlying problem in its dynamical formulation, for which a direct connection can be made with ResNet architecture that are commonly used in cycleGANs. They illustrate these results on simple examples, involving pairing swapped digits from MNIST and celebA male to female examples. As a matter of facts, results presented with the OT formulation are more constant. The main proposition of the paper is that the task at hand can be efficiently coded through the distance (cost) function of OT.

Overall the paper is well written and the proposition is reasonable. Yet some parts seem unnecessary long to me, or bring little information, notably the formalisation of 2.1 and 2.2. The fact that cycleGANs are severely ill-posed problems is well known from the computer vision community. Variants that can include a few paired samples can be found (not exhaustive): 
Tripathy, S., Kannala, J., & Rahtu, E. (2018, December). Learning image-to-image translation using paired and unpaired training samples. In Asian Conference on Computer Vision (pp. 51-66). Springer, Cham.
Oe that try to regularize the associated flow:
DLOW: Domain Flow for Adaptation and Generalization Rui Gong, Wen Li, Yuhua Chen, Luc Van Gool; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2477-2486

In this spirit, I wonder if a comparison with only the vanilla cycleGAN is sufficient to really assess the interest of using the OT formulation of the problem. Notably, in the example of digit swaps, a cost is learnt by finding a representation of the digits that eliminates the importance of position in the representation. Training such a classifier assumes having some labelled data, that could theoretically be paired, and thus making amenable variants of cycleGans that use a few paired samples. In this sense, I think that the paper fails in giving convincing arguments that advocate the use of OT here. As the dynamical formulation is known and already used to learn mappings (  see Trigila, G., & Tabak, E. G. (2016). Data‐driven optimal transport. Communications on Pure and Applied Mathematics, 69(4), 613-648. For instance). Also variants of OT that estimate a Monge mapping could have been included (e.g. V. Seguy, B. B. Damodaran, R. Flamary, N. Courty, A. Rolet, M. Blondel, Large-Scale Optimal Transport and Mapping Estimation, International Conference on Learning Representations (ICLR), 2018.)

As a summary:
Pros:
A nice interpretation of CycleGAN with OT
The paper is fairly well written
Cons: 
Overall the quantity of novelties is, in the eyes of the reviewer, somehow limited. At least the contributions should be clarified;
The experimental section is not convincing in explaining why the OT formulation is better than variants of cycleGAN or also other schemes for computing OT than the dynamical formulation

Minor remark:
A reference to Benamou, Brenier 2000 could have been given regarding section 3.2 and the dynamical formulation of OT. 

review_title: Official Blind Review #1

Meta-review: The paper examines the problem of unsupervised domain translation. It poses the problem in a rigorous way for the first time and examines the shortcomings of existing CycleGAN-based methods. Then the authors propose to consider the problem through the lens of Optimal Transport theory and formulate a practical algorithm.

The reviewers agree that the paper addresses an important problem, brings clarity to existing methods, and proposes an interesting approach / algorithm, and is well-written. However, there was a shared concern about whether the new approach just moves the complexity elsewhere (into the design of the cost function). The authors claim to have addressed in the rebuttal by adding an extra experiment, but the reviewers remained unconvinced.

Based on the reviewer discussion, I recommend rejection at this time, but look forward to seeing the revised paper at a future venue.
Decision: Reject