Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a topology-guided sampling strategy (TGSS) to mitigate the gap between sampled data within a mini-batch and global data. The proposed model, which is called TopoZero, consists of a topology alignment module (TAM) and a distribution alignment module. TAM is capable of preserving multidimensional geometric structures in each of the latent visual and semantic spaces. Experimental results show that TopoZero performs well on standard ZSL benchmark datasets.
strength_and_weaknesses: - Strength

-- A topology guided sampling strategy (TGSS) is proposed to mitigate the gap between sampled data in a mini-batch and global data.

-- A theoretical analysis is presented for TGSS.

TopoZero shows good performance on standard ZSL benchmark datasets.

- Weaknesses

-- The paper is not self-contained, which hinders understanding of the paper.

-- The paper is poorly written, making it difficult to read the paper smoothly.

-- There are some typos.

-- Evaluations of the proposed method are insufficient.

clarity,_quality,_novelty_and_reproducibility: To mitigate the gap between sampled data within a mini-batch and global data, a topology-guided sampling strategy (TGSS) is proposed, which has certain novelties.

On the other hand, the paper is not self-contained, which hinders understanding of the contents of the paper. For example, the explanation of Persistent Homology on page 2 is not familiar to many readers, so the reviewer would like to see an intuitive and qualitative explanation in the text and a detailed explanation in the Appendix. The lack of explanations makes the middle part of the description in 3.2 difficult to understand.

The paper is not well written, and it is difficult to read through. For example, the description in 3.1.1 does not explain more than that the calculations are performed using Equations 1 and 2. Qualitative and intuitive descriptions of each equation should be provided. 

The description of equation 2 is particularly insufficient. At the time of reading this text, it would be more appropriate to write $\mathcal{C}$ instead of $\mathcal{U}$, and it would be more helpful to write $x$ as $x_{max}$.

Figure 2 is difficult to understand. In particular, the explanation of (d)-(f) is insufficient.

The evaluation of the proposed method is insufficient. The model is a combination of CADA-VAE and TAP, thus the performance of TAP alone should be presented.

Also, each batch must be sampled according to Algorithm 1. The reviewer would like to know how long this computation takes, and would like to see some experiments on the order of computation and actual computation time.

What is the problem of "The latent visual and semantic space fails to preserve multiple dimensional geometry structure, especially high dimensional structure information." The reviewer would like to see a detailed explanation as to whether this problem has been solved along with the basis for the experiment.

There are some spelling errors.
- In equation 2, the $($ in $(x_0$ is not necessary.
- In equation 2, it is $X$, not $\mathcal{X}$.
- In the definition of $X$ in 3.1.2, it is $x_m$, not $x_n$.

summary_of_the_review: Although the proposed method, called TGSS, has a certain degree of novelty and shows improvement in performance from the baselines, it is not considered to have reached the stage of publishing at this time due to a lack of experiments and insufficient explanation due to the low quality of the writing.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a topology-guided sampling strategy as a parallel pipeline to CADA-VAE. Empirical and theoretical analysis of the topology property is provided. The method is evaluated on the three common benchmarks of CUB SUN and AWA and achieves promising performance.
strength_and_weaknesses: + The motivation for topology structure alignment is clear and strong.
+ The theoretical analysis of TGSS is solid and inspiring.
+ The paper is well presented and easy to follow.
- The overall Objective Function (Eq 18 and 19 should be labelled once only) contains duplicates. For example, the reconstruction is considered multiple times in AE, TP and CA.
- The hyperparameter evaluation is odd. Given AE TP and CA are counting, visual-semantic reconstruction, topologic alignment, and distributional alignment, why the effects of the three hyperparameters are so consistent? It usually needs to find a balance between complementary objective functions. Otherwise, at least one of the terms must be useless.
- Despite the intuitive rationale in Figure 1 and solid theoretical analysis in 3.1, the idea is lack quantitative analysis supported by empirical examples.
clarity,_quality,_novelty_and_reproducibility: The paper is presented at very high quality. Intuitive illustrations are provided which makes the idea very easy to follow. The focus on the exploration of topological properties of visual-semantic information is novel and can have wide impacts in ZSL and ML domains.
summary_of_the_review: Overall, this work meets the standard expected by ICLR. However, the model design contains duplication and redundancy which normally indicates risks in "hyper-parameter overfitting". The evaluation of hyperparameters in the appendix is very contradictory and needs further elaboration.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposesa Topology-guided Sampling Strategy (TGSS) to mitigate the distribution gap between sampled and global data points for Zero-Shot Learning. In addition, a Topology Alignment Module (TAM) is proposed to perserve multi-dimensional geometry structure in latent visual and semantic space. The proposed method is evaluated on several benchmark datasets for ZSL and generalised ZSL and achieves superior performance.
strength_and_weaknesses: ++ The proposed methods facilitate the idea of topology preserving during representation learning which has been proven important for zero-shot learning. The proposed TGSS has been theoretically and empirically validated in this work.

++ The proposed method is justified theoretically and empirically. The experiments are thorough and convincing.

-- The authors fail to compare with more recent/advanced ZSL approaches in the experiments.

-- I wonder if the TGSS strategy also works for other ZSL frameworks than CADA-VAE.

-- The authors should have discussed the impact of batch sizes on the performance of TGSS. Why is the batch size set as 50 when the number of classes could be as large as 150/645? Is there any simple baseline methods as alternatives to TGSS? e.g., class-balanced sampling for each batch?
clarity,_quality,_novelty_and_reproducibility: The proposed methods are clearly presented and the paper is easy to follow; There is some novelty in the batch sampling strategy and topology preserving strategies for ZSL; The experimental results should be reproducible given the details of experimental settings in the paper.

summary_of_the_review: The authors aim to design an approach to facilitating topology preserving in ZSL from a novel perspective. The proposed method is proved to be effective when combined with a classic ZSL framework CADA-VAE which, however, is a relatively old technique for ZSL. The authors fail to compare with more advanced ZSL approaches.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a TopoZero framework to improve structure alignment for common space learning methods. Based on two weaknesses of HSVA (trains model with local structure and loses some high-dimensional structure information), the authors propose a Topology-guided sampling strategy and a Topology Alignment Module. Extensive experiments show the effectiveness of the proposed approach.
strength_and_weaknesses: Strength:
1. This paper is well-organized and well-written, which is easy to read.
2. The idea of topology alignment is novel, but it is only reflected in sampling data, where the relations among data are not modeled, which is a bit trivial. 

Weakness:
1. Figure 1 is difficult to understand. What are the lines in figure 1 (a-c) represent? What does ‘0-dim’ represent?
2. The framework is too completed. It contains 4 autoencoders, which seems an ensemble of the proposed approach and CADA-VAE. There are too many losses in the framework. Is it efficient to learn?
3. The representations in the formulation are inconsistent.
(a)Is ‘x’ in Eq 2 ‘x_max’?
(b)Are ‘Z_v’ in Eq 9 and ‘z_x’ in Eq 11 the same? ‘z_x’ is not mentioned in the paper.
4. All the compared methods are before 2022 and the performance is not good compared with the recent 2022 papers. 
5. The proposed method is an incremental work based on HSVA, why not use the same baseline with HSVA. It is difficult to judge whether the improvement is made by the proposed TAD or an ensemble of CADA-VAE.
6. No qualitative analysis is made to show why preserving the global structure is good. What does it benefit? Is it good to generate more effective features? However, no feature visualization is conducted.
clarity,_quality,_novelty_and_reproducibility: 1. The framework is clear and some formulations may be inconsistent.
2. The novelty may lie in the sampling of training examples and using the Persistent Homology to measure the difference between feature space and latent space.
3. The framework is too complicated, which is an ensemble of 4 autoencoders. Though each part is clear, the training may be difficult.
summary_of_the_review: The proposed approach aims to preserve the global topology of training samples to train the models, which seems novel. However,  the framework is too complicated, which ensembles CADA-VAE. It is difficult to judge which part improves the performance. Moreover, the compared methods are not recent state-of-the-arts and there is no qualitative analysis of the proposed approach.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Based on the collected information from all reviewers and my personal judgment, I can make the recommendation on this paper, **reject**. Here are the comments that I summarized, which include my opinion and evidence.

**Research Problem**

This paper studies the well-defined generalized zero-shot learning problem

**Presentation**

In general, this paper is difficult to follow. Many format issues including the abstract format on the OpenReview system, missing punctuation, incorrect citation format, spelling errors, and typos. Some sentences and paragraphs lack logic. For example, the authors criticize the existing design as sub-optimal. How to verify this point? What is the optimum? Can the proposed method achieve the optimum? In the first paragraph on the second page, the authors illustrate the two issues in HSVA. However, in what follows, the authors use the whole paragraph to introduce persistent homology. Figure 1 is also confusing, where many notations and concepts are used without any definition. 

**Motivation**

The motivations come from two aspects. (1) The structure in a mini-batch is different from the global one and (2) the latent visual and semantic space fail to preserve multiple dimensional geometry structure. Frankly speaking, these two motivations are too general and not strong.  

**Philosophy**

This paper lacks philosophy on how to tackle the above challenges. In Section 3, the authors directly introduce the algorithm, without any rationality. 

**Technique**

Several reviewers have concerns about too complicated techniques, which contain many losses and hyperparameters.  Please see their detailed comments.

**Experiments**

The authors only provide the performance on the benchmark datasets. Unfortunately, the authors fail to verify their motivations.

No objection was raised from the reviewers on the rejection recommendation.
justification_for_why_not_higher_score: This paper lacks clear motivations and presentation and its technical part is the combination of the existing losses. More details can be found in my meta review. In sum, the current version of this paper is not mature.
Decision: Reject