Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to use synthetic data to learn a better representation that can enforce the model to learn holistic features. The synthetic data is generated using persistent homology (PH). This method does not require real images. In the experiment, authors try to apply the learnt representation to downstream tasks. The experiment is done on the animal dataset, which consist of binary images. 
strength_and_weaknesses: Strength:
- Training a better representation without using real data is an interesting direction. This paper presents an innovative way to learn holistic features.

Negative:
- The effectiveness of this method is uncertain. It is uncertain whether the feature learned using this synthetic data can work well together with real data and tasks. The authors mentioned, "The image features obtained by PH are topological and global, whereas the features extracted through iterated convolutions by CNNs are mostly local."  This disadvantage might not hold for various cases. Many models can encode long-term context effectively.
- The experiment is not persuasive enough to show whether this method can be generalized to in-the-wild real data. The binary image is an effective way to prove the concept, but it is also helpful to include results for more challenging real data to show its effectiveness.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to understand.
The idea is simple and easy to follow. I think it is novel and interesting, but not certain whether it is effective for other datasets and other model architectures.
The authors will release the code to ensure reproducibility.
summary_of_the_review: The paper proposes an interesting way to generate synthetic data. Then use this data to train a better representation that can encode holistic information. The experiment is limited to a binary dataset, which makes it hard to judge whether the training strategy can work well on real-world data. Due to the challenge and complexity of real data, many existing models can already encode long-term context and have large receptive fields. This fact makes the advantage of the proposed method seem less significant. Based on these reasons, I think the paper is below the threshold of ICLR.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a technique to learn local and global visual representation via Persistent Homology.
The overall structure of the proposed method is to perform self-supervised learning (regression) on synthetic images for Persistent Homology as a pre-training, and then natural images for fine-tuning.

The synthetic image utilizes frequency-based (FFT) random generation method, where there are patterns at different scales
The PH label computation (using Cubical Ripser) aims at extracting shapes using contours by using signed distance function.
The author then utilizes four different vectorization techniques: the persistence image, the persistence landscape, the Betti number curve, and the birth-life histogram.

The algorithm is tested on ImageNet and CIFAR100. It also uses 2000 binary images of animal contours for a second (mid-step) topology learning.
That is, for ImageNet for example, it uses 90 epoch synthetic data, 300 epochs for animal dataset, then 45 epochs for ImageNet.
The first experiment is testing various pre-training (including vectorization techniques) and fine-tuning training combinations, as well as other methods, such as MoCo v2 and FractralDB-pre-trained models.

The PH-pre-trained models show better convergence than without pre-training (just fine-tuning) or fine-tuned twice, and comparable to FractalDB.
Among the vectorization, all are similar, with Betti count performs worst. This may be because of the lack of persistence structure learning.
The experiments also show that the number of PH-training images is important as there is only 1600 of the animal training images.

strength_and_weaknesses: Strengths: The paper explains the intentions and technical parts quite well. It is tested reasonably thorough, comparing the various vectorization techniques, as well as with other modules.

Weaknesses: it is hard to understand what is actually being learned. It would help to know what kind of images are now better classified after the pre-training and fine-tuning regiments to demonstrate the usage of structure learned in improving final task accuracy.
That is, would it be possible to show example testing images that are moving from incorrect to correct.
Also, it would be helpful to describe what types of global structures (and invariances) are eventually being learned by PH synthetic image pre-training.

clarity,_quality,_novelty_and_reproducibility: The paper explains the problem and technique clearly. 
summary_of_the_review: As stated the problem and solution (technique) is explained well. However, the testing and analysis can be more in-depth.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new scheme to pretrain the neural networks, which combines the calculated persistent homology of a sublevel set filtration of images as the topology information. The topology information aims to guide the neural networks to learn non-local topological structures. To avoid the human bias in the annotated datasets, the authors utilize synthetic images generated by specific formulas during the pre-training. The authors provide empirical evaluations and show that the topology-related pre-training scheme can improve the convergence with comparable accuracy on natural image classification tasks.
strength_and_weaknesses: **Strengths**:

1. The idea of guiding the neural networks to learn global structures with non-local features, like persistent homology is novel and interesting.

2. Pre-trained with synthetic images generated by specific formulas seems to prevent the human bias existing in the annotated datasets and ensure fairness to some extent.

3. The paper is clearly structured and well written.

**Weaknesses**:

1. The authors only provide empirical results, without giving theoretical analysis of the proposed pre-training scheme.

2. The authors mention the weights of all layers are updated during the phase of fine tuning. However, it is unclear that the neural network is still able to preserve the learned global topological features if we update all layers’ parameters during the phase of fine tuning.

3. The experiments are all conducted on small datasets of natural image classification tasks, which is not sufficient to indicate that the proposed scheme can be used in other larger datasets or different downstream tasks.

**Questions**:

1. It may be helpful to conduct experiments on how the learned global features change between the pre-training phase and the fine-tuning phase.

2. Using hand-crafted features to guide the training of neural networks seems alright. However, how to prove that your hand-crafted features actually reflect the global information of the training dataset? I would expect more theoretical analysis to support the main claim of this paper.

3. It seems that the models are only tested on rather small datasets within only the natural image classification tasks. Is it possible to conduct experiments on larger datasets on other scenarios, like segmentation or tracking, to see the performance?



clarity,_quality,_novelty_and_reproducibility: Clarity: This paper is well written and easy to follow.

Quality: This paper contributes some new ideas but the empirical evaluation is limited.

Novelty: The proposed idea is somehow interesting.

Reproducibility: I am optimistic to reproduce the main results of this paper as the paper provides lots of details.
summary_of_the_review: Overall, this paper contributes some new ideas but the empirical evaluation is limited. Also, it's unclear to me what the model is actually learning (interpretability).
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes to improve the ability of CNNs to capture long-term dependencies by supervising with an analytical topological signature on randomly generated synthetic images. The idea is novel, and the method is tested on a few popular datasets. The paper received mixed review scores (5, 5, 6). The major issues include the interpretation to how CNNs learn long-term dependency is inadequate and how this method differs from popular approaches such as transformers in learning global context is not well articulated/evaluated. 
justification_for_why_not_higher_score: See the weakness and summary of AC-reviewer meeting. The flaw is critical from the perspective of reviewers/meta-reviewer. Therefore, we recommend to reject the work.
Decision: Reject