DiffuLT: Diffusion for Long-tail Recognition Without External Knowledge

Authors: jie  shao
ke  zhu
hanxiao  zhang
jianxin  wu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 16:27:43

Abstract: This paper introduces a novel pipeline for long-tail (LT) recognition that diverges from conventional strategies. Instead, it leverages the long-tailed dataset itself to generate a balanced proxy dataset without utilizing external data or model. We deploy a diffusion model trained from scratch on only the long-tailed dataset to create this proxy and verify the effectiveness of the data produced. Our analysis identifies approximately-in-distribution (AID) samples, which slightly deviate from the real data distribution and incorporate a blend of class information, as the crucial samples for enhancing the generative model's performance in long-tail classification. We promote the generation of AID samples during the training of a generative model by utilizing a feature extractor to guide the process and filter out detrimental samples during generation. Our approach, termed Diffusion model for Long-Tail recognition (DiffuLT), represents a pioneer application of generative models in long-tail recognition. DiffuLT achieves state-of-the-art results on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT, surpassing leading competitors by significant margins. Comprehensive ablations enhance the interpretability of our pipeline. Notably, the entire generative process is conducted without relying on external data or pre-trained model weights, which leads to its generalizability to real-world long-tailed scenarios.

Main paper: 
 introduction: 
 Deep learning has exhibited remarkable success across a spectrum of computer vision tasks, especially in image classification, e.g., as exhibited by He et al. [2016] , Dosovitskiy et al. [2021] , Liu et al. [2021] . These models, however, encounter obstacles when faced with real-world long-tailed (LT) data, where the majority classes have abundant samples but the minority ones are sparsely represented. The intrinsic bias of deep learning architectures towards more populous classes exacerbates this issue, leading to sub-optimal recognition of minority classes despite their critical importance in practical applications.Conventional long-tailed learning strategies such as re-weighting (Lin et al. [2017] , Cao et al. [2019a] ), re-sampling (Zhou et al. [2020a] , Zhang et al. [2021a] ), and structural adjustments (Wang et al. [2020] , Cui et al. [2022] ), share a commonality: they acknowledge the data's imbalance and focus on the training of models. They demand meticulous design and are challenging to generalize. Recently, a shift towards studying the dataset itself and involving more training samples through external knowledge to mitigate long-tailed challenges has emerged (Zhang et al. [2021b (Zhang et al. [ , 2024a]] ). Yet, in many real-world scenarios, access to specialized data is limited and closely guarded, such as in military or medical contexts. This predicament prompts a critical inquiry: Is it feasible to balance long-tailed datasets without depending on external resources or models?Figure 1 : The samples generated by diffusion models improve long-tail classification on CIFAR100-LT, showing a correlation between FID and accuracy and a stronger correlation between the proportion of AID samples and accuracy. Our method significantly boosts classifier accuracy compared with others (left). Feature space visualization reveals that different diffusion models generate samples with varying distributions, and our model biases the generative process toward AID samples (right).Our answer is yes. Recent advances in diffusion models have demonstrated their significant potential in generating high-quality images (Ho et al. [2020] , Song et al. [2020] , Rombach et al. [2022] ). Assuming that diffusion models are proficient at learning distributions, we develop a diffusion model trained from scratch on only the long-tail distributed dataset. This model creates new samples for underrepresented classes, which are then used to train a classifier on the re-balanced dataset, leading to improved accuracy. We are the first to demonstrate the effectiveness of using generated samples, without relying on external data or models, in improving long-tail classification. We observe a notable pattern: enhancing the performance of the generative model with a loss modification called CBDM (Qin et al. [2023] ) also enhances the classifier's accuracy, as illustrated in fig. 1 . This phenomenon implies that a generative model with better performance tends to generate samples that are more beneficial for the classification task. This observation raises an important question: What are the most valuable generated samples for classification, and how are they generated? This question is critical, as it determines whether a diffusion model is going to be beneficial or detrimental for LT recognition.We answer this question by analyzing features of generated samples, as visualized in fig. 1 using t-SNE (Van der Maaten and Hinton [2008] ). We categorize the generated samples into three groups: in-distribution (ID), approximately in-distribution (AID), and out-of-distribution (OOD). Our research indicates that AID samples are pivotal in enhancing classifier performance. Through experiments, we conclude that a diffusion model can assimilate patterns from the head classes and integrates them into the tail ones to produce AID samples. These samples significantly enhance the quantity and diversity of the tail classes, thereby substantially improving their performance. Then the important question to solve is: How can we generate AID samples efficiently?To encourage the model to predominantly generate AID samples, we introduce a novel type of loss. This loss employs a feature extractor to penalize the generation of ID and OOD samples. Such a strategy not only elevates the performance of the generative model on long-tail datasets but also renders it more effective and efficient in enhancing classifier performance.In general, we introduce a new pipeline, DiffuLT (Diffusion model for Long-Tail recognition), for long-tail datasets. It has three steps: initial training, sample generation, and retraining. Initially, we train a feature extractor and a diffusion model incorporating a supervision term to encourage the generation of AID samples. Subsequently, this generative model is employed to augment the dataset towards balance. The final step involves training a new classifier on the enriched dataset, with a minor adjustment to reduce the impact of synthetic samples. It is crucial to underscore the importance of training the diffusion model without external data or knowledge, to maintain fairness in comparison.Our contributions are summarized as follows:• We pioneer in addressing long-tail recognition by synthesizing images using diffusion models without relying on external data.• Our research delves into the mechanisms underlying our approach, highlighting the significance of the generated AID samples. These samples emerge from a fusion of information from both head and tail classes, playing a crucial role in enhancing classifier performance. • We introduce a novel loss function that enhances the performance of diffusion models on long-tailed datasets and biases them towards generating AID samples, thereby making the generation process more effective and efficient for classification.Extensive experimental validation across CIFAR10-LT, CIFAR100-LT, and ImageNet-LT datasets demonstrates the superior performance of our method over existing approaches.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The conclusions and methods in the abstract and introduction accurately encapsulate the contributions of our paper, as detailed in section 3. The experimental results discussed are fully documented in section 4, confirming the claims' validity.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 Long-tailed recognition Long-tailed recognition is a challenging and practical task (Cui et al. [2019] , Zhou et al. [2020b] , Cao et al. [2019b] , Zhang et al. [2023a] , Zhu et al. [2024] ), since natural data often constitute a squeezed and imbalanced distribution. The majority of traditional long-tailed learning methods can be viewed as (or special cases) of re-weighting (Cao et al. [2019b] , Kang et al. [2020] , Zhong et al. [2021a] , Wang et al. [2024a] ) and re-sampling (Cui et al. [2019] ), with more emphasis on the deferred tail class to seek an optimization trade-off. There are variants of them that adopt self-supervised learning (Zhu et al. [2023] , Li et al. [2021] ), theoretical analysis (Li et al. [2022] , Menon et al. [2021] , Yang et al. [2024] ) and decoupling pipeline (Kang et al. [2020] , Zhou et al. [2020b] ) to tackle long-tailed learning from various aspects, and they all achieve seemingly decent performance in downstream tasks.One of the core difficulties in long-tailed learning is the insufficiency of tail samples. And recently, quite some works start to focus on this aspect by involving more training samples through external knowledge (Zhang et al. [2021b] , Ramanathan et al. [2020] , Dong et al. [2022] , Shi et al. [2023a] ). Nevertheless, the most distinct drawback of these works is that they either rely on external data source or strong model weights. This condition can seldomly hold true in practical scenarios where only a handful of specialized data are available and are secretly kept (consider some important military or medical data). We thus raise a natural question about long-tailed learning: can we utilize the advantage of generating tail samples without resorting to any external data or model? That is, the whole process is done in an in-domain (also called held-in) manner. In this paper, we propose to adopt the off-the-shelf diffusion model to learn and generate samples from the data at hand.Diffusion models and synthetic data Diffusion models have been highly competitive in recent years (Ho et al. [2020] , Song et al. [2020] ), producing promising image quality in both unconditional and conditional settings (Dhariwal and Nichol [2021] , Rombach et al. [2022] , Ramesh et al. [2021] ). Despite the predominant use in creating digital art, the application of diffusion models in scenarios of limited data remains under-explored. This paper affirms the utility of diffusion models in enhancing representation learning, particularly within the long-tailed learning framework, offering a novel insight into their application beyond conventional generative tasks.The integration of synthetic data into deep learning, generated through methods like GANs Goodfellow et al. [2014] , Isola et al. [2017] and diffusion models (Dhariwal and Nichol [2021] , Rombach et al. [2022] ), has been explored to enhance performance in image classification (Kong et al. [2019] , Azizi et al. [2023] , Zhang et al. [2024a] , Trabucco et al. [2023] ), object detection (Zhang et al. [2023b] ), and semantic segmentation (Zhang et al. [2021c (Zhang et al. [ , 2023b]] ). These approaches often depend on substantial volumes of training data or leverage pre-trained models, such as Stable Diffusion, for high-quality data generation. Yet, the efficacy of generative models and synthetic data under the constraint of limited available data and in addressing imbalanced data distributions remains an unresolved inquiry. This paper specifically addresses this question, evaluating the viability of generative models and synthetic data in scenarios where data is scarce and imbalanced.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: The paper transparently acknowledges the main limitations in section 3.3 and section 5. Additionally, potential limitations are explored in the appendix B, ensuring a comprehensive discussion of the constraints and challenges encountered.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 For image classification, we have a long-tail dataset D = {(x i , y i )} N i=1 , y i ∈ C with each x i representing an input image and y i representing its corresponding label from the set of all classes The naive idea is to train a generative model θ on the long-tail dataset D and use the trained model to generate new samples and supplement the tail classes. Inspired by its superior performance, we select diffusion models as the generative model in our pipeline. In our approach, we follow the Denoising Diffusion Probabilistic Model (DDPM by Ho et al. [2020] ) framework. Given a dataset D = {x i , y i } N i=1 , we train a diffusion model to maximize the likelihood of the dataset. At every training step, we sample a mini-batch of images x 0 from the dataset and add noise to obtain x t ,EQUATIONwhere ᾱt = t i=1 (1 -β i ) is calculated through pre-defined variance schedule {β t ∈ (0, 1)} T t=1 . After training a diffusion model θ to get p θ (x t-1 | x t , t), we reverse the above process step by step to recover the original image x 0 from pure noise x T ∼ N (0, I). The training objective is to reduce the gap between the added noise in forward process and the estimated noise in reverse process:EQUATIONwhere ϵ t ∼ N (0, I) is the noise added to original images and ϵ θ is the noise estimated by the trainable model with parameters θ. DDPM can be conditional by transforming y into a trainable class embedding and incorporating the label y directly as am input, similar to time t. To improve the performance of DDPM on long-tailed dataset, several works (Qin et al. [2023] , Zhang et al. [2024b] ) have been proposed to adjust the distribution of generated samples. CBDM adds a distribution adjustment regularizer at the loss term. This term is designed to promote the generation of samples for tail classes, which is defined as (where sg means stop gradient): , where N gen = cj ∈C max(0, N t -|c j |) represents the total number of generated samples. These generated samples are then integrated with the original dataset, forming an augmented dataset D ∪ D gen , on which a classifier is trained to enhance classification performance.L CBDM = τ t |Y| y ′ ∈Y (∥ϵ θ (x t ,We conducted experiments on CIFAR100-LT with an imbalance ratio of 100 and set N t = 500 to supplement the data. The results, detailed in the second line of table 1, show a 5.5% accuracy increase for the classifier trained on D ∪ D gen compared to the baseline. This improvement underscores the effectiveness of our straightforward method in boosting overall performance. Considering the generated samples (especially for tail classes) may be of lower quality due to limited data availability, Class 90 (truck) is selected randomly as a representative example in CIFAR100-LT. A baseline classifier (φ 0 ), trained exclusively on the original dataset D, is used to analyze the generated data. This classifier extracts features which are then visualized using t-SNE, as shown in fig. 2 . The visualization reveals that samples generated via CBDM tend to be more centralized. For deeper analysis, we define the center f o of a class's features as the average of the real data in feature space, and set the maximum Euclidean distance between two real samples' features as a threshold d f . We then define 3 types of the generated samples based on their distance to f o :EQUATIONwhere ID denotes in-distribution samples, which closely match the patterns of the original data. We define and name approximately in-distribution (AID) samples, which exhibit slight deviations. OOD stands for out-of-distribution ones, which are significantly differing from the center. We summarize the composition of samples generated by each model in table 2 . Notably, the CBDM model generates a lower proportion of OOD samples, consistent with its FID score. For evaluating the impact of each type, we train classifiers using only the ID, AID, and OOD samples generated by CBDM with τ = 1 respectively as D gen , combined with D, and present the results in table 3 . Surprisingly, classifiers trained with AID samples achieve the highest accuracy and show the greatest average improvement per sample. Based on this finding, our hypothesis is that AID samples are the most beneficial in enhancing classifier performance.Mechanisms behind the AID samples. We conducted experiments to explore how AID samples enhance classifier performance and where their new and useful information originates. A diffusion model (CBDM with τ = 1) is trained using images from tail classes (fewer than 100 samples), supplemented by a variable proportion p h of head class images. This model generates samples Generation of AID samples. How can we efficiently generate AID samples? While a filtering strategy can be used to collect AID samples, it is not the most efficient method. A more effective approach involves encouraging the generation model to specifically produce AID samples. Given that AID samples are defined by their distance from the center of real images in feature space, we can utilize the baseline classifier φ 0 as a feature extractor to guide the generation of AID samples.Our goal is to encourage a controlled deviation within feature space. After T denoising steps, the deviation should ideally be within the range of d f to 2d f . Assuming that the deviation in each step is proportional to the noise strength, we introduce an additional term in the loss function to encourage small, stepwise deviations. We define the deviation at each step in feature space asEQUATIONwhere φ 0 (x 0 + ϵ t -ϵ θ (x t , t, y)) represents the de-noised images' feature. The new AID loss is thenEQUATION)where α is a hyper-parameter and defaulted to 0.1. We incorporate this term into both L DDPM and L CBDM to train the generation model. After training, we use this model to generate data. During the generation process, we employ φ 0 to filter out harmful OOD samples, resulting in D gen . We then train the classifier using the combined dataset D ∪ D gen . Recognizing that generated data are less crucial than real images, we introduce a weighting term to the cross-entropy loss to adjust the influence of the generated samples:EQUATIONwhere ω controls the weight of generated samples and is set to 0.3 by default. y g is an additional label assigned to each image x, which distinguishes between generated and original samples. Specifically, y g = 1 is used for generated samples, while y g = 0 marks the original ones.Now we are ready propose a new pipeline called DiffuLT to address long-tail recognition. The pipeline is shown in fig. 4 with four steps:• Training: Initially, we train a feature extractor φ 0 and a conditional, AID-biased diffusion model θ using the original long-tailed dataset D alone. • Generating: We establish a threshold N t and employ the trained diffusion model θ to generate and supplement samples. Using φ 0 , we filter out OOD samples, resulting in a refined dataset D gen .• Training: We then train a new classifier f φ on the augmented dataset D ∪ D gen using weighted cross-entropy, forming our final model.Compared to traditional methods that focus primarily on training, ours not only enhances performance but is also reusable for model updates. Our method requires more training time, typically four times longer, to train the generation model and produce samples. However, our methods prove valuable when performance improvement is critical. Unlike typical data expansion methods, our approach offers both practical and theoretical benefits because it don't rely on any external dataset or model. For detailed analysis, please refer to the appendix B.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA] Justification: The focus of paper is primarily empirical; thus, our conclusions and methods are derived from and validated by experimental results rather than theoretical proofs.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experimental setup: 
 Datasets. Our research evaluate three long-tailed datasets: CIFAR10-LT (Cao et al. [2019a] ), CIFAR100-LT (Cao et al. [2019a] ), and ImageNet-LT (Liu et al. [2019a] ). Following the methodology described in (Cao et al. [2019a] ), we construct long-tailed versions of the first two datasets by adjusting the long-tail ratio r to 100, 50, and 10 to test our method against various levels of imbalance.Baselines. In our comparative analysis, we benchmark against a broad spectrum of classical and contemporary long-tailed learning strategies. The methods compared can be classified into multiple genres like re-weighting and re-sampling techniques, head-to-tail knowledge transfer approaches, dataaugmentation, and so on. Some methods have issues such as unfair comparisons or implementation problems. We document both their results and our implementation outcomes in the appendix A.Implementation. We set α = 0.1, and ω = 0.3. The generation thresholds N t for CIFAR10-LT and CIFAR100-LT were fixed at 5000 and 500, respectively. We employ ResNet-32 as the classifier backbone. For ImageNet-LT experiments, we set a generation threshold of N t = 300. The classifiers were based on ResNet-10 and ResNet-50 architectures with ω = 0.5.More details about the experimental setup are available in the appendix A.Generative Results. We assess the efficacy of our specially designed loss function, detailed in table 5. This function improves the FID by reducing OOD samples, while also increasing the number of AID samples and the classifier's accuracy. Further experiments in table 6 highlight the necessity of our training loss. For benchmarking, we use a basic filtering strategy for CBDM, with all models generating samples to meet the threshold N t for each class. The terms "Kept" and "G-Num" denote the types of samples retained and the total number of samples generated before filtering, respectively. Our methods enhance the generation process's efficiency and achieve the highest accuracy.CIFAR100-LT & CIFAR10-LT. We benchmark our approach against a range of methods on the CIFAR100-LT and CIFAR10-LT datasets, with results detailed in table 7. The results not shown in the original papers are indicated as "-" in the table. On CIFAR100-LT, our method surpasses competing models, achieving accuracy improvements of 13.2%, 12.4%, and 8.1% compared with the baseline for r = 100, 50, and 10, respectively. On CIFAR10-LT, our model also demonstrates strong competitiveness, enhancing accuracy by 14.3%, 12.1%, and 4.3% across the long-tail ratios, further validating the effectiveness of our method. Since our methods solely modify the training data, they can be easily integrated with other methods to achieve better results.For CIFAR100-LT with an imbalanced ratio of 100, performance is also assessed across three categories: many (classes with over 100 samples), medium (classes with 20 to 100 samples), and few (classes with fewer than 20 samples). While our approach does not lead in the "Many" category, it excels in "Med." and "Few", significantly outperforming others in the "Few" group with a 29.7% accuracy -8.3% above the nearest competitor and 20.6% beyond the baseline.ImageNet-LT. On the ImageNet-LT dataset, our methodology is evaluated against existing approaches, with results summarized in table 8. Utilizing a ResNet-10 backbone, our method registers a 50.4% accuracy, outperforming the nearest competitor by 4.5%. With ResNet-50, the accuracy further escalates to 56.4%, marking a substantial 14.8% enhancement over the baseline. Despite a slight decline in the "Many" category relative to the baseline, our approach excels in "Med." and 'Few", with the latter witnessing a remarkable 33.6% improvement over the baseline. Our method can be combined with others to achieve enhanced results. Different modules in our pipeline. Our methodology comprises several critical components: generated samples (using CBDM), AID-biased loss, filtering, and weighted cross-entropy. We conduct ablation experiments on CIFAR100-LT with r = 100. The results, presented in table 9, highlight the crucial role each component plays in enhancing the overall performance. Notably, the generated samples and AID-biased loss are the most influential factors.Hyper-parameters. We adjust the parameters ω in the weighted cross-entropy and α in L AID on CIFAR100-LT with r = 100 and evaluate the classification results. These results are summarized in table 10 . Through iterative adjustments, we find that the optimal performance, a 51.5% classification accuracy, is achieved when ω = 0.3. Similarly, the best setting for α is determined to be 0.1. Consequently, we establish ω = 0.3 and α = 0.1 as the default settings for our method.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We have developed a straightforward and easily replicable pipeline, fully detailed in section 3. All necessary hyper-parameters and experimental settings are outlined in section 4.1. Additional information required for reproduction is provided in the appendix A. 
conclusion: 
 In this research, we proposed a novel, data-centric approach designed to address the challenges of long-tail classification. We defined and identified AID (approximately in-distribution) samples as the important ones. We then revised a diffusion model trained with an AID-biased loss term on only the original dataset for the purpose of generating more AID samples, thereby significantly enriching the dataset. Following sample generation, we trained a classifier on this enhanced dataset and employed a weighted cross-entropy loss. Our method has shown to deliver competitive performance, highlighting its efficacy in real-world applications. The experiments conducted as part of this study notably emphasize the critical role played by AID samples and their significant impact.We propose that this approach introduces a new paradigm for tackling long-tail classification challenges, offering a substantial complement to existing methodologies. It provides a robust framework that can be adapted to various scenarios where performance is a critical factor. Despite its advantages, the training of the diffusion model and the generation of samples are time-consuming. The need for optimization in training and generation speeds represents a limitation of our current method. We will leave this point as future work to further improve the effectiveness and efficiency of our method. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Our research doesn't involve models capable of generating high-risk content nor does it collect data from the internet, thus eliminating the need for such safeguards. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: All the data, code, and methods employed in this paper are open-source. We ensure proper citation of these resources and adherence to their licenses and terms of use. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: This paper does not introduce any new assets, hence there is no associated documentation.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: This paper does not involve any crowdsourcing experiments or research with human subjects, therefore this question is not applicable.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? 
 
 Appendix: A.1 Experimental settingsDue to space constraints in the main paper, we only include essential information about the experimental setup. Additional details are provided here.CIFAR100-LT & CIFAR10-LT. The original CIFAR100 and CIFAR10 datasets each consist of a training set with 50,000 images evenly distributed across 100 or 10 classes, respectively. The CIFAR100-LT and CIFAR10-LT datasets, derived from CIFAR100 Krizhevsky [2009] and CI-FAR10 Krizhevsky [2009] , feature a long-tail distribution where the class frequency decreases exponentially from class 0 to the last class. ImageNet-LT. ImageNet-LT, comprising 115,846 images across 1,000 classes with a maximum of 1,280 images per class and a minimum of 5, follows the specifications set in (Liu et al. [2019a] ). This dataset is derived from ImageNet Russakovsky et al. [2015] by sampling a subset according to a Pareto distribution with a power value of α = 6. For perspective, the original ImageNet training set contains 1,281,167 images, making ImageNet-LT less than 10% the size of the original dataset. The test set of ImageNet-LT mirrors that of ImageNet, containing 100,000 images.Due to its considerable size and image resolution, ImageNet-LT necessitates modifications from the standard CBDM framework to address inefficiencies. We adapt the codebase from Dhariwal and Nichol [2021] for this purpose, extending the training to 1,980,000 iterations. This setup uses the AdamW optimizer with a learning rate of 3e-4 and a batch size of 64, adjusted from the original 256 due to GPU memory constraints. Moreover, the model is trained at an image resolution of 256, and the total diffusion time step is set to 1,000.Classifier training for ImageNet-LT employs the framework from Zhang et al. [2021a] . Specifically, the model is trained over 100 epochs with a batch size of 512 using the SGD optimizer at a learning rate of 0.2. All training tasks are carried out on 8× NVIDIA GeForce RTX 3090 GPUs.We synthesize images for CIFAR-100, CIFAR-10, and ImageNet-LT to demonstrate their utility in enhancing long-tail recognition. Randonly selected examples of the generated samples are displayed, particularly for CIFAR100-LT in fig. 5 , where images have a resolution of 32 × 32 and focus on "few-shot" classes-those with fewer than 20 instances. Despite the limited examples available in these classes, our diffusion model effectively utilizes the entire dataset to produce high-quality samples. While these synthesized images maintain some similarities with their original counterparts, the notable variations make them valuable for model training. However, because the generated images may sometimes distort essential features or introduce inaccuracies, it becomes crucial to filter out the OOD samples to maintain their usefulness.In fig. 6 , we showcase the generated images for CIFAR10-LT, which demonstrate superior quality compared to those from CIFAR100-LT, due to a more abundant training dataset. In CIFAR10-LT, 50.4 56.5 DiffuLT (3) 50.5 56.5 each class generally contains ten times more images than in CIFAR100-LT. The smallest class in CIFAR10-LT has 50 images, greatly exceeding the minimum of 5 in CIFAR100-LT. This increase in sample size, however, requires a more stringent filtering process to prevent potential information loss due to the larger volume of images generated.For ImageNet-LT, the generated images, displayed in fig. 7 , feature a resolution of 224 × 224, which is significantly clearer than those from CIFAR-10 and CIFAR-100. While some finer details, such as text within the image or textures like fur, may not be fully distinct, the generated images effectively capture the essential patterns of the classes. These images can significantly aid the long-tail recognition task, particularly for classes with fewer original samples.Since CIFAR100-LT and CIFAR10-LT are typically sampled randomly from their original datasets, we tested our methods across various sampled sets to assess their effectiveness and robustness. The results, displayed in table 11, demonstrate that our method's performance is stable, exhibiting only minimal variations. For ImageNet-LT, which is a fixed dataset, we generated samples three times to 12 . Therefore, our method proves to be robust in enhancing long-tail classification performance.A.4 Details of Baseline Methods.In the main paper, we outline various approaches to long-tail classification and benchmark our methods against these strategies. Here, we provide a concise overview of the methods within each category. For re-weighting and re-sampling techniques, we examine Cross-Entropy (CE), Focal Loss (Lin et al. [2017] ), LDAM-DRW (Cao et al. [2019a] ), cRT (Kang et al. [2019] ), BBN (Zhou et al. [2020a] ), CSA (Shi et al. [2023b] ), and ADRW (Wang et al. [2024b] ). In the realm of head-to-tail knowledge transfer, we include methods such as OLTR (Liu et al. [2019b] ) and H2T (Li et al. [2023] ). Label-smoothing strategies are represented by MisLAS (Zhong et al. [2021b] ) and DiVE (He et al. [2021] ), while in data augmentation, we compare our approach with CAM-BS (Zhang et al. [2021a] ), CMO (Park et al. [2022] ), and CUDA (Ahn et al. [2023] ). Lastly, SAM (Rangwani et al. [2022] ) exemplifies an advanced optimization technique, and RIDE (Wang et al. [2020] ) showcases a mixture of expert technique in our comparison. A.5 Other Methods.For various reasons, some methods are not included in our experimental comparisons. This decision primarily stems from two factors. Firstly, several methods, despite demonstrating impressive results, do not have publicly available code (Zada et al. [2022] ), limiting our ability to perform direct comparisons. Secondly, methods such as those in Cui et al. [2021] and Cui et al. [2023] achieve commendable results and can be replicated. However, their comparisons may be considered unfair. These methods utilize AutoAugment (Cubuk et al. [2018] ), with parameters optimized across the entire dataset, rather than specifically for the long-tailed segment. This approach significantly boosts their baseline performance, as reported in Ren et al. [2020] . For instance, on CIFAR100-LT with an imbalance ratio of 0.1, baseline accuracy improves from 38.3 to 45.3, as shown in table 13 . The first "Baseline" line reflects the standard settings, while entries marked with * use the enhanced settings, demonstrating a substantial improvement. Comparing these results with those obtained under standard conditions would be unfair. Our methodology could also be adapted to such settings and combined with these techniques to achieve excellent results, as illustrated in table 13. However, these results are omitted from the main paper to maintain a fair comparison.B.1 Comparison of our method with other type of methods.The comparison is divided into two parts: comparing our methods with traditional long-tail recognition approaches and contrasting them with data synthesis methods, as shown in fig. 8 .Compared to traditional long-tail classification methods that predominantly focus on training, our approach offers a novel perspective. Rather than designing intricate methods to facilitate training on long-tailed datasets, we straightforwardly enhance the dataset using a generative model. This approach is not only innovative but also compatible with existing training methodologies and demonstrates improved performance. However, the main limitation is the training of the generative model, which is time-consuming and challenging to scale. These points will be further discussed in the subsequent subsection.When comparing with data synthesis methods, it is clear that our approach does not surpass those employing technologies like Stable Diffusion or CLIP on long-tailed datasets. Nevertheless, the use of such large models trained on extensive data eliminates the core challenge of long-tail problems-data scarcity. For instance, the class "train" in CIFAR100-LT has only ten images, whereas Stable Diffusion has been trained on thousands of train images. In practical settings, accessing such expansive models tailored to specific datasets is unrealistic. These methods mainly benefit from data leakage. Our approach, in contrast, is designed for real long-tail scenarios without reliance on external data or models, providing practical and theoretical value. We address several previously unanswered questions:• Is a diffusion model trained from scratch beneficial for long-tail recognition? Yes. The primary limitation of our methods is the extensive training time required for the generative model. For instance, training a diffusion model on CIFAR100-LT takes 24 hours, while ImageNet-LT requires approximately six days. As the quality and quantity of data increase, the training costs scale up significantly, making it challenging to apply our methods to larger datasets such as iNaturalist and Places-LT due to resource and time constraints.Despite these challenges, our methods are highly effective in addressing the long-tail recognition problem. To improve training efficiency, we are exploring two potential solutions. The first involves adopting techniques that accelerate the training and inference processes of diffusion models. The second strategy considers the use of pre-trained generative models in real long-tail scenarios. This approach does not contradict our previous assertion that using large, pre-trained models on familiar long-tailed data is unfair and not truly representative of long-tail challenges. Instead, we advocate for the use of pre-trained models on long-tail datasets they have not previously encountered, ensuring fairness and practical applicability. Employing a pre-trained model could significantly expedite our pipeline by eliminating the need to train from scratch. This topic extends beyond the scope of this paper and will be addressed in future research.NeurIPS Paper Checklist• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The code related to this paper will be released as open-source after it is accepted.Guidelines:• The answer NA means that paper does not include experiments requiring code. Justification: Our research concentrates on the long-tail recognition issue, a technical challenge within the field of computer vision that typically has minimal societal impact. We utilize open-source datasets and ensure that no harmful information is generated. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: This work designs a diffusion based long-tail recognition method - DiffuLT, leveraging diffusion models to generate samples to balance the long-tailed dataset. Authors found that approximately-in-distribution (AID) samples are important in enriching tail class samples , and then they proposed to incorporate revised diffusion model to generate such samples. The idea is new and the performance gain is good. The authors' rebuttals have addressed most of the concerns of the reviewers.

Reviews: review: paper_summary: paper_summary: This paper introduces a diffusion based long-tail recognition method termed DiffuLT. DiffuLT uses diffusion model to generate supplement samples in order to balance the long-tailed dataset. The authors first discover that approximately-in-distribution (AID) samples are crucial in enhancing the generative model’s performance in long-tail classification, as it blends the useful information from other classes into the tail class. Inspired by such discovery, the authors propose to first train a feature extractor $\varphi_0$ merely from the long-tail dataset, then utilize  $\varphi_0$ to guide the generation of AID samples, and finally train the desired classification model using the original dataset and newly generated AID samples together. Experiments validate the enhanced performance of DiffuLT.
strength_weakness: strength_weakness: 1. It would be better if you show how “centralized” figure 2 is, for example, add support data of the proportion of ID, AID and OOD data. This figure is not intuitive enough. 
2. Citation suggestions. I believe your work would benefit from referencing some additional literature to provide a more comprehensive context for your study. Specifically, i recommend citing the following articles:
  -  A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning (NeurIPS 23)
  - Deep long-tailed learning: A survey (TPAMI 23)
  - Effective Data Augmentation With Diffusion Models (ICLR 24)
  - Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition (ICML 24)
3. Minor typos. Line 137 “as *an* input”. Line 227 “because it *doesn’t* rely on any *external* dataset or model”.
questions: questions: Could you explain how your method differs from other data augmentation methods?
limitations: limitations: See *Weaknesses*.
paper_summary: paper_summary: This paper introduces a novel pipeline for long-tail recognition that differs from conventional strategies by focusing on dataset composition. The method focuses on the dataset and introduces a diffusion model, namely DDPM to generate data for tail classes. The analysis reveals that approximately-in-distribution (AID) samples, which slightly deviate from the real data distribution and blend class information, are essential for enhancing the generative model’s performance in long-tail classification. Experiments conducted on CIFAR 10-LT, CIFAR 100-LT, and ImageNet-LT demonstrate significant performance improvements.
strength_weakness: strength_weakness: 1.	Some typos: line 137 ‘am input’
2.	The baseline setup in Table 1,2,3 requires further explanation.
3.	The paper mentions the time cost is four times longer than baseline, indicating that this method may be impractical for datasets with a significantly large imbalance ratio due to the extended generation and retraining times.
4.	Some baseline results in Tables 8 and 9 are indicated as ‘-’ and ‘ ’; an explanation for these forms of absence is missing.
5.	This method focuses on generating data for tail classes, resulting in significant improvements for these classes compared to other baseline methods. A fairer comparison would be with the same pipeline using different generative models.
6.	A long-tailed diffusion model baseline[1] is missing.
7.	The experiment are mainly conducted on manually-selected long-tailed datasets, it would improve the robustness of results by including real-world long-tailed datasets such as iNaturalist.

[1] Long-tailed Diffusion Models with Oriented Calibration. ICLR 2024.
questions: questions: please refer to the weakness part.
limitations: limitations: The limitations are discussed in this paper. No potential negative social impact.
paper_summary: paper_summary: This article uses a Diffusion model to generate tail class samples, achieving balance among classes at the data level and thereby addressing the long-tail distribution problem. Additionally, the authors believe that AID samples are the most beneficial in enhancing classifier performance; therefore, they design an L_AID loss to constrain the diffusion model to generate richer AID samples. Qualitative and quantitative comparisons have been made on three benchmark datasets, showing the efficiency of their method.
strength_weakness: strength_weakness: 1. The innovation is limited. Essentially, the authors' method designs a reconstruction loss to constrain the generative model to learn a distribution more consistent with real distributions, and many existing studies have proposed similar ideas.
2. The authors emphasize that this method has generalizability in real-world applications; however, I hold a negative view on this point. I believe that the performance improvement observed in this paper is fundamentally due to training data and test data having identical distributions. Therefore, constraining the diffusion model to learn a distribution closer to training data benefits classifier learning. However, in real-world applications where limited training data may not share identical distributions with potentially infinite test data, using this method might lead to negative impacts.
3. On what principle was Equation 4 determined? Why define [df, 2df] as AID? What impact would modifying this equation? For instance, would defining [df, \sqrt(2)df] as AID affect performance?
4. In Table 6, why does combining ID and AID result in worse performance than using only AID for CBDM?
5. Authors should provide ablation studies for Nt.
questions: questions: Please refer to weaknesses for corresponding questions.
limitations: limitations: The authors have addressed the limitation of this method.  This method will significantly increase training time.

Comments: title: Gratitude for your response and additional explanation
Thanks for raising your score
Discussion of novelty and methodological design
Follow-up
Follow-up
comment: Dear Reviewer,

Thank you for your prompt response and patience. We greatly appreciate your valuable advice and your support for our paper.

Due to an oversight, we did not provide a satisfactory answer to your question. Recognizing its importance, we would like to address it comprehensively here.

### Q1

> Could you explain how your method differs from other data augmentation methods?

We conducted a thorough survey of data augmentation methods for long-tail learning, categorizing them into two groups: feature augmentation and image augmentation.

Feature augmentation methods, such as RSF[1] and OFA[2], primarily focus on biasing the decision boundary towards tail classes. These methods are similar to re-weighting or re-sampling techniques and do not inherently increase data diversity. Moreover, because features are indirect and often difficult to map back to images, these methods differ fundamentally from image augmentation techniques.

Image augmentation methods share our motivation of increasing dataset richness. We specifically examined M2m[3], MiSLAS[4], Remix[5], CMO[6], and CSA[7], discussing each as follows:

- **M2m**: This method augments tail classes by transforming head-class samples into tail-class ones through perturbation-based optimization. However, the generated samples often closely resemble the originals (the head images), failing to truly enrich the diversity of tail classes.
- **MiSLAS and Remix**: These methods use Mixup for augmentation. However, MiSLAS experiments indicate that applying Mixup during classifier learning does not yield significant improvements and may even harm performance, requiring additional strategies to mitigate this issue.
- **CMO and CSA**: Both methods introduce rich contexts from majority samples into minority samples. CMO uses CutMix, while CSA employs a more precise segmentation approach. While effective in enriching context, they do not enhance the diversity of objects within tail classes.

Data richness, particularly in tail classes, is crucial for addressing the long-tail problem. While recent methods have acknowledged this and used data augmentation to tackle it, their effectiveness is limited to context enrichment. In contrast, diffusion models in our pipeline represent the **most powerful data augmentation method**. Unlike mixup-based approaches, diffusion models avoid domain shift by accurately estimating data distributions. Additionally, they enrich both context and objects, offering greater diversity since they combine the entire dataset rather than just two images like CMO or CSA. Therefore, with proper utilization, diffusion models hold the highest potential. Below is a performance comparison of various data augmentation methods, including ours, on CIFAR100-LT with an imbalance ratio of 100, using ResNet-32 as the backbone.

| Method | CIFAR100-LT Acc (%) |
|--------|---------------------|
| RSF    | 43.1                |
| OSA    | 42.9                |
| M2m    | 43.5                |
| MiSLAS | 47.0                |
| Remix  | 46.8                |
| CMO    | 47.2                |
| CSA    | 46.6                |
| Ours   | 51.5                |

[1] RSG: A Simple but Effective Module for Learning Imbalanced Datasets. CVPR 2021.

[2] Feature Space Augmentation for Long-Tailed Data. ECCV 2020.

[3] M2m: Imbalanced Classification via Major-to-minor Translation. CVPR 2020.

[4] Improving Calibration for Long-Tailed Recognition. CVPR 2021.

[5] Remix: Rebalanced Mixup. ECCV workshop 2020.

[6] The Majority Can Help the Minority: Context-rich Minority Oversampling for Long-tailed Classification. CVPR 2022.

[7] How Re-sampling Helps for Long-Tail Learning? NeurIPS 2023.
Dear Reviewer,

Thank you for raising your score! We're encouraged that our rebuttal addressed your concerns, and we sincerely appreciate your support for our work.
Dear Reviewer,

Thank you for your feedback and patience. We would like to address your concerns regarding the novelty and methodological design of our work.

### Novelty

The novelty of our approach lies in two main areas: primarily in the long-tailed learning domain and, to a lesser extent, in the generative modeling domain.

**Long-Tailed Learning:**

- **Diffusion Model without External Knowledge:** Our key contribution is the novel application of diffusion-based models to long-tailed learning ***without relying on external knowledge***. While recent works like [1] and [2] utilize Stable Diffusion to address data scarcity in classification tasks, they often face criticism due to the extensive pretraining on ***external large datasets*** (much larger than those for classification tasks), which can overshadow the novelty of their approach. We specifically address the question of whether a generative model, trained only on the limited data available in classification tasks, can still be effective. Our work demonstrates that this is indeed possible for the first time, providing a significant insight, especially when the training data is not composed of natural images (as discussed in our response to W3 of reviewer #vaue). This contribution may open new research avenues within the long-tailed learning community, offering a fresh perspective on the integration of generative models for classification tasks.

- **Understanding Diffusion Models:** We also provide an explanation of why diffusion models, even without external knowledge, are effective in this context. Our work is the first to introduce the concept of AID samples and the role of generation model in blending information of each class to generate beneficial and novel samples.

**Generative Modeling:**

- **A New Objective for Generative Models:** Traditionally, models like VQ-GAN are trained with objectives such as reconstruction loss and perceptual loss to generate realistic images. Our work introduces a novel objective: optimizing the generative model to produce images that are beneficial for classification tasks. This represents a significant shift from the traditional goal of generating visually appealing images to a more practical application, where the generated images directly enhance discriminative tasks. This objective is inherently challenging to optimize, and our introduction of AID samples provides a novel pathway for achieving this goal.

### Methodological Design

To address your concerns, we highlight the key designs of our method:

**New Designs:**

- **Classification of Generated Images:** We classify generated images based on their distribution in the feature space relative to real data.
- **Novel Loss Function:** We introduce a new loss function that uses classifiers to guide the training of the generative model. This simultaneously addresses the long-tailed generation problem and enhances the utility of the generative model for classification tasks.
- **Diffusion Model without External Knowledge:** As previously mentioned, our method employs a diffusion model trained without external knowledge to generate images that aid long-tailed learning.
- **Weighted Synthetic Samples:** We apply weights to synthetic samples during backpropagation to improve training efficacy.

**Adapted Designs:**
- **Sample Filtering:** We use a trained classifier to filter out harmful samples, similar to existing methods but without relying on models like CLIP.

We hope this addresses your concerns. Please feel free to reach out if further discussion or additional experiments are needed. We look forward to your feedback. Thank you again!

[1] Expanding Small-Scale Datasets with Guided Imagination. NeurIPS 2023.

[2] Feedback-guided Data Synthesis for Imbalanced Classification. NeurIPS Workshop 2023.
Thanks for the response. Most of my concerns have been addressed and I would like to raise my score to '5'.
Thank you for the reply, I have read your rebuttal and the comments from other reviewers. I believe the novelty and methodological design of this paper do not convince me to immediately revise the score. I will wait until the Reviewer-AC Discussions phase to decide on the final score.
Thank you for your response. I would keep my score.
Dear Reviewer,

We hope that our rebuttal has effectively clarified any confusion and that the additional experiments we provided have strengthened the validity of our approach. We eagerly await your feedback on whether our response has adequately resolved your concerns or if further clarification is needed.
Dear Reviewer,

We hope that our rebuttal has effectively clarified any confusion and that the additional experiments we provided have strengthened the validity of our approach. We eagerly await your feedback on whether our response has adequately resolved your concerns or if further clarification is needed.