Gandalf : Data Augmentation is all you need for Extreme Classification
Authors: Siddhant Kharbanda
Devaansh Gupta
Erik Schultheis
Atmadeep Banerjee
Vikas Verma
Rohit Babbar
Venue: Submitted to ICLR 2023 (2023)
Keywords: Extreme Classification
Data Augmentation
Search and Recommendation

Abstract: Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent works in this domain have increasingly focused on the problem setting with short-text input data, and labels endowed with short textual descriptions called label features. Short-text XMC with label features has found numerous applications in areas such as prediction of related searches, title-based product recommendation, bid-phrase suggestion, amongst others. In this paper, we propose Gandalf, a graph induced data augmentation based on label features, such that the generated data-points can supplement the training distribution. By exploiting the characteristics of the short-text XMC problem, it leverages the label features to construct valid training instances, and uses the label graph for generating the corresponding soft-label targets, hence effectively capturing the label-label correlations. While most recent advances (such as SiameseXML and ECLARE) in XMC have been algorithmic, mainly aimed towards developing novel deep-learning architectures, our data-centric augmentation approach is orthogonal to these methodologies. We demonstrate the generality and effectiveness of Gandalf by showing up to 30% relative improvements for 5 state-of-the-art algorithms across 4 benchmark datasets consisting of up to 1.3 million labels. 

pipelines over Under review as a conference paper at ICLR 2023
GANDALF: DATA AUGMENTATION IS ALL YOU NEED FOR EXTREME CLASSIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
Extreme Multi-label Text Classification (XMC) involves learning classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent works in this domain have increasingly focused on the problem setting with short-text input data, and labels endowed with short textual descrip- tions called label features. Short-text XMC with label features has found numerous applications in areas such as prediction of related searches, title-based product recommendation, bid-phrase suggestion, amongst others. In this paper, we propose Gandalf, a graph induced data augmentation based on label features, such that the generated data-points can supplement the training distribution. By exploiting the characteristics of the short-text XMC problem, it leverages the label features to construct valid training instances, and uses the label graph for generating the corre- sponding soft-label targets, hence effectively capturing the label-label correlations. While most recent advances (such as SIAMESEXML and ECLARE) in XMC have been algorithmic, mainly aimed towards developing novel deep-learning architec- tures, our data-centric augmentation approach is orthogonal to these methodologies. We demonstrate the generality and effectiveness of Gandalf by showing up to 30% relative improvements for 5 state-of-the-ar algorithms across benchmark datasets consisting of up 1.3 million labels.
INTRODUCTION
Exireme Multilabel Classification (XMC) has found multiple applications in the domains of related searches (Jain et al.) 2019), product recommendation (Medini et al. 2019), dynamic search advertising (Prabhu et al. (2088). ) etc. which require predicting the most relevant results that either frequently co-occur or are highly correlated with the given product instance or search query. In the XMC setting, these problems are often modelled through embedding- based retrieval-cum-rrnktee millions of possible web pages/products/adlpiiee -hrrases considered as labels.
Nature of short-text XMC and Extreme class imbalance Typically, in the tasks of related search prediction, bid-phrase suggestion, and related-product recommendation based on titles, the input data instance is in the form of a short-lexi query. These short-text instances (names or titles), on average, consist of only 3-8 words. In order to effectively model these scenarios, there has been an increasing focus on building encoders as part of deep learning pipelines that can capture the nuances of such short-text inputs (Dahiya et al. 2021b Kharbanda et al. 202T).
The real world datasets in XMC are highly imbalanced towards popular trending ad- phrases/products. Moreover, these datasets adhere to Zipf's s law (Ye et al. 2020), i.c.. most labels in these extremely large output spaces are tail labels, having very few (< 5) instances in training set spanning hundreds of thousands data points Tab T. Appendix). While there is already an insuffi ciency of training data, the short-text nature of training instances makes even more challenging for the models to learn meaningful, non-overfitting encoded representations for tail words and labels.
Frugal architectures and Label features Due to the low latency requirements of XMC applications, most recent works are also focused on building lightweight and frugal architectures that can predict in milliseconds and scale up to millions of labels (Dahiya et al. 2021a). Despite being frugal in terms of number of layers/parameters in the network, these models capable of fitting well enough on the training data, although their generalization to the test samples remains poor (Fig : lt). Hence, creating deeper models for better representation learning is perhaps not optimal under this setting.

---

Under review as a conference paper at ICLR 2023
100 Poores i1sSmnee
A/0 a 500
60
FrsppiionMLLLFF
Misu?
IncestionMMLLL
Inception LacelMix
Maxu'
Labeliix
Gandal
LabelMix
Gandall
Gendalf
PgB
PSP
PSP@1
10
Epeedes
Eppctei
Esaacto
301
Figure 1: Effect of different data augmentations on INCEPTIONXML-LF over LF-Amazon 131K dataset. (a) shows that a significant generalization gap exists between Train and Test P@ 1. However, remarkable improvements can be noted in (b) and (c) as result of using the proposed data augmentation Gandalf. While text mixup (Chen et al. 2020) provides a regularization effect and is effective in reducing overfitting, our proposed alternative LabelMix baseline performs much better.
Recent works, however, make expensive architectural adjustments (Mittal et al. 2021a) lab to leverage the text associated with labels ("label features", discussed in $2) in order to improve generalization.
1.1 RELATED WORK: XMC WITH LABEL FEATURES
Earlier works in XMC primarily focused on problems consisting of entire long-text documents, consisting of hundreds of words/tokens, such as those encountered tagging for Wikipedia (Babbar & Schölkopf] 2017 You et al. 2019). On the output side, the labels were identified by numeric IDs and hence devoid of any semantic meaning. Most works under this setting are aimed towards scaling up transformers as encoders for XMC tasks (Chang et al. 2020 Zhang et al. 2021).
By associating labels with their corresponding texts, which in turn, product titles, document names or bid-phrases themselves, the contemporary application of XMC has gone beyond standard document tagging tasks. With the existence of label features, there exist three correlations that can be exploited for better representation learning: (i) query-label (ii) query-query and (iii) label-label correlations. Recent works have been successful in leveraging label features and pushing of the-art by exploiting the first two correlations. For example, S1AMESEXML. (Dahiya et al. 202 employs a siamese pre-training stage based on a contrastive learning objective between a data point and its label features optimizing negative log- oog-likelhhood loss. GALAXC (Saini et al. 2021) employs a graph convolutional network over a combined query-label bipartite graph. DECAF AF and ECLARE (Mittal et al. 2021al0 b) make architectural additions to exploit higher order query-label correlations by extending the DeepXML pipeline to accommodate extra ASTEC-like encoders (Dahiya et al., 20216).
In contrast to the recent algorithmic developments for short-text XMC with label features, and following the work of (Banko & Brill 2001). which posits higher relevance of developing more training data compared to choice of classifiers in small data regimes, take data-centric approach and focus on developing data augmentation techniques for short-text XMC.
1.2 CONTRIBUTIONS
This work has three-fold contributions:
As our primary contribution, we propose Gandalf GrAph iNduced Data Augmentation based on Label Features - a simple data augmentation algorithm to efficiently leverage label features as valid training instances in XMC. Augmenting training data via Gandalf faciliates the core objective of short-text XMC by enabling the model to effectively capture label-label correlations in the latent space without the need of making architectural modifications,
Empirically, we demonstrate the generality and effectiveness of Gandalf by showing up 30% relative improvements in -ffte-of-the-ee art extreme classifiers across public benchmark datasets. We show that by using Gandalf, methods which inherently do not leverage label features beat strong
2

---

Under review as a conference paper at ICLR 2023
baselines which either employ complicated training procedures (Dahiya al. 2021a) or make heavy architectural modifications (Mittal et al. b) to benefit by leveraging label features.
In order to test Gandalf against a strong data- autmmmtttgtntnio baseline, we propose LabelMix as an effective interpolation-basee
data augmentation baseline, which currently does exist for short-text XMC. In the process of arriving at LabelMix, we also discuss the effectiveness of mixup (Zhang al. 2018) and its variants and aim at answering "Can we extend mixup to feature-label extrapolation to guarantee a robust model behavior far away from the training data?", a question posed in (Zhang et al. 2018) as a future work.
WHAT EXACTLY ARE LABEL FEATURES?
To elaborate label features, take examples relevant to our datasets (i) LLF-WikiTitle-50000 where the model needs to predict the relevant categories, given only the title of a wikipedia page. and (ii) L.-AmazonTiitee
where given product's s name, model needs to recommend related products.
For the wikipedia page "2022 French presidential election", we have the available categories April 2022 events in France 2022 French presidential election 2022 elections in France Presidential elections in France. Further, a google search of the same query, leads to the following related searches- French election 2022 The Economist French presidential election coverage on F FRANCE 24 Presidential Election 2022: A Euroclash Between "Liberal... French polls, trends and election news
Example 1:
newsfor for France POLITICO.ev amongst others.
Example 2: For "Mario Kart: Double Dash!!" on Amazon, we have available Super Smash Bros Melee Super Mario Sunshine Mario Party 7 Super Mario Strikers as the recommended products.
Observations: In view of these examples, one can affirm two important observations: (i) the short- text XMC problem indeed requires recommending similar items which are either highly correlated or co-occur frequently with the queried item, and (ii) the queried item and the corresponding label- features form an "equivalence class" and convey similar intent (Dahiya et al. 2021a). For example, a valid news headline search on search engine should either result in page mentioning the same headline or similar re-phrased headlines from other news media outlets (see Example 1). As a result, it can be argued that data instances are interchangeable with their respective labels' features.
3 GANDALF: DATA AUGMENTATION FOR EXTREME CLASSIFICATION
Notation & Background For training, we have available a multi-label dataset D [(x, y= {f 1 I comprising of N data points. Each i E [AN] is associated with small ground truth label set y; C [L] from L ~ 10" possible labels. Further, x;, E X' denote the textual descriptions of the data point and the label which, this setting, derive from the same vocabulary universe V (Dahiya al. 202 The goal is to learn parameterized function f which maps each instance x; to the vector of its true labels y, E 13 where yı 1 + / E y
common strategy for handling this learning problem. called the two towers approach, is to map instances and labels into common Euclidean space 3 R in which the relevance s (xx) of label I to an instance is scored using an inner product. (xx) = (x)) (1)).. We call ((x) the encoding representation of the instance x, and w, == (I) the decoding representation of label I. If labels are featureless integers, then Vi turns into simple table lookup. In our setting, / is associated with features Z₁, so we identify (1) = V (z.).
The prediction function selects the & highest-scoring labels, f(x) = topp (x)(x), V(-))). Training is usually handled using the one-vs-all paradigm, which applies binary loss function to each entry in the score vector. In practice, performing the sum over all labels for each instance is prohibitively expensive, so the sum is approximated by a shortlist of labels S(x,) that typically contains all the positive labels, and only those negative labels which are expected to be particularly challenging for classification You et al. 2019 Dahiya et al. 2021b Kharbanda et al. (021), leading to
N
N
L V
C(ya. (x))), ((O) ~
Σ
Σ E(ya, (0xx),((1))
(1)
i=1 (ES(
hold symbols y indicate vectors, captial letters Y indicate random variables, and sans-serif y denotes set
3

---

Under review as a conference paper at ICLR 2023
Label Features as Data Points It known that standard training XMC datasets can easily lead to overfitting even with simple classifiers (Guo et al.) 2019), which is confirmed for our setting in Fig: To reduce the generalization gap, regularization needs to be applied to the label decoder V, either explicitly as a new term in the loss function (Guo et al. 2019), or implicitly through the
inductive biases of the network structure (Mittal et al. 2021a b). Exploiting the interchangability of
label and instance text, SIAMESEXML (Dahiya et al. (2221a) proposes to tie encoder and decoder together and require V/ (4) Φ (z)). While indeed yielding improved test performance, this approach has two drawbacks: Firstly, the condition { ((z)) turns out to be too strong, and it has to allow for some fine-tuning corrections m7 yielding (1)
((z:) + 7]: Secondly, training SIAMESEXML becomes multi-staged process. Initially, contrastive loss needs to be minimized, followed by fine-tuning with a classification objective.
Dahiya et al. (202Ta) motivates its approach by postulating a self-annotation property (Label Self Proximity), which claims that a label l relevant to its own textual features with high probability, PLY -
zi] > 1 - for some small < 1. One natural question thus arises, in a label space spanning- ~ 10 labels, what are the other labels which annotate 2, when posed as a data point? Therefore, to effectively augment the training set with z as data point, we need provide values for the other entries of the label vector yı. These labels should be sampled according to Yt P[Y X 21). which means need find sensible approximations to the probabilities for the other labels P 1 X = zi]. When using the cross-entropy loss, sampling can be forgone by replacing the discrete labels y: E 10" by soft labels y P X z/)
Exploiting Label Co-Occurrences In order to derive a model for P[Y, = = = we can take inspiration from the GLAS regularizer (Guo al.] 2011).. This regularizer tries to make the Gram matrix of the label embeddings (Wi, Wy) reproduce the co-occurrence statistics of the labels S,
RELaS [D]] = L-
(WW, wy) - Swy)
(2)
[=1
Here, S denotes the symmetrized conditional probabilities,
Sw 0.5(P[Y) = 1| 1 PPY = 1 Yi 1]).
(3)
Plugging in w, (z;). this regularizer reaches its minimum if
(V(z), 1 (zz)) Sw.
(4)
By the self- proximixx postulate, we can assume (z;) = (2)). For a given augmented instance with target loft-label (z,, y/f ). the training will try to minimize E( ((()(z)) V (z )} ). To be consistent with equation 4 therefore want to choose yis such that Sp arg min l(., yūr f. This is fulfilled for y/r" soff o (Swv) ) for l being the binary cross-entropy, where denotes the logistic function. If l is the squared error, then the solution is even simpler, with you Sur.
For simplicity, and because of good empirical performance, we choose y = Sy even when training with cross-entropy loss. This results in the following, extended version of the self- self-proximity postulate:
Postulate 1 (Solt-Labels for Label Features) Given a label I with features E X, and proxy for semantic similarity of labels S, the labels features, when interpreted as an input instance, should result in predictions
PLV = X = 2i] = Syr-
(5)
Label Correlation Graph The label-similarity measure equation used in the original GLaS regularizer uses only direct co-occurences of labels, which results in a noisy signal that does not capture higher-order label interdependencies. Therefore, we propose to replace it with the label correlation graph (LCG) as constructed in ECLARE. LCG E random walk (with restarts) over a bipartite graph between input data instances and their corresponding ground-truth truth labels. Since entries in LCG are normalized and skewed in favor of tail labels, the LCG can be interpreted as a smoothed and regularized variant of the label co-occurrence matrix. More intuitively, (Mittal et al.) 2021bb show that the LCG correctly identifies set of semantically similar labels that either share tokens with the queried label, or co-occur frequently in the same context (for details, see Fig 4n Appendix A), thus making it good candidate for label-similarity measure.
is inferred by performing
4

---

Under review as a conference paper at ICLR 2023
While ECLARE uses the LCG to efficiently mine higher order query tail-label relations by augmenting the classifier with graph information, we propose to leverage the graph weights (with an additional row-wise normalization get values in range [0, 1J) probabilistic soft labels for z data instance. Further, to restrict the impact of noisy correlations large output spaces (Babbar & Schölkopt 2019), empirically find it beneficial threshold the soft labels obtained from LCG ă = 0.1 (uniformly for all datasets). The algorithmic procedure of the data augmentation via Gandalf is shown below:
Algorithm I: Gandalf Augmentation
label index, label feature natrix def (andalE(j, LCG, delta=0.1):
u[i]
y LCsij, :] LeGij, ii frow-nozmalize LCS to obtain values in [0, 11
y humpy.whezeee della, y, C) Athreshold nolsy correlations
return (x, y!
Capturing Label-label Correlations The models benefit from Gandalf in two ways: (i) from Fig. 3 it is evident that ((z)) does not exist in the vicinity of (xx:), for / E yt. for either head or tail labels. Thus, Gandalf essentially expands the dataset by adding label features as data points which are far from training instances in D and, (ii) as labels are product names or document titles themselves, the new data points created through Gandalf essentially capture the apriori statistical correlations between products/documents that exist in the label space. As result, the encoded representation of correlated labels, learnt by an underlying algorithm, are closer in the representation space. This especially benefits the tail labels which, more often than not, either get missed out during shortlisting or rank outside the desired top-k predictions. As shown in the experimental results (Table 2). the data points generated by Gandalf, indeed, lead to significant improvements for suite of existing algorithms. It may be noted that apart from LCG, other sources of modeling correlations, such as those capturing global and local label correlations combination thereof, are also equally applicable (Huang & Zhou 2012 Zhu et al. 2017).
Gandalf
The Hobbit
Rings
The Lond
ofeee
Colbenbbr
Gander
Libenture
of Drbor
J.ft
Lord
labee
Last Gings
Oeteroo
The
Label- Core--Cornaaion Graph
Unaugmented Datapoint
Labellix
Figure 2: A pictorial representation of the proposed Gandalf and LabelMix strategies formed as per Alg: and Egn. 9 The title of each plot denotes the data point, the y-axis its labels and the x-axis their target values. We demonstrate our augmentations on the data point Of the Rings of Power and the Third Age, which is the final book in the Lord of the Rings(LOTR) series along with labels The Hobbit and The Lord of the Rings. Notably, the labels found through soft targets through the LCG are all related to the LOTR universe, with J. R. R. Tolkien being the author, The Quest of Erehor is central plot line and, Celebrimbor and Gandalf are major characters. Beyond this, the soft targets also cover generic labels like 1954/55 in Literature, which is the correct timeline for book release.
5

---

Under review as a conference paper at ICLR 2023
4 LABELMIX: QUERY-LABEL INTERPOLATION
Since the introduction of mixup for images Zhhng et al. 2018), approaches adapted for textual data (Guo et al. 2020 Chen et al. 2020) have also been proposed. Similar to Verma al. (2019), these approaches propose to mix (interpolate) intermediate representations after layers (X,),, (x()) of the encoder (xx)= (x)) along with the corresponding label vectors as:
de (x,, x3) (xx) (1 (x)) y== + (1- A)y,
(6)
where the mixing parameter À E [0, 1] is sampled from Beta(a: ax). The mixed latent representation de is propagated through the rest of the encoder layers and the loss is calculated using the mixed label vector as (()), V). y). However, we observe that while using such formulation of mixup does reduce overfitting by acting data (refer to Mixup curves in Fig These observations are in line with (Chou et al., 2020), who argue that such formulation of y does not make sense under the imbalanced data regime and hence argue to create the mixed label vector to favor the minority class. In this section, we thus propose a new mixup technique LabelMix as strong data augmentation baseline for XMC, which favors tail labels and is more suitable for highly imbalanced problem as encountered in XMC.
regularizer, it does not improve prediction performance on unseen
Mixup techniques draw inspiration from vicinal risk minimization VRM) (Capelle al. 2000). In VRM, a model is not trained to minimize the risk over the empirical distribution dP y) Di= 8x; (x) dy, (y), but instead over smoothed out version P, which also comprises the vicinity of x. The key task is then to determine what constitutes the vicinity of a data point.
Query-Label Interpolation In recommendation problems, formulated as short-text XMC tasks, works have focused on reducing distance between (xx;) and (z)) A E y; in order to ensure high recall rate during the retrieval step and high efficiency while ranking the relevant labels (Mittal et al.) 2021a Saini et al., 2021). Thus, for the short-text XMC task at hand, we require the model to be invariant under a novel mixup transformation that relates more closely to the aforementioned recommendation objective. Since (2:) is already expected to be in the vicinity of (xx)) and also exhibit such behaviour encoded representations of data point with one of its annotating label features opposed to another data point in standard mixup formulations. We, therefore, propose to use a new definition of vicinity: given a data point (x).y) E D, its vicinity is given by V(x) == [Ø(x, x)) : y:}.
trained classifier Fig 3), the VRM perspective motivates to mix the
Sampling Label for Mixup In imbalanced data regimes, tail labels often have very few data points and thus it makes more sense to sample these labels more often. We thus use an iettance-indepeneee weight vector RR (specifically, label frequency raised to the power 0.5 (Mikolov et al. (203)), the probability of choosing 2 for interpolation from y; is given by y, O (ys, r), where the denominator term ensures summation to unity.
While Dahiya et al. 2222aa) employ a siamese contrastive loss between o(x:) and ((z) s.t. E y; in order bring these closer in the latent space, we posit that an interpolation between these encoded representations in the latent space should result in an invariance i.e. keep the annotating labels unchanged. Intuitively, since the encoded representation of data point is being mixed with that of one of its label's text, this should result in a Label-Affirmiu Invariance. More formally, we propose a novel postulate for query-label interpolation in shared embedding space:
Postulate 2 (Label-Affirming Invariance) Let (x. y) be a training data point in 'D and I E be label relevant to x. Then the classifier should be invariant under mixup with zt in the latent space
top (( V) = top, (o zi)), 4/ = = K K=yy
Modifying Eqn. using postulate 2 for a data point (x. y), we arrive at: th (x, 2:) = (x) + (1 A) (21); y y
(8)
However, we find it empirically beneficial (ref. Tab 3 to also accommodate for the label vector of as proposed in postulate 1. This gives us LubelMix:
tx 2)) Xox (x) + ())x(z)); ý = min(1, y y )

---

but also Under review as a conference paper at ICLR 2023
Figure 3: obtain this plot, we take 50,000 product titles from LF- AmazonTitles- 131K dataset and evalu- ate average cosine similarity between ((x:) and (i) ((zz) where 2 a label feature of of the annotating labels of x,, and (ii) Φ (x4)), where x; and x, are "co-documents" i.c. share a label. Evi- dently, ((x,) ) is already closer to (zi) in the embedding space as compared to & (xx)) and this correlation increases by using the proposed augmentations.
IooptennWiLLLF
Labelt4s
Gandarl
0.54
Neege Ooine Co Smiarty
0.52
0:50
0.48
048
Not only does LabelMix perform much better than standard mixup techniques (ref. Fig serves strong data augmentation baseline for short-text XMC, shown in Tab
EXPERIMENTS & DISCUSSION
We benchmark our experiments on 4 standard public datasets, the details of which are mentioned in Tab 1] To test the generality and effectiveness of our proposed Gandalf, we apply the augmentation across multiple state-of-the-art short-text extreme classifiers: (i) ASTEC, (ii) DECAF, (iii) BCLARE, and (iv) INCEPTIONXML. Additionally, transformer-encoder based XR- Transformer (Zhang et al. 2021 and SiameseXML To compare Gandalf with conventional data augmentation approaches, we test it against LabelMix which serves as strong mixup-based data augmentation baseline more suited for short-text XMC.
Benchmarks, Baselines & Metrics
also compare against
As an algorithmic contribution, we extend the INCEPTIONXML encoder leverage label features in order to further the state-of-the-an on benchmark datasets and call it INCEPTIONXML-LA For this, we augment the OVA classifier with additional label- text embeddings (LTE) and graph-aus graph-augmented label embeddings (GALE) done in (Mittal et al. 2021b), The implementation details and training strategy can be found in Appendix B We measure the models' performance using standard metrics precision @ k, denoted P@k, and its propensity version PSP@k (Jain et al. 2016).
Datasets
N
L
APpL
ALpP
AWpP
AF-azoonniitls 131K
294,805
131,073
5.15
2.29
6.92
LF- Tioleee 693,082
312,330
4.67
2.11
3,01
5O-W 1,813,991
501,070
17.15
4.74
3.10
I.F-AmazonTitles
2,248.619
1,305,265
38.24
22.20
8.74
Table 1: Details of short-text benchmark datasets with label features. APpL stands for avg. points per label, ALpP stands for avg. labels per point and AWpP is the length i.e. avg. words per point.
5.1 MAIN RESULTS
We can make some key observations and develop strong insights not only about the short-text XMC problem with label features but also about specific dataset properties from Table 2] For example, the training on data points generated via Gandalf gives remarkable improvements on top of the base -3llees-11KK 131K and LF- WikiSeeeAlsoiiteee
versions of existing algorithms especially on LF-Amazon Titioonniiitee---i 320K where most labels have ~55 training data points on average. these low data regimes, Gandalf helps capture correlations which are not inherently captured by most existing models. In contrast, improvements on L.F.Wikiiitles-5000 for the models to be inherently able to capture these correlations.
remain relatively mild where there enough data per label
Gandalf With Gandalf, gains of up to 30% can be observed case ASTEC and INCEPTIONXML which, by default, do not leverage label features and yet perform DECAF and ECLARE, and INCEPTIONXML-LE across all datasets. While architectural modifications help capture higher order query-label relations and help model predict unseen labels better, they are computationally expensive, e.g. DECAF (having LTE) takes ~ 2x time to train while ECLARE (having both LTE & GALE) takes 3x compared to its base model ASTEC Candall-augmentte
with their LF-counterparts, i.e.
7

---

Under review as a conference paper at ICLR 2023
Method
P@1
P@3
P@5
PSP@1
PSP@3
PSP@5
P@1
P@3
P@5
PSP@1
PSP@3
PSP@5
LF-Amazon TinlssnTittle-3111 131K
LF-AmazzonTides--LMMM
AftentionXML
32.25
21.70
15.61
23.97
28.60
32.57
45.04
39.71
36.25
15.97
19.90
22.54
XR- UR-TRANSFORMER
38.10
25.57
18.32
28.86
34.85
19.59
44.07
39.98
20.06
27.79
SIAMESEXML++
41.42
30.19
21.21
35.80
40.96
46.19
49.02
42.72
38.52
27.12
30.43
32.52
ASTEC
37.12
25.30
18.24
29.22
64
39.49 .9
48.82
42.62
38.44
21.47
25.41
27.86
+ LabelMix
37.95
25.65
18.59
29.91
35.58
40.63
49.56
44.01
39.57
21.69
26,13
28.05
Gandalf
43.95
29.66
21.39
37.40
43.03
53.02
46.13
41.37
27.32
31.20
DECAF
38.40
25.84
18.65
30.85
36.44
41,42
50.67
44.49
40.35
22.07
26,54
29.30
+ LLbelMix
39.30
26.60
19.23
31.81
37.67
49.63
43.77
39.72
27.61
Gandalf
42.43
28.96
20.90
35,22
42.12
47.61
53.02
46.65
42.25
25.47
30.14
32.83
ECLARE
40.46
27.54
19.63
38.18
39.55
50.14 14
44.09
40.00
27.90
+ LLabelMix
40.34
27.54
19.96
33.44
39.74
45.11
50.55
44,50
40.38
23.40
27.92
30.58
Gandalf
42.51
28.89
20.81
35.72
42.19
47.45
43.00
32.90
INCHPTIONMMI
36.79
24.94
17:95
28.50
34.15
38.79
48.21
42.47
38.59
20.72
24.94
27.52
LabelMix
27.45
19.82
32.12
43.08
39.21
23.67
25.73
Gandalf
44.67
30.00
21.50
37.98
43.83
48.93
50,80
44.54
40.25
25.49
29.42
31,59
INCEPTIONMMI.LF
40.74
27.24
19:57
34.52
39.40
49.01
42.97
24.56
+ LabelMix
41,90
28.20
20.35
35.60
41.07
46.20
49.84
43.71
40.42
26.31
30.14
32.46
Gandalf
43.84
29.59
21.30
38.22
43,90
47.23
42.84
30,02
33.18
LF-
AttentioNMLL
17.56
11.34
8.52
9.45
10.63
11.73
40.90
21.55
15.05
14.80
13.97
13.88
+
31.97
21.43
16.24
26.82
28.42
30.36
42.08
22.80
16.01
23.53
21.64
21.41
ASTEC
22.72
15.12
11.43
13.69
15.81
17.50
44.40
24.69
17.49
18.31
18.25
18.56
LabelMix
22.91
15.79
12.02
13.99
16.57
18.04
44.63
24.91
18.35
19.21
19.53
19.32
Gandalf
31.10
21.54
16.53
23.60
26.48
28.80
45.24
25.45
18.57
21.72
20.99
21.16
DECAF
25.14
16.90
12.86
16.73
18.99
21.01
44.21
24.64
17.36
19:29
19.82
19.96
+ LabelMix
26.55
18.04
13:75
17.86
20.46
22.61
44.22
24.47
17.3
21.37
20.72
20.69
Gandalf
31.10
21.60
16.31
24.83
27.18
29.29
45.27
25.09
17.67
22.51
21.63
21.43
ECLARR
29.35
19.83
15.05
22.01
24.23
26.27
44.36
24.29
16.91
21.58
20.39
19.84
LLbbeliix
29.42
19.94
15.17
22.05
24.36
26.46
44.41
24.49
17.13
21.21
20.34
19.9
Gandulf
31.33
21.40
16.31
24.83
27.18
29.29
45.12
24.45
17.05
24.22
21.41
20.55
INCEPTIONXML.
23.10
15.54
11.52
14.15
16.71
17.39
44.61
24.79
19.52
18:65
18.70
18.94
+ LabelMix
25.16
17.03
12.97
16.11
18.72
20.76
44.85
24.91
19.73
19.37
18.98
19.56
Gandalf
32.54
22.15
16.86
25.27
27.76
30,03
45.93
25.81
20.36
21.89
21.54
22.56
INCEPTIONXMI-LL
28.99
19.53
14.79
21.45
23.65
25.65
44.89
25.71
18.23
23.88
22.58
22.50
LabelMix
29.68
20.16
15.32
22.24
24.69
26.80
45.64
26.35
18.78
24.09
22.98
23.00
Gandalf
33.12
22.70
17.29
26.68
29.03
31.27
47.13
26.87
19.03
24.12
23.92
23.82
Table 2: Results showing the effectiveness and generality of Gandalf on state-of-the-art extreme classifiers.
base encoders, on the other hand, do not need to make any architectural modifications or employ complicated training pipelines to imbue necessary invariances.
LabelMix While being effective capturing query tail- laiellabel correlations. LabelMix can only imbue limited additional inductive bias into the model. ECLARE, on the other hand, is able to better capture these higher order correlations through its label graph-augmented classifier (GALE), and thus only gains trivially from LabelMix. DECAF gains non-trivially on both LF-Amazon Titles-13 131K and WewiiiseeAlloTilee--
datasets as it only encodes label text embeddings (LTE) in its classifier, which leaves out the scope to capture query tail-label correlations further. Similarly, INCEPTIONXMI stands to gain significantly more from LabelMix compared to its LF-counterpart which also employs GALE. Notably, LabelMix works much better on INCEPTIONXM((LE than ECLARE because of their dynamic negative mining, which enables the augmentation to work more effectively.
Gandalf vs GALE ECLARE leverages LCG to encode label-label correlations in w; through GALE which helps the model improve prediction performance on new unseen labels. However, this only allows the classifier to distribute the loss gradient from training instance {x,, Yz} across y; and correlated labels as per LCG. This essentially captures higher order query-label correlations while not exploiting label-labels correlations in a way Gandalf does. Since the correlations learnt from GALE and Gandalf are independent of each other, we find ECLARE and INCEPTIONXML-LE. both of which employ GALE, to benefit off training on data points generated using Gandalf.
5.2 ABLATION STUDY
We try using Gandalf and LabelMix without labt- (SL) from LCG in Table 3, where Gandalf w/o SL. is essentially equivalent using label features as data points with self-annotation property
8

---

Under review as a conference paper at ICLR 2023
alone. However, that only helps the model learn tabel-to-woord associations, like LTE in DECAF. Notably, soft-targets play an important role in enabling the encoder to intrinsically learn the label-label correlations Table 3 and imbue the necessary inductive bias in the models. For further analysis, provide visualizations depicting differences in prediction performances obtained with and without by our proposed augmentations in Appendix B Table 5).
Method
P@l
P85
PSP@1
PSP@3
PSP
P@8
P85
PSP@1
PSP
PSP@5
UF-AmazoonTills-13
WWWkdeeeAtooottess 320K
InceptionMMI.
35.62
24.13
17.35
27.53
33.06
37,50
21.53
14.19
10.66
13.06
16.33
+Snoonym Replacement
35.07
23.71
17.08
27.20
32.41
36.77
20.08
13.13
9.92
12,00
13,50
14.90
LabelMix wdo SL.
37.25
17.98
34.58
22.61
14.98
14.02
17.55
LabelMix
39.05
26.52
19.15
30.98
37.20
42.26
23.90
16.10
12.28
15.20
17.60
19.56
Gondolf who
37.59
25,25
18.18
30.75
35.54
40.06
24.43
16.16
12.15
16.89
18.45
20.02
+ Gandalf
43.52
29.23
20.92
36.96
42.71
47.64
31.31
21.38
16.22
24.31
26.79
28.83
Table 3: Results demonstrating the effectiveness of using Gandalf soft-labels (denoted SL. and synonyms replacement on single InceptionXML model.
6 OTHER RELATED WORK: DATA AUGMENTATION AND XMC
Architectural design choices are often complemented with data augmentation methodologies which have been found to be successful in imbuing necessary problem-specific invariances in the model, thereby improving model's generalization capability on unseen data. Textual augmentations in the discrete space such as making spelling errors (Xie et al. 2017). WordNet-based (Miller et al. 1990) replacement with synonyms (Kolomiyets et al. 2011 Li al.) 2017: Wang et al.) 2018)) text-fragment
switch (Andreas) 2020), random insertion, swap and deletion as proposed in versions of EDA (Wei
& Zou 2019 Karimi et al., [02I] have been shown to bring some performance improvements
(Coulombe (2018). However, such transformations can lead to semantic inconsistency and illegibility
and, thus decrease performance for classification tasks (Qiu et al. 2020 Anaby- et al. 2020).
More recent methods have tried filling these gaps in semantic consistency; (Zhao et al. 2022) improve upon EDA by converting the requirements of diversity and semantic consistency as a min- max optimization problem. Many methods leverage language models to suggest context-specific replacements for masked tokens either discretely via a single synonym (Kobayashi 2018 Wu et al. 2019) or weighted sum of word embeddings of semantically similar words (Gao al. 2019).
Even though the above approaches help mitigate semantic inconsistency to some extent, they are not able to preserve the annotating label, especially in low data regimes (Hu et al. 2019) where major chunk of XMC data lies. These issues of semantic inconsistency and label distortion can be more explicit, particularly for short-text instances in XMC i.e. document titles product names, where each word in the query has high correlation with the labels. Deletion or insertion of a word in the query could completely alter the search either a more generalized or narrowed down one, or result in something with little sense. For example, changing the search query from "Beats Wireless headphones" "Beats Wireless headphones with microphone" would lead to filtered result. Furthermore, similar to label-altering random crops in images (which can be considered as the visual equivalent of word deletion) as pointed out by (Balestriero et al. 2022), altering the aforementioned query to remove or replace "Beats" with synonym might lead to result not having the intended brand in top 10 hits.
7 CONCLUSION
In this paper, we proposed Gandalf, a data augmentation strategy, which is particularly suited for short-text extreme classification. not only eliminates the need for complicated training procedures in order to imbue inductive biases, but dramatic increase in prediction performance of state-of-the-a methods in this domain. Additionally, we also developed LabelMix. as a baseline data augmentation which is motivated from previous interpolation based textual mixup techniques. is expected that treatment towards studying invariances in this domain will spur further data-centric research designing other data augmentation methods to effectively replace architectural additions in order to leverage label features and achieve faster inference times.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlo- mov, Naama Tepper, and Naama Zwerdling. Do not have enough data? deep learning to the rescue! Proceedings of the AAAI Conference on Artificial Intelligence, 3400 Apr. 2020. doi: http://aaaa.i3.....2222..3
article/view/622 6233
Jacob Andreas. Good-enough compositional data augmentation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7556-7566, Online, July 2020. Association for Computational Linguistics. doi: (01553//112022.al-mmmae
URL https
org. 2020. aaw-mmainn.............aaaaaaaae 676
R. Babbar and B. Schölkopf. DiSMEC: Distributed Sparse Machines for Extreme Multi-label Classification. WSDM, 2017.
R. Babbar and B. Schölkopf. Data scarcity, robustness and extreme multi-label classification. Machine Learning, 108: 1329-1351, 2019.
Randall Balestricro, Leon Bottou, and Yann LeCun, The effects of regularization and data augmenta- tion class dependent. arXiv preprint arXiv:2204.036332 036322, 2022.
Michele Banko and Eric Brill. Scaling to very very large corpora for natural language disambiguation. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics, pp. 26-33, Toulouse, France, July 2001. Association for Computational Linguistics. doi: 10.3115/ 1073012. 1073017. URL https: / ac anthonatoooooo logy. 0rggP01-1100
W-C. Chang, H.-F Yu, K. Zhong, Y. Yang, and I. Dhillon. Taming Pretrained Transformers for Extreme Multi-label Text Classification. In KDD, 2020.
Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. Vicinal risk minimization. In T. Leen, Dietterich, and V. Tresp (eds.), Advances in Neural Information Processing Systems, volume 13. MIT Press, 2000. httphhhtttts:::/oceeeeee ngs neur i pss cctpppppr//000000
Jiaao Chen, Zichao Yang, and Diyi Yang. MixText: Linguistically-inforned interpolation of hid- den space for cuni-supervieed text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2147-2157, Online, July 2020. Asso- ciation for Computational Linguistics. doi: 10. 18.18653/v a2l.0/0acm.mmmii.11. 194. URL https
1/aclanthologe Sagachooooy..or/222000 acl-mmiig innin... 194
Hsin-Ping Chou, Shih-Chieh Chang, Jia-Yu Pan, Wei Wei, and Da-Cheng Juan. Remix: Rebalanced mixup. In Computer Vision - ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Part VI, pp. 95-110, Berlin, Heidelberg. 2020. Springer- Verlag. ISBN 978-3- 030-65413-9, doi: 10. 00.1007/978-3-030-654444 978 1300-5414-6--
URL httph:tts//// //tpi.//iiio///oo org/10. http:///oo.oo/....////o 1007/
Claude Coulombe. Text data augmentation made simple by leveraging nlp cloud apis. arxiv. Retrieved June, 15:2020, 2018.
K. Dahiya, A. Agarwal, D. Saini, K. Gururaj. Jiao. A. Singh. S. Agarwal, P. Kar, and M. Varma. Siamesexml: Siamese networks meet extreme classifiers with 100m labels. In ICML, July 202
K. Dahiya, D. Saini, A. Mittal, A. Shaw, K. Dave, A. Soni, H. Jain, S. Agarwal, and M. Varma. DecpXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents. In WSDM, March 2021b.
Fei Gao, Jinhua Zhu, Lijun Wu, Yingee Xia, Tao Qin, Xueqi Cheng, Wengang Zhou, and Tie- Yan Liu. Soft contextual data augmentation for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5539-5544, 2019.
C. Guo, A. Mousavi, X. Wu, Daniel N. Holtmann- Rice, S. Kale, S. Reddi, and S. Kumar. Breaking the Glass Ceiling for Embedding- -ased Classifiers for Large Output Spaces. In NewrIPS, 2019.
10

---

Under review as a conference paper at ICLR 2023
Demi Guo, Yoon Kim, and Alexander Rush. Sequence-level mixed sample data augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5547-5552, Online, November 2020. Association for Computational Linguis- orrg/020
tics. doi: 10. 2tt55///122200..mmmmmmiiii
URL
p-m ma 447
Zhiting Hu, Bowen Tan, Ruslan Salakhutdinov, Tom Mitchell, and Eric P. Xing. Learning Data Manipulation for Augmentation and Weighting. Curran Associates Inc., 2019.
Sheng-Jun Huang and Zui-Hua Zhou. Multi-label learning by exploiting label correlations locally. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pp. 949-955, 2012.
Himanshu Jain, Yashoteja Prabhu, and Manik Varma. Extreme multi-label loss functions for recom- mendation, tagging. ranking & other missing label applications. In KDD, pp. 935-944, 2016.
Himanshu Jain, Venkatesh Balasubramanian. Bhanu Chunduri, and Manik Varma. Slice: Scalable linear extreme classifiers trained on 100 million labels for related searches. Proceedings of the Twelfth ACM International Conference Web Search and Data Mining, 2019.
Akbar Karimi, Leonardo Rossi, and Andrea Prati. AEDA: An easier data augmentation technique for text classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2748-2754, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653w.12022..nnddl findinns-emnllp org/2021. findingss-nnnnlppp.. .234
URL https://aclattholooyy
Siddhant Kharbanda, Atmadeep Banerjee, Akash Palrecha, Devaansh Gupta, and Rohit Babbar. Inceptionxml A lightweight framework with synchronized negative sampling for short text extreme classification. arXiv preprint arXiv:2109.073199 2021.
Sosuke Kobayashi. Contextual augmentation: Data augmentation by words with paradigmatic relations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 452-457, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18853/v1/N18- 2072.. URL https:: ac anthologyhyoyy.oooooo www.tthhoooooo.o.///g...... 2072
Oleksandr Kolomiyets, Steven Bethard, and Maric-Francine Moens. Model-portability experiments for textual temporal analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 271 276, Portland, Oregon, USA, June 2011 1, Association for Computational Linguistics. URLhtt http:p/s///caaaatoolooooooooo P11 211-2047 2047
Yitong Li, Trevor Cohn, and Timothy Baldwin. Robust training under linguistic adversity. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 21-27, 2017.
Tharun Kumar Reddy Medini, Qixuan Huang, Yiqiu Wang, Vijai Mohan, and Anshumali Shrivastava. Extreme classification in log memory using count-min sketch: case study of amazon search with 50m products. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Curran As- sociates, Inc., 2019, URL BtRL/ttttss///roceeetli wwwpediiiinnnnnnnnrneeeeeeeeeeee neurips httpe//ppppppccappppp/////////// .pdf
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen- tations of words and phrases and their compositionality. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (eds.), Advances Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL htt.htttsss//prreeeiiing neurips cc/
George A. Miller, Richard Beckwith, Christiane D. Fellbaum, Derek Gross, and Katherine J. Miller. Introduction to wordnet: An on-line lexical database. International Journal of Lexicography, 235-244, 1990.
11

---

Under review as a conference paper at ICLR 2023
A. Mittal, K. Dahiya, S. Agrawal, D. Saini, S. Agarwal, P. Kar, and M. Varma. Decaf: Deep extreme classification with label features. In WSDM, March 2021a.
A. Mittal, N. Sachdeva, S. Agrawal, S. Agarwal, P. Kar, and M. Varma. Eclare: Extreme classifi- cation with label graph correlations. In Proceedings of The ACM International World Wide Web Conference, April 2021b.
Jeffrey Pennington, R. Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014.
Yashoteja Prabhu. Anil Kag, Shrutendra Harsola, Rahul Agrawal, and Manik Varma. Parabel: Partitioned label trees for extreme classification with application to dynamic search advertising. In Proceedings of the 2018 World Wide Web Conference, WWW '18, pp. 993-1002, Republic and Canton of Geneva, CHE, 2018. International World Wide Web Conferences Steering Committee. ISBN 978 4503563999 doi: 10.114453178886.11358 185998. / bottps://doi.orgo 3178876. 1 3185998
1145/
Siyuan Qiu, Binxia Xu, Jie Zhang, Yafang Wang, Xiaoyu Shen, Gerard de Melo, Chong Long, and Xiaolong Li. Easyaug: An automatic textual data augmentation platform for classification tasks. Companion Proceedings of the Web Conference 2020, 2020.
D. Saini, A. K. Jain, K. Dave, J. Jiao, A. Singh, R. Zhang, and M. Varma. Galaxe: Graph neural net- works with labelwise attention for extreme classification. In Proceedings of The ACM International World Wide Web Conference, April 2021.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz. and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning. volume 97 of Proceedings of Machine Learning Research, pp. 6438-644444 PMLR, 09-15 Jun 2019. URL /tt/:tpps///prreeedcss.... wttp:eewdddinng....e.e..... wwt..mmw...p..ssseeeemmmm
Verma 1 9a. html
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. SwitchOut: an efficient data augmentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 856-861, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 10.18653/11/D1--11 7/aclanthologg wwt./orggg/iii.
URL https
Jason Wei and Kai Zou. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNLP), pp. 6382-6388, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D11-11777
orq/D19-1670
Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. Conditional bert contextual augmentation. In International Conference on Computational Science, pp. 84-95. Springer, 2019.
Ziang Xie, Sida I Wang, Jiwei Li, Daniel Lévy, Aiming Nie, Dan Jurafsky, and Andrew Ng. Data noising as smoothing in neural network language models. arXiv preprint arXiv: 1703. 02573, 2017.
Brandon Yang, Gabriel Bender, Quoe V Le, and Jiquan Ngiam. Condconv: Conditionally pa- rameterized convolutions for efficient inference. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bar Buc, Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpps://rroceeeee Pwwp:/4466-ppeppppepp
neuripppp ps. cc/
H. Ye, Z. Chen, D. Wang, and Davison B. D. Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification. In ICML, 2020.
R. You, Z. Zhang, Z. Wang, S. Dai, H. Mamitsuka, and S. Zhu. Attentionxml: Label based attention-and -anttion-awwreeeee deep model for perfomaeeee extreme multi-label text classification. In NeurIPS, 2019.
12

---

Under review as a conference paper at ICLR 2023
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018. URL https
/openneview.net// oot.::/oooommmmddr1o00d--------
Jiong Zhang, Wei-cheng Chang, Hsiang-fu Yu, and Inderjit Dhillon. Fast multi-resolution transformer fine-tuning for extreme multi-label text classification. Advances in Neural Information Processing Systems, 34, 2021.
Minyi Zhao, Lu Zhang, Yi Xu, Jiandong Ding, Jihong Guan, and Shuigeng Zhou. EPiDA: An easy plug-in data augmentation framework for high performance text classification. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4742 4752, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v hatp:/22a22l2mmmiiiaaa m49.. URL https:
wgw.ooog...oo....oo............. wry.orrg/200 2022. naacl n. 349
Yue Zhu, James T Kwok, and Zhi-Hua Zhou. Multi-label learning with global and local label correlation. IEEE Transactions on Knowledge and Data 108 2017.
13

---

Under review as a conference paper at ICLR 2023
A VISUALIZATIONS
The highly sparse nature of the XMC problem makes the LCG noisy. In order to reduce this noise from our soft targets, we threshold the correlation values 6, and quantify its effect by varying the parameter, as shown in Table 4 Additional visualizations capturing the label correlations and their first order-neighbors are shown in Figure To better denote the impact Candalf on tail label prediction, we perform quantile analysis by distributing the labels into aqui-voluminous bins based on the label frequency in the training data, as shown Figure Finally, the qualitative comparison of correctness of outputs generated by the baseline model, and those as result of the proposed augmentations shown in Table
Membrann fouling
Proceats enginebring
Quantom field meccon
Loen quantum gravity
Conralinate free
Lattic gauge mutory
Quantum gra pravity
EU wrnter policy
String cheenn
Regge Calculus
Treatment pand
of wisste-watar treatment technologies
Rica
Asquatic plant
Water supply and sanitation in the United States
Drinking water
Asymptotic freedom
Vector clock
Lagrangian system
Canonical qua quantiration
Einstein symchronissatin
Coupling constant
jam sync
History playiics
Time signal
Network Time Protocol
Fine structure constant
Antiaythera mech mech
Timerode
Binery cinery-ccded decimal
Precision Time Protocol
Gluan Neld
Wistory computer science
Lirt monennt commmnu science
Theory relativity
Hissory Society
History computing hardware
Causal dynamical trinnsdaation
Chartes Batibaye Insttture
Computer museums
mnneine
of
Winer Purifoall on
Clock By nchionication
History Computing
Theory
Dyramical Triangulation
Figure 4: Correlations between labels and their first-order neighbours, as found by the LCG on the LF- WikiWiiitee-50000 dataset. The legend shows the label in question, the bar chart shows the degree of correlation with its neighbouring labels. Correlated labels often share tokens with each other and/or may be used in the same context.
P@
P@3
P@5
PSP@1 I
PSP@3
PSP@5
P&
P@3
P@5
PSP@1
PSP@3
PSP@5
AF-AAmzooTTitls-1133
L
0.0
41.71
28.03
20.14
36.94
41,93
46.64
31.40
21.56
16.53
26.01
27.89
29.99
42.09
28.38
20.45
37.09
42.19
47.04
32.20
21.86
16.60
26.06
28.01
30.03
0.2
41.73
28.10
20.18
37.01
41.99
46.67
31.29
21.35
16.28
25.68
27.59
29.65
0.3
41.39
27.74
19.89
36.71
41.51
46.09
31.03
20.92
15.99
25.11
27.12
29.14
Table 4: Results demonstrating the sensitivity of Gandalf with respect to 8, as defined in Algorithm All experiments were performed on the InceptionXMLL-LF model, augmented with Gandalf. As shown, the empirical performance peaks at 5 value of 0.1 which is sufficient to suppresses the impact noisy correlations.
14

---

Under review as a conference paper at ICLR 2023
InceptionXML-L
20.0
Astec
DECAF
17.5
ECLARE
Gandalf
15.0
12.5
P@S
10.0
7.5
Complete (#131K) (5. 15)
5
3
2
(#68K) (1.96)
(#28K)
(#18K)
(#11K)
(#4K) (31.32)
(4.72)
(7.38)
(12.11)
Bins with Increasing Label Frequency
(a) Contributions P@S in AF-AmazonTitle 131K
InceptionXML-LF
16
Astec
DECAF
14
ECLARE
Gandalf
12
10
P@5
Complete (#312K) (4. 67)
(#195K) (1.49)
(#74K)
(#31K)
(#9K) (30.98)
(#1K) (200.78)
(3.94)
(9.14)
Bins with Increasing Label Frequency
(b) Contributions P@5 in JFWWikiSeeAlloTTieee330000
Figure 5: Analysis demonstrating the effectiveness of Gandalf in improving performance over tail labels. For this graph, labels were divided into equi-voluminous bins in increasing order of frequency. The graph shows contribution of each bin to P@5 on different datasets and short-text extreme classifiers.
15

---

Under review as a conference paper at ICLR 2023
Method
Datupoint
Baseline Predictions
LabelMix Predictions
Gandalf Predictions
Pontryagin duality, Topological
Topolopical oriler,
Algebraic
Compact group, Haur messure,
INCRIIONMI-LLF
der, Topalogical quantum fied the- ory, Topological quanturn number,
group. Topological ring. Topo-
group. Algehraic group, Topalogi-
logical quantum
Bold
theory,
cal ring
Quuntain topology
Topological quantum number
Topological quantum computer,
Topological order.
Algebraic
Compact group, Haur messure,
Topological onder. Topological
group. Topological ring. Topu-
group. Algebraic group, Topologi- cal ring
DECAF
quantum (ield theory, Topological
logical quaniumn field
theory
Topological group
quantuum number, Quantim topol-
Topolingical quantum number
ogy
Topolopical quantum compater,
Topological order, Topological
Compact proup, Topological order,
Topological order, Topological
ring. Topological quantum field
Lie group, Algebraic group, Topo- logical ring
ECLARE
quaatum held theory, Topological
theary, Topological quantum num-
quantum number, Quantum topol- ogy
ber, Quantum topulogy
List of lighthouses in Soviland, Northem Lighthouse Board lighthouse,, Outcake, Communes of the Finistere department, Com- munes of the Cotes-d Armor de-
Outcake, Out milk, Rolled cats,
Oatcakke, Outmeal, Out milk, Por- ridge, Rolled cats
List of discases, Goboat
INCEPTIONMMLLLE
Oascake, Oarmeal, Design for All (in ICT,, Oalley Point Reserve, Outley Pleasure Grounds
Oatcake, Outmeal, Oat milk,
Oatcake, Oatmeal, Out milk, Por. ridge, Rolled oats
DECAF
ley Point Reserve, Ondey Pleasure
Grounds
Ostmeal, Out milk, Purls in Syd-
Oatmeal, Rolled cats. McCann's
Outcake, Porridge, Rolled
ECLARE
mey, Outley Point Reserve, Oatley Pleusure Grounds
Point Reserve Onlley Plensure
Cul Irish Oatmeol, Oniley
Ouiley Resetve, Ouiley sare Grounds
Grounds
Colorado metropolitta press, Pront
Colorado metropolitin areas.
Coloraoo metropolitaan
Range Urhan Comdor, Outline of Colorado, Index of Colerado related articles, State Colorado
Outline of Colorad, Index of Colorado related articles, State of Colondo, Colorado countes.
Outline of Colorado, Index of
IXCEPTONXML-LF
Colorado-related articles, Col- orado cities and towns, Colorado counties
Colorado metropolitum urtas, Frunt Ronge Urbon Coridoe, Stute of Colorado, Colorado municipali- lies, National Rogister of Histooii Places listings in Grand County, Colondo
Frout Ronge Urbun Currider, In- dex of Colorako-reltted artickes National Registor of Historic Places listings Grand County. Coloradks, Grand County, Cal- orados, Last takes Colordo State of Colorado, Colorado cities and Colorado counties, Na- tional Register of Historic Places listings in Grand Councy, Colorado, Grand County. Colorado
Outline of Colorado, State Cul- oradit, Colondo cities and towns, Colorado municipalities Colorado wumtiies
Grand Lake, Colondo
DECAF
Colorado metropolitaan urcos, State of Colondo, Calorddo cities and towns, Colorado counties, orado census de designated places
Outline of Colorado, Index of Colorado-related artic State Colorad,, Colorado towns, Colorado counties
ECLARE
Lunar Orbiter Image Recovery Project, uime Oibiier 3, Lanar Dr-
Exploration of the Muon, List of
Surveyor prugram, Luna pro-
inl missinss to the Monn, Lunar Or- biber Image Recovery Project, Lu-
gramine Lunar Orbiter Image Ro- covery Project, Lunar Orbiser
INCEPTIONML-LF
biter S, Chinese Lanar Exploration Program, List Future lorar stons
3, Orbiter
Lunar Osbiter
Explocation the Moon, List of
Exploration of the Moon. Lunar Orbicer program, Lunaz Otbiter Im-- age Recovery Project, Lunar Or- biter Lunar Orbiter
Expleration of the Moon, Apollo program, Surveyor program, Luna programme. Lanar Orhiter gram
Lunar Orbiter program
man made objects the Moun,
DECAP
Lunar Orbiter Image Recovery Project, Lunar Orbiter Lanar On- biter 5
Explocation of the Moon, Lanar
Exploration of the Moon, Lanar Otbiler program, Lunze Orbiler Inn- age Recovery Project, hiter Lamar Orhiter
Exploration of the Moon, Pioneer
ECLARE
Octiler program, Lunar Otbiler Imt- age Recovery Project, Lamir Or-
program, Surveyot program, Luna progrumm,
Or-
hiter Laman Othiter
prim
Royal Soudi Air Defense, Royal
Sandi led intervention in Bahrain,
Military of Saudi Arabia, Royal Saudi Air Force, Royal Air Defense. Royal Sandi Strategic Missile Fonce, King Khalid Mili- tary City
Saudi Strategic Missile Force,
Royal Saudi Navy, Royal Saudi
INCEPTINNML-LL LP
Saudi Royal Goard Regimentt Ter-
Air Defense, Royal Saodi Strategic Missile Force, Saudii Royal Guard Regiment
rorism in Saudi Arabia, Capital
punishment in Saudi Arabia
Saudi Aralion-- led intervention
Sandi-Jed intervention Balrain, Saudi Arahin, Military Sandi
Royal Saudi Air Forvs, Royal Smadi Navy, Royal Saudi De- fense, Royal Saudi Strategic Mis- sile Fonce, Saudi Arabiun National Guand
Yemen,, Saudi-lod intervention in Rahrain, Human rights in Sandi Arabis, Legal system Saudi Art- bin, Joint Chiee's of Staff (Sandi
DECAF
Armed Forces of Saudi Arabia
Arbhia, Royal Saudi Strategic Mis- sile Foree, Saudi Arabian National Guard
Arahiaa
List of unteed groups the Syrian
Military of Saudi Arabia, Royal Soudi Air Defense, King Khalid Military City, Saudi Royal Guard Regimeni, List rulers of Saudi Arabia
Military of Saudi Arabia, Royal Soudi Air Detense, Royal Saudi Strategic Missile Force, King Khalid Miliaary City, Royal Guard Regimentt
Civil War, Military of Smuli Ani- bia, Royal Saudi Strategic Missile Force. King Khalid Milisayy City.
ECLARE
Joint Clicils of Staff (Sandi Ama- his)
Table 5: Prediction examples of different datapoints from the L.F-WikiSeeAlsoTiiiie Tittllsss dataset. Labels indicate mispredictions. It may be noted that queries with even just a single word, like "Oat", which predicts unrelated labels in the case of the baseline prediction, gets all the labels right with the addition of Gandalf. Furthermore, even mispredictions get closer when our data augmentation strategy is introduced.
16

---

Under review as a conference paper at ICLR 2023
B INCEPTIONXML--F
Wr
Gonv 2010
Cheurir
Shortia:
Paperrs
Dewnooppk
Midil
Purp
And
ST1PP
ConNAGE
Burtes
Input Tokeers and Embeddingo
Embedding Emnrrcomnt Modulo
Label teat
Label-eet Enbodding
Embooding
Figure 6: INCEPTIONXML-LF. The improved Inception Module along with instance attention is shown detail. Changes to the INCEPTIONXML framework using the ECLARE classifier are also shown.
Model Outlook: Short-text queries encoded by modified InceptionXML encoder, which encodes an input query x; using an encoder 4 = (E, 0) parameterised by E and 6, where E denotes a D-dimensional embedding layer of V/ //x for vocabulary tokens V = [t1 ......, tv] and l denotes the parameters of the embedding enhancement and the inception module respectively. Alongside Φg. INCEPTIONX MI.-L. learns two frugal ASTEC-llke Dahiya et al. (2021b) encoders, each label-text encoder T; FE R) and graph augmented encoder 9 E. R.). Here, R denotes the parameters of fully connected layer bounded by a spectral norm and the embedding layer É is shared between all y Φ and 4 for joint query-label word embedding learning. Further, an attention module A, meta meta-classifier Wm and an extreme classifier W. are also learnt together with the encoders. Next, we specify the details of all components of INCEPTIONXML-LE.
B. INSTANCE- -ATTENINATTENTIOO QUERY ENCODER
We make two improvements to the inception module INCEPTIONXML. for better efficiency. Firstly. the inception module, the activation maps from the first convolution layer are concatenated before passing them onto the second convolution layer. To make this more computationally efficient, we replace this "inception- like" setting with a "mixture of expert" setting Yang et al. (2019). Specifically, route function is added that produces dynamic weights for each instance to perform dynamic element-wing wise weighted sum of activation maps of each filter. Along with the three convolutional experts, also add an average pool down sampling residual connection ensure better gradient flow across the encoder.
Second, decouple the second convolution layer have each for the meta and extreme classification tasks.
B.2 DYNAMIC HARD NEGATIVE MINING
Training one-vs-all (OvA) label classifiers becomes infeasible in the XMC setting where we have hundreds of thousands or even millions of labels. To mitigate this problem, the final prediction or loss calculation is done on a shortlist of size VL comprising of only hard-negatives label. This mechanism helps reduce complexity of XMC from an intractable O(NDL)
computationally feasible ( DvL) problem. INCEPTIONXML-LF inherits the synchronized hard negative mining framework used in the INCEPTIONXML. Specifically, the encoded meta representation is passed through the meta-classifier which predicts the top-K relevant label clusters per input query. All labels present in the top-K shortlisted label clusters then form the hard negative label shortlist for the extreme task. This allows for progressively harder labels to get shortlisted per short-text query as the training proceeds and the encoder learns better representations.
17

---

Under review as a conference paper at ICLR 2023
B.3 LABEL-TEXT AND LCG AUGMENTED CLASSIFIERS
INCEPTIONXML-LFF extreme classifier weight vectors W. comprise of weights, as in Mittal al. (02 Specifically, the weight vectors result of an attention-based sum of (i) label-text embeddings, created through P, (ii) graph augmented label embeddings, created through graph encoder dg and, (iii) randomly initialized per-label independent weights Wi.
first obtain label-text embeddings as z E zf. where zi are the TF-IDF weights of label feature corresponding to label I. Next, we use the label correlation graph G to create the graph-weighted label-text embeddings zi Eme[r)] Gin = capture higher order query-tail label correlations. z/ and z are then passed into the frugal encoders d and &y respectively. These encoders comprise only of a residual connection across a fully connected layer as a- R (R))(2) + B &, where ž = { [ zi J. G represents GELU activation and a and 8 are learned weights. Finally, the per-label weight vectors for the extreme task obtained
As shown in Fig. 6
We,1 A(z), zf. wi) a² 2 + z a
where A is the attention block and a112,33) are the dynamic attention weights produced by the attention block.
B.4 TWO-PHASED TRAINING
We find there to be a mismatch the training objectives in DeepXML--based ap- proaches like ASTEC, DECAF and ECLARE which first train their word embeddings meta-labels in Phase I and then transfer these learnt embeddings for classification over extreme fine-grained labels in Phase III Dahiya et al. (2021b). Thus, in our two-phased training for INCEPTIONXML-LI we keep our training objective same for both phases, Note that, in INCEPTIONXML- are always learnt on labels instead of meta-labels or label clusters and we only augment our extreme classifier weight vectors W. with label-text embeddings and LCG weighted label embeddings. We keep the meta-classifier Wm as a standard randomly initialized classification layer.
Motivation:
the word embeddings
Phase I: In the first phase, we initialize the embedding layer E with pre-trained GloVe embeddings Pennington et al. (2014), the residual layer R in o and Φ2 is initialized to identity and the rest the model comprising of 2g, g. Wm. and A is randomly initialized. The model is then trained end-to-ent but without using free weight vectors w; in the extreme classifier W.. This set up implies that W. only consists of weights tied E through 4 and $ which allows for efficient joint learning of query-label word embeddings Mittal et al. (2021a) in the absence of free weight vectors. Model training in this phase follows the INCEPTIONXME without detaching any gradients to the extreme classifier for the first few epochs. In this phase, the final per-label score is given by:
pipeline as described KKarbanda et al. (202I)
Pt = ((), D (z(2))) P,((x)
Phase II: In this phase, we first refine our clusters based on the jointly learnt word embeddings. Specifically, we recluster the labels using the dense z, representations instead using their sparse PIFA representations Chang et al. (2020) and consequently reinitialize W.. We repeat the Phase I training, but this time the formulation of We also includes w: which are initialised with the updated z well. Here, the final per- label score is given by:
= (z), $z(2). wi) FT(x))
18


Reviews:
[{"review_id": "yKuUnyQWRmg", "reviewer": "Reviewer_xmDr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studied extreme multi-label classification problems where labels short text descriptions are available. The author proposed Gandalf, a data-augmentation technique that augmented the label text as input data, and used the label correlation graph as its soft-labels. The proposed Gandalf technique was applied to various XMC models, and showed consistent improvement on four XMC benchmarks.", "summary_of_the_review": "This work presented two novel data augmentation techniques, Gandalf and LabelMix, for the XMC problems. When applied to various downstream XMC  models, both Gandalf and LabelMix demonstrated consistent improvement over the baseline. Overall writing and presentation also are clear to follow.\n\nThe paper quality can be improved if the following technique questions are addressed.\n- Choice of label-to-label graph: Gandalf with Co-occurrence graph instead of LCG?\n- Sensitivity of hyper-parameters such as delta in Algo 1 and the design choice of y_tilde in Eq(9) versus Eq(8).\n- Does Gandalf enjoys larger performance gain on tail labels (e.g., label count < 5)? Can you plot the performance gain in y-axis, and the groups of label freq in x-axis?\n- I suppose Gandalf and LabelMix are complementary to each other? Can we apply both?\n- Can you apply Gandalf/LabelMix to two-tower models (e.g., DPR/ANCE) and do the evaluation on both XMC and IR benchmark datasets?\n", "strength_and_weaknesses": "*Strength*\n- The data augmentation techniques (Gandalf and LabelMix) are novel in XMC\n- Promising performance gain\n\n*Weaknesses*\n- Lack of ablation study to back the effectiveness of certain component in Gandalf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "q3WpWEMK9H", "reviewer": "Reviewer_qcbk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Extreme classification (or deep information retrieval) has been a popular research field that matches an input text (query) to a label that often is also a text. This paper focuses on a sub-field where both the input text and the label text are short. The proposed training algorithm is a data augmentation algorithm that generates similar examples to the observation based on the label to label similarities. Authors claim this new data augmentation technique was able to improve significantly on benchmark datasets.\n", "summary_of_the_review": "The paper does not convey a significant novelty in this field and was not sufficiently compared to many core techniques in this field.\n\n-----\nPost rebuttal response:\n\nI thank authors to provide detailed feedback. I also read other reviewers' feedback and concluded this paper does require further development to contain further contribution in this field. I still concerns the novelty of this paper (similar to Reviewer g2YZ).\n\nHere are some detailed points that the author replied.\n\n> Gandalf is only a method to leverage Label-Text/Features as data points whose ground truth labels are unknown.\n\nThis actually limits the novelty of the method. Gandalf's novelty is limited in introducing a heuristic method to generate additional labels. \n\n> Contrast to Hard Negative Mining \n\nThanks for the detailed description. I wonder then what's the actual benefit of using Galdalf's soft-labels. We often found strong negatives that are closer to the decision boundary helps achieving more discriminative models especially when the label space is large (XMC). Wouldn't it be more beneficial Galdalf to focus on closer but not positive labels?\n", "strength_and_weaknesses": "**Strength**\n* Since Gandalf is a pure data augmentation technique that can be applied to some set of algorithms easily. (however, it’s a bit arguable for some other sets of algorithms that can do similar techniques more organically. see weakness #1).\n* Authors try to set up a reasonable data-augmentation baseline, LabelMix, influenced by the popular “Mix-up” technique.\n\n**Weakness**\n* Selecting k most confusing examples via label similarities is not novel. It is sometimes called “hard-negatives” that is widely adopted in this field (for example, DPR [1], RocketQA [2] and ANCE [3]). The main difference is to use dense label space or to use the label relationship graph (they are quite interchangeable, but the dense space has nicer properties). It is also partially observed in Table 3 as using the soft-labels from the label graph further drastically increases the performance over hard labels.\n* Defining the label graph solely based on the label co-occurrences might have its limitations because it does not use any textual understanding of each label. Commonly, the dense label space is trained jointly with the actual retrieval task [1,2,3] and hence captures deeper understanding of the label to label similarities based on both bipartite co-occurrence relationships and textual understanding. Moreover, solely depending on co-occurrence relationships can be very noisy when we have large label space (e.g. thousands of millions or billions).\n* The work was not evaluated against common dense retrieval algorithms such as [1,2,3].\n\n[1] Karpukhin, Vladimir, et al. \"Dense passage retrieval for open-domain question answering.\" arXiv preprint arXiv:2004.04906 (2020).\n[2] Qu, Yingqi, et al. \"RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering.\" arXiv preprint arXiv:2010.08191 (2020).\n[3] Xiong, Lee, et al. \"Approximate nearest neighbor negative contrastive learning for dense text retrieval.\" arXiv preprint arXiv:2007.00808 (2020).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MvhPVXBsAJ", "reviewer": "Reviewer_g2YZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper present two data augmentation strategies, Gandalf and LabelMix, that adds label features or the instance-label interpolations as training instances in extreme multi-label classification (XMC) training. Empirical results show that the proposed Gandalf method is able improve the performance of various XMC models on the benchmark dataset where label features are present.", "summary_of_the_review": "The paper present two independent data augmentation tricks for the XMC-LF applications. Apart from the limited novelty concerns mentioned above, I think the topic of this paper is more related to a data mining venue rather than ICLR.", "strength_and_weaknesses": "**Strength**\n\n* The proposed method Gandalf shows relatively large improvement over 5 XMC models across 4 different XMC-LF dataset.\n\n**Weaknesses**\n\n* The proposed method, Gandalf, is rather straightforward. The core idea seems to be just applying normalization and some thresholding on top of the LCG constructed in ECLARE.\n* It is not clear to me how the Gandalf and LabelMix are related to each other. They seem to be two independent data augmentation tricks and I failed to see the necessity in introducing LabelMix in this paper.\n* In Section the authors claim that LCG is better than Label co-occurrence matrix but both are not as good as Gandalf which adopts a heuristic thresholding after row-wise normalization. However, I didn't see any empirical study that compares these three settings. It would be more convincing if there are some label-to-label augmentation baselines in the comparison.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "8K_Z0gGIzN", "reviewer": "Reviewer_k6DK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose to conduct graph induction for data augmentation for extreme classification based on label features. Specifically, they capture label-label correlations based on a label-correlation graph and use label features as data points. They also leverage label features to have query-label interpolation and then sample labels for mixing up augmented labels. The experiments based on four extreme classification benchmarks with label features demonstrate that the proposed method can enhance some extreme classification models. ", "summary_of_the_review": "In sum, I would recommend “3: reject, not good enough” as there are so many flaws in the experiments.", "strength_and_weaknesses": " Strengths\n* Simple and effective approaches that potentially can apply to arbitrary methods.\n* Publicly available benchmark datasets for reproducibility.\n\nWeaknesses\n* The authors “propose” LabelMix as a method as the “baseline”, but the more authentic approach should be to compare with published state-of-the-art works in data augmentation for XMC tasks.\n* Experiments only cover datasets with only short text (i.e., titles), which are comparatively limited in the real-world settings. The authors should consider conducting experiments on more comprehensive datasets with general information, such as LF-Amazon-131K, LF-WikiSeeAlso-320K, and LF-Wikipedia-500K.\n* Although the proposed techniques are potentially able to be applied to arbitrary methods, some methods like AttentionXML, XR-Transformer, and SiameseXML++ are not experimented with either LabelMix or Gandalf. Especially when vanilla XR-Transformer and SiameseXML++ outperform LabelMix and Gandalf in certain cases, I cannot see why the authors do not demonstrate that the proposed techniques can further improve the best performance.\n* The proposed techniques require label features, but in real-world settings, it usually lacks label features. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject