Image and Text: Fighting the same Battle? Super Resolution Learning for Imbalanced Text Classification
Authors: Romain Meunier
Benamara Farah
Véronique Moriceau
Patricia Stolf
Venue: EMNLP 2023 Findings (2023)
Keywords: Data Augmentation
Social media
Crisis management

Abstract: In this paper, we propose SRL4NLP, a new approach for data augmentation by drawing an analogy between image and text processing: Super-resolution learning. This method is based on using high-resolution images to overcome the problem of low resolution images. While this technique is  a common usage in image processing when images have a low resolution or are too noisy, it has never been used in NLP. We therefore propose the first adaptation of this method for text classification and evaluate its effectiveness on urgency detection from tweets posted in crisis situations, a very challenging task where messages are scarce and highly imbalanced. We show that this strategy is efficient  when compared to competitive state-of-the-art data augmentation techniques on several benchmarks datasets in two languages.

Fighting the Same Battle? Super-resolution Learning for Imbalanced Text Classification
Image and Text:
Romain Meunier and Farah Benamara and Véronique Moriceau' and Patricia Stolf! (1) IRIT, Université de Toulouse, CNRS, Toulouse INP, UT3, Toulouse, France
(2) IPAL, CNRS-NUS-ASTAR, Singapore firstname, Hirstname..astnam@@@i.. lastname@irit..r
Abstract
costs classes with small proportion (Lin et al., 2020; Li et al., 2020), (b) specific architectures such as multi-task learning leveraging auxiliary tasks (Spangher et al., 2021) and knowledge distil- lation that trains smaller model with soft labels (Tan et al., 2022), or (c) text interpolation that mixes up original and perturbed samples in the hidden space (Chen et al., 2022). While data-based techniques are easy to implement with low compu- tational cost compared model- ones (Wei and Zou, 2019), current methods have difficulty to generate meaningful instances as augmented data are often too close to the original ones, or semanti- cally different which may result in instances that do not make sense to humans (Feng et al., 2021; Ka- malloo et al., 2022). Prompt- based augmentation alleviates these issues but the price of training large-scale language models. In addition, they turn to be sensitive to prompt design terms of the choice and order of the input examples (Reynolds and McDonell, 2021).
In this paper, propose SRLANLP, new approach for data augmentation by drawing an analogy between image and text process- ing: Super-resolution learning. This method is based on using high-resolution images to overcome the problem of low resolution im- ages. While this technique is common usage in image processing when images have a low resolution or are too noisy, it has never been used in NLP. We therefore propose the first adaptation of this method for text classification and evaluate its effectiveness tection from tweets posted in crisis situations, a very challenging task where messages are scarce and highly imbalanced. We show that this strategy is efficient when compared to com- petitive state-of-thed atet data augmentation tech- niques several benchmarks datasets in two languages.
urgency de-
1 Introduction
The performances of NLP classification tasks are largely influenced by the availability of good qual- ity training data with sufficient number repre- sentative instances. This is particularity salient for (highly) imbalanced text classification problems, like malware detection (Oak et al., 2019), relation extraction (Gao et al., 2019), hate speech detection (Schmidt and Wiegand, 2017) as well urgency detection (Sánchez et al., 2023).
In this paper, we propose new approach for data augmentation by drawing an analogy between im- age and text processing: Super-vesolution learning (Yang et al., 2019). This method is based on using high-resolution images to overcome the problem of low resolution images and has been successfully employed in image processing when images have low resolution are too noisy (Peng et al., 2016; Koziarski and Cyganek, 2018; Tian et al., 2023; Fukami et al., 2023). It has however never been adapted to NLP classification tasks. In this con- text, we make a parallel between image and text by considering noisy and short user generated content (e.g., social media posts) as a low resolution image and long well written documents (c.g., news papers, Wikipedia, scientific articles) as a high resolution image which enables augmentation with more di- verse and high quality data with less cost. Mixed data from both types of resolution are then used to train supervised classifiers.
To augment minority classes, many data augmen- tation strategies have been proposed while ensuring models generalization. Roughly, these strategies can operate either at the data or model level. Data- based approaches generate synthetic new instances relying on lexical/syntactic variations (paraphras- ing, substitution, etc.) or pre-trained generative models (e.g., GPT2, GPT3) (Anaby-Tave) -avor al., 2020; Yoo et al., 2021; Chia et al., 2022). Model- based approaches on the other hand fight class im- balance during training via (a) some dedicated loss functions by assigning higher wrong classification

---

We measure the effectiveness of super-resolution data augmentation on single label text classification tasks which: (i) show a step-imbalanced distribu- tion with high fraction of minority classes (Buda et al., 2018), (ii) are characterized by long-tailed distribution with many instances for a small num- ber of classes, but few for the remaining classes (Henning al., 2023), (iii) involve low-quality in- stances (e.g., few tokens, ill-formed with grammar spelling/symtactic issues), and (iv) are about media events such as disasters (e.g., flood, earthquakes), health (e.g., vaccines, cancer, suicide) and sport, that are more likely to be discussed in factual exter- nal sources of information, adding therefore mean- ingful context. Among NLP applications meeting these criteria (e.g.. cybersecurity (Oak et al., 2019), medecine (Mayer et al., 2021; Ansari al., 2021)), we experiment with urgency detection from tweets (Castillo, 2016; Alam et al., 2021), a very challeng- ing task aiming at classifying messages posted dur- ing crisis situations according to fine-grained inten- tion to act categories (such as human/infrrstrucctue damages, warning/advice). In this task, messages that require an urgent and rapid action from res- cue teams are scarce and highly imbalanced com- pared non-urgent ones (Reuter al., 2018; Sari- oglu Kayi et al.. 2020), making it very interesting case study for super-resolution learning. In this paper, our contributions are:
settings used in this study, Section 5 presents the results. We compare our approach to related work in Section 6. Finally, we conclude drawing some perspectives for future work.
2
Data Augmentation with Super-Resolution
2.1 Super-resolution for Image Processing
Data augmentation has first been explored in the field of image processing with algorithms for geo- metric transformations, color changes, filters, ran- dom erasing, etc. (see (Shorten and Khoshgoftaar, 2019) for survey). Classifiers are mainly trained on high-resolution labeled data but do not perform well when tested on low-rcsolution data. Thus, super-resolution method, consisting in enhancing the resolution of an image, largely approved and applies many fields such as face recognition (Zou and Yuen, 2012), microscopic image (Nehme et al., 2), the domains of flow (Liu et al., 2020; Yousif et al., 2021; Fukami et al., 2023) or medicine (van Sloun et al., 2019).
Super-resolution gained interest for deep learn- ing with Peng al. (2016) who evaluated different training strategies for CNN models to improve fine- grained category classification of very low resolu- tion images. They artificially reduce the resolution of some high-resolution training data and show that mixing them during the training phase improves accuracy when models are tested on low-resolution images compared to training either high low- resolution only.
SRL4NLP, a simple yet effective adaptation of a well-established data augmentation strat- egy in image processing to text classification which does not rely on artificially generated data but existing high quality texts.
Thus, multi-resolution training, which randomly resizes images to different resolutions, allows to ac- commodate varying resolutions during testing and is used mainly to train CNN models. Recently, Tian et al. (2023) proposed ResFormer, a vision trans- former trained on multi-resolution images (several scalings of images) which obtains better results than transformer trained single resolution.
A study of the performances of this strategy disaster management from social media relying on benchmark datasets in both English and French.
A quantitative and qualitative evaluation in an out-of-type configuration where classifiers are evaluated in unseen event types during train- ing. Our results suggest that super-resolution learning is achieving competitive results when compared to state-of of-the-art data augmenta- tion techniques.
2.2 SRL4NLP: Super-resolution for NLP
Inspired by these results, we draw an analogy be- tween images and texts: (1) we consider that tweets are low-resolution texts as they are short and of- ten low-quality, noisy texts; and (2) we propose to augment low-resolution training data with similar texts with higher resolution (see Figure 1). We consider high-resolution texts as clean texts with a higher number of tokens than tweets. Instead of generating artificial high-resolution texts from low-
In the following, Section 2 presents the origi- nal machine-learned super-resolution method and how we adapted it for NLP. Section 3 presents the datasets and Section 4 presents the experimental

---

Train
Peng (2016)
Test
High-Resolution
CNN
Low Low-Resolution
NLP Adaptation
Label: :
Damage
Approximately 32,000 people were displaced in shelters across the state by August 31. The George R. Brown Convention Center, the state's largest shelter, reached capacity with 8,000 evacuees. The NRG Center opened a large public shelter accordingly. More than 210,000 people registered with FEMA for disaster assistance.
Label:
High Resolution
Still on guard. Possible flooding threat for Friday with remnants of #Harvey OH Valley and Carolinas.
Language Model
Label: Damage
Low-Resolution
#HHarvey has far cost at least 31 lives, forced 30,000+ people from homes in TX and caused extensive destruction.
Figure 1: Adaptation of Peng et al. (2016) super-resolution learning text classification, illustrated here the task of urgency detection, our case study.
resolution ones before during training often done in image processing and avoid additional training costs, we propose augment data with existing texts such as newspaper, encyclopedic ar- ticles, or scientific publications, and to mix them with tweets for the training step.
et al., 2013; McCreadie al., 2019; Sarioglu Kayi et al., 2020; Kozlowski et al., 2020). We evaluate several data augmentation strategies on different datasets English and French since datasets for crisis management are available in these languages (see Table 1).
SRLANLP implies to have high-resolution dataset about the same topics the low-resolution one, and annotated with the same labels. If not available, we propose method in Scction to collect and automatically annotate high-resolution data from structured texts, such as Wikipedia arti- cles.
3.1 Low-Resolution datasets
KoZLoWsKLnerge' This dataset is the largest French dataset for crisis management composed of 16,202 tweets about various expected (c.g., storms) and sudden crisis (e.g.. explosions) events (Ko- zlowski et al., 2020; Bourgon et al., 2022). It contains tweets annotated for three urgency cat- egories as well as 6 intentions act categories: (1) URGENT that applies messages mentioning HU- MAN/MATERIAL DAMAGES as well security in- structions (ADVICE- (AECCEN)) limit these dam- ages during crisis events, (2) NOT URGENT that groups SUPPORT messages to the victims, CRITICS or any OTHER messages that do not have an im- mediate impact on actionability but contribute in raising situational awareness, and finally (3) NoT USEFUL for messages that are not related to the targeted crisis. To be comparable with the follow- ing English dataset, some intent classes have been merged into more coarse-graine gaasee-rraied classes: HUMAN- DAMAGE and MATERIAL-DAMAL into DAM- AGE; SUPPORT and CRITICS into SOCIAL while
3 Datasets
The literature emergency detection has been growing fast in the recent years and several datasets (mainly tweets) have been proposed account for crisis-related phenomena. Messages are annotated according to relevant categories that are deemed to the information needs of various stakeholders like humanitarian organizations, local police and firefighters. Well-known dimensions include re- latedness to identify whether the message content is useful (Jensen, 2012) and situation awareness (also known urgency) filter out on-topic rel- evant vs. on-topic irrelevant information (Imran
See https:// qwwp.e.crroooooooo org/ / for an overview.

---

URGENT
NOT URGENT
NOT USEFUL
TOTAL
DAMAGE
ÁDVICE WARNING
SOCIAL
ОTHER
KoZLowSKlmery
908
1,456
1,128
1,352
11.358
16,202
HUMAID meegge
908
1,456
1,128
1,352
11,358
16,202
FRAUTO
603
276
0
1,032
HRAUTO_NO, SO___NOSS
603
276
0
879
ENAUTO NO SOCIAL
1,160
0
0
1,160
Table 1: Distribution of each dataset in terms of urgency (3 classes) and intent (5 classes).
keeping imbalanced distribution with 71 71,, 8,98% and 5.60% for NoT USEFUL, SOCIAL and DAM- AGE respectively.
bel c; is triggered by a set of keywords K
(ki,..., kz). and
(di,.., dm) where dj =< (S P,t)+ > such that S [8 8jm) the n sections of dj E D, and Pa {Pn.l....,, Pn,r) the r paragraphs of Sj.
D
HUMAID merge: merge:: It is an English dataset of 77,196 tweets annotated for crisis management following 10 intent categories and one NoT USEFUL class (Alam et al., 2021). This latter class having few instances (1, (1,986) and as we wanted ity class as KoZLOwSKlmerges we added 9,372 NOT USEFUT. instances from other crisis datasets annotated for relatedness (CRisISLEXT6 (Olteanu et al., 2014) and CRISISLEXT26 (Olteanu et al., 2015)). To allow better comparison, we aligned HUMAID annotations with KozLOwsKlmerge by merging similar categories (e.g.. injured/dead peo- ple and missing or found people into the unified label DAMAGE) while discarding irrelevant ones (e.g., no decision). We have also randomly selected the same number of tweets from the same classes as in KOZLOWSKI merge to have an equivalent dis- tribution and comparable size.
be a major-
Algorithm 1: HIGH-RESOLUTION TEXT
SCRAPPER AND ANNOTATION
Input: C, K, D P Output: label(p,)
for d, in D do
for Sij in S. do
3
for on in C do if kn 8y then for Pr in P; do label(p,) = Ca
3.2 High-Resoe Miggh-Resolution datasets
For our use case, we decided to use Wikipedia as external source as it contains a large variety of factual and valuable information for understanding the nature, causes, impacts, and responses to dif- ferent crises (e.g., damage and weather reports, re- sponses and mitigation efforts, media coverage and public reactions) making well suited for super- resolution. In addition, it is copyright free, easy to scrap, and comes with structured format with ded- icated sections and paragraphs for each Wikipedia page. Thus, the following datasets, built with Al- gorithm
SRLANLP implies to have high-resolution dataset about the same topics the low-resolution one, and annotated with the same labels, so that both datasets be mixed during the training step. However, as there no available high-resolution dataset for crisis management which fit our needs, we defined a simple but generic method to collect and annotate automatically high-resolution texts from existing structured documents. This method avoids additional computational costs due to arti- ficial data generation and manual annotation. The selection and annotation of high-resolution texts from structured documents follow Algorithm which annotates paragraphs of relevant documents according to the title of the section they belong to; assuming set of labels of a classification task for a given use case and a set of documents D re- lated to this use case structured in terms of sections S and paragraphs P. such that:
composed of high-resolution para- graphs from Wikipedia, used to augment all mi- nority classes, except NoT USEFUL, the majority class:
AUTO: the set relevant documents is composed of the 152 links in the French Wikipedia page Natural disasters by year". The sets of key- words K used for the matching of section titles
hhttps://fr.. http:///ff.iiiipeii.. www.wwwiwiiiiiiii..cm..m attastrophe__ntturelll...gg pan__an3ee33
C = [((1, ((c,,))). (Ca, kn)) where each la-

---

with classes are given in Appendix A. This results in 1,032 labelled paragraphs, each of them being related to a crisis type given by the document title. Among them, we manually checked 144 randomly selected paragraphs and agreed with the automatic annotation (see an example in Appendix C).
to FlauBERT (e.g. GPT3 was only able cor- rectly classify 35% of messages into intention act categories).
Following the general trends in crisis manage- ment (Kersten et al., 2019; Algiriyage et al., 2021; Bourgon et al., 2022), we designed an out-of-type evaluation protocol by training related to different types of crises (e.g., Hurricane. Storm) and testing on particular different type (e.g., Earthquake). The aim is to evaluate if model can deal with new types of crisis, which is crucial to ensure the portability of the models to unseen events.
unlike Twitter, texts in Wikipedia which can be labelled as SOCIAL (i.e. critics, support) are rare. For this reason, we have created a subset of FRAUTO. named FRAUTO_M LOs__NOOSOCCLLLL which contains only the data labelled DAMAGE and ADVICE_ WARNING.
AUTO_NO_SOOC
pool of events
ENN AUTO_NO, AUTO_NO_SOCIAN AO in the same way, 213 links in the English Wikipedia page Natural dis- asters by year" have been scrapped leading to the automatic selection and annotation of 1,160 para- graphs as DAMAGE or ADVICE ADVICE_WARIE WARNING (see Appendix B for the English matching lexicon). Table shows the distribution of these aug- mented datasets.
Each corpus contains different types of crisis: Fire, Flood, Storm, Hurricane, Collapse and Ter- rorist attack for KOZLOWSKl and Fire, Flood, Storm, Hurricane, Earthquake for HUMAID merge- Thus for each dataset, an experiment consists the average of n runs, each run with n 1 crisis types for training and the remaining crisis type for testing. For HUMAID merge: as a random data sam- pling is done so that the dataset can be comparable with the distribution of the French one and as the initial English dataset is pretty large (around 86K), created 5 independent batches of 16,202 ran- dom tweets per batch conserving the same class distribution
4 Models and Evaluation Protocol
4.1 Experimental Settings
For testing the datasets in French, we used FlauBERT, Fine-TTued which is the best perform- ing model for intent classification on the French dataset according to (Kozlowski et al., 2020). It is FlauBERT model (Le et al., 2020) that has been fine-tuned 358,834 unlabelled tweets posted dur- ing crises. For English, we used RoBERTa (Liu ul., 2019) which is reported to be efficient for crisis management in English (Koshy and Elango, 2022; Rocca et al., 2023; Madichetty et al., 2023). For both models, we used the Focal Loss with an Adam optimizer.
KoZLOWSKInerss and reported the averaged macro F-scores.
For both datasets, we replaced all numbers and URLs by a dedicated token and removed user men- tions.
4.2 Methods
In addition to our super-resolution data augmen- tation method, we evaluate several state-of-the-art approaches:
Regarding the classification task, we do not re- port any comparative analysis with existing main- stream large language model such as GPT for three main reasons. First, our aim is to design a sim- ple approach that requires very low computational cost. Second, as we used GPT3 to generate the new augmented data and so learned from these data, we decided to not use it for classification because results may be biased. Finally, we also ex- perimented classification with GPT3 on 100 tweets of KozLLoSSKImerge- However, it was unable to distinguish OTHER and NOT USEFUL messages, achieving therefore low performances compared
4.2.1 Data-based approaches
Synonym replacement (Zhang et al., 2015): This method substitutes a random word in text by one of his synonym selected in thesaurus based language model prediction. We used the im- plementation from the NLPAUG library."
Contextual augmentation: Based on Kobayashi (2018), this method requires a deep learning model calculating the word probability at a given position based on its context. Then, it replaces the word at this position by the predicted word.
Back Translation (Yu et al., 2018): This method translates text from a source language
"nttps://en.wikiiedai.a. http://ww...aaaogg//////eegggo/e datural.disasters_byy...
potttps//giien httpttttb..oommmmacceeere///apap

---

Text
Method
(I) #Harvey has far cost at least 31 lives, forced 30,000+ people from homes in TX and caused extensive destruction.
Original tweet
(2) #Hrrey has cost at least 31 lives so far, forced more than 30,000 people to leave their homes
Back translation
in TX and caused considerable destruction.
(3) # Moreve It has so far cost quigo 31 liveds, forced 3 ct+ over from homes in TX that causes
Synonym replacement
some extensive destruction.
(4) Old #lroovey the has
lar cost at high leostast 31 lives, forced 30,000+ people from homes in TX and caused extensive place destruction.
Contextual augmentation
(5) The impact #Harvey has been devastating, with 31 lives lost, over 30,000 people displaced from their homes in TX, and widespread destruction. Let's stand together and support those ToobuiiidigTTggeheee
Generative augmentation
affected by this crisis. #SupportFortHarve Victees Victims
(6) Approximately 32,0000 people were displaced in shelters across the state by August 31. The George R. Brown Convention Center, the state's 's largest shelter, reached capacity with 8,000 evacuees. The NRG Center opened as a large public shelter accordingly. More than 210,000
Super-resolution
people registered with FEMA for disaster assistance.
Table 2: Examples of augmented data from tweet.
into a target language and then translates it back to the source. We used Helsinki-NLPPOPUU--MT train (Tiedemann and Thottingal, 2020)² to trans- late the French dataset into English and then back into French. For the English dataset, we translated it into French and then back into English.
Multi-task learning (Ye et al., 2019): This strategy trains several layers on different but related tasks to improve the performance. This learning architecture has already been used for crisis man- agement by Kozlowski et al. (2020) and Khanal and Caragea (2021). Following Kozlowski et al. (2020), the tasks are utility (useful vs. not useful), urgency (urgent vs. not urgent vs, not useful) and intention (5 classes) detection, the latter being our main task. For all tasks, we used the Focal loss function and FlauBERT Fine-TTned Toned for French and RoBERTa for English.
Generative augmentation: Inspired by Yoo et al. (2021), we used GPT3 (Brown et al., 2020) to generate new data similar to tweets of our datasets. The prompt asks to generate 10 new tweets similar to 5 tweets from the original dataset that share the same label but can be from different crises.
To be able to make comparison between classification models, we used the above methods to augment data with the same dis- tribution as FRAUTO_AN L.NNSOSSILLLL to augment KOZLOWSKIInrgg merge and EN ENAUTO_ME ___NN_SOCIIAL to augment HUMAID merge. Table 2 presents examples of augmented data obtained from one tweet with the above methods, showing for some of them the low quality of what is generated.
5 Results
5.1 Quantitative results
Table 3 presents the results obtained by FlauBERT Fine Taned on the KOzI.oWSKI dataset (5 classes) and its augmented versions. The model obtains the best results on the dataset augmented with FRAUTO_NO _O__OCIIL via the automatic super-resolution method. We note that augmenting only the classes DAMAGE and WARNING with few high-resolution texts from Wikipedia can even help improving the performance on the class SOCIAL which has not been augmented. Indeed, Wikipedia contains informative data, similar to what is found in tweets for both classes DAMAGE and ADVICE- WARNING whereas it contains almost no SOCIAL data. Thus, augmenting only the first two classes helps the model to be more discriminant on the class SOCIAL.
4.2.2 Model-based approaches
We also compare with the following two model- based methods they have been shown to be quite effective for crisis management.
Manifold Mixup (Verma et al., 2019): It has been adapted to crisis management by Chowdhury et al. (2020a). It makes semantic interpolation during training, obtaining neural networks (here, FlauBERT Fine-Teed for French and RoBERTa for English) with smoother decision boundarics."
Table 4 presents the results obtained by
hhttps://gthhu. wtwptttuw..omm/essiiniiii-t. APUS-WT-train 'https: http://gggghuubccc//isseeeeeeeee / /owp
RoBERTa on the HUMAID merge dataset (5 classes)
mixup

---

Dataset
Damage
Advice Larning
Social
Other
Not Useful
Total
KoZLOWSKImerge
54.87
46.94
54.62
23.86
82.83
52.62
Manifold Mixup
55.95
46.67
50.65
26.00
82.64
52.38
Multi-task
57.76
47.15
50.99
30.50
82.86
53.85
KozLowSKlmers
+Back Translation
48.94
46.48
49.25
26.40
83.00
50.81
DOZLOWSKlmert
+Gnerattive augmentation
52.77
47.66
50.74
22.09
81.83
51.02
KozLowSKl murrge
+ Contextual augmentation
58.37
48.04
50.84
24.41
82.05
52.74
KoZLoWSKImerg
+Synonym replacement
57.04
47.38
51.35
28.12
81.22
53.02
FR AUTO
55.67
47.75
50.18
22.51
81.37
51.50
FR MAUTT_NN_SOCCIII
58.65
48.33
56.00
25.73
82.26
54.19
Table 3: Results FlauBERT Fine-Twned in terms average F-score on KoZLOWSKlmerge for out-of-type experiment.
Dataset
Damage
Advice Warning
Social
Other
Not Useful
Total
HUMAID merrg (5 batches)
69.87
73.04
82.51
39.18
92.05
71.33
Manifold Mixup
70.42
72.78
82.30
41.26
92.26
71.81
Multi-task
73.99
68.04
81.16
37.43
91.47
70.42
HUMAID merge
+Back Translation
74.86
72.66
81.21
37.31
90.77
71.36
HUMAID merge
+Genraative augmentation
68.46
71.14
75.89
31.26
91.12
67.57
HUMAID merge
+Contextual augmentation
87.59
71.11
82.06
41.78
91.56
74.82
HUMAID merge
+Synonym replacement
87.61
73.50
81.17
41.03
91.78
75.02
EN NO. SOCIAL
84.90
73.02
83.52
42.03
91.38
74.97
Table 4: Results of RoBERTa in terms of average F-score on HUMAID merge out-of-rype experiment.
Dataset
Damage
Advice Warning
Social
Other
Not Useful
Total
HUMAID merge batches)
71.23
65.58
93.66
42.31
80.48
70.65
Manifold Mixup
70.79
67.12
93.57
42.58
80.12
70.83
Multi-task
73.39
68.49
78.63
40.35
93.87
70.95
HUMAID merge +Back Translation
70.99
72.37
93.49
41.76
79.13
71.54
HUMAID merge +Gencrative augmentation
69.77
64.63
93.88
33.68
72.05
66.80
HUMAID merge +Contextual augmentation
69.33
93.44
45.35
80.00
74.99
HUMAID merge
+Synonym replacement
72.63
86.14
93.46
44.65
78.48
75.07
ENAUTO NO_ SOCIAL
71.52
86.99
93.88
45.74
81.70
75.97
Table Results of RoBERTa in terms average F-score on HUMAID for out-of-type experiment without the test Earthquake.
and on its augmented versions. Except Genera- five augmentation and Multi-task, all augmenta- tion methods outperform the baseline on the orig- inal dataset. We can make similar observation for the French dataset: augmenting only the class DAMAGE with EN AUTO_NO_SOO _NO_SOCCILLLLL helps improving the performance on the class SOCIAL. SRL4NLP 1. is second behind Synonym replacement and its results are statistically significant compared to those obtained on the original HUMAID mergge dataset (p-value=1.66e-11 12 with the McNemar test). When looking into details at the 5 runs on each crisis type, we noticed that Earthquake was out- lier for which our model underperformed. The test run on Earthquake has F-score of 70.99 when augmented with SRL4NLP whereas the F-score is 74.05 on the original dataset: this crisis type has the strongest decrease compared to other cri-
sis types. This may be explained by the fact that only 57 paragraphs have been added to augment the data for earthquake, representing much less data compared to other crisis types (at least 100 paragraphs). Table 5 presents the results obtained without the run earthquake. In this configu- ration, SRL4NLP outperforms all taate-of-the-and augmentation methods and is more efficient for all classes except DAMAGE.
Concerning recall and precision, FlauBERT Pine-Taned obtained a precision of 55.24 (resp. 55.98) and recall of 53.33
(resp. 55.53) on KozZOWsKIImeoo (resp. when augmented with SRL4NLP). RoBERTa obtained precision of 72.01 (resp. 74.03) and recall of 73.11 (resp. 77.85) on HUMAID merge (resp. when augmented with SRL4NLP). SRL4NLP increases mainly recall, which fits particularly

---

with the needs of crisis management where the emergency services need be aware of all urgent messages.
augmentation for image classification and propose SRLANLP, a novel augmentation paradigm adapt- ing the well-established super-resolution learning to text classification. Our approach allows for adding new linguistic patterns in the train set re- lying on structured external textual sources and avoids generating any new data by using existing high-resolution data for low computational cost.
5.2 Error analysis
A manual error analysis highlights three main causes of misclassification. For example, in Ta- ble 6, the predicted label in messages (1) and (2) seems more accurate (support) than the gold label. Indeed, the tweet (I) is about displaced people so it can be considered as DAMAGE but the main ob- jective of the tweet is an intent to support these people. In tweet (2), the support is represented by the hashtag #PrayfooGrreece In tweet (3), the word damage misled the model to consider this tweet as DAMAGE. Finally, message (4) could have multi- ple labels: the gold label ADVICE- WARNING since the tweet declares a state of emergency but also the predicted one DAMAGE since it mentions people evacuation.
6.2 NLP for Crisis Management
Social media, such as Twitter, can give precious real-time information in crisis situations (Reuter et al., 2018) (c.g., more than 1,000,000 tweets posted during in Türkiye and Syria earthquake in 2023 (Toraman et al., 2023)), making NLP-based crisis management hot research topic. Recently, shared tasks have proposed, such TREC-IS (In- cident Streams track)? (McCreadie et al., 2019, 2020), and CrisisFACTS20022 (McCreadie and Buntain, 2023) which allows the development of classifiers in three settings: (1) relatedness (Is message useful/relevant for emergency depart- ments?), (2) urgency (Is a message urgent for emer- gency departments? Possibly what is the urgency degree?), and (3) intents (damage reports, warnings. critics, etc.).
6
Related Work
6.1
Data Augmentation in NLP
Various methods have been developed to generate artificial data in order to enhance the size and the quality of training datasets, avoiding collecting and annotating data manually (Feng et al., 2021). Lan- guage models can be used for back translation (Yu et al., 2018) 8) or for synonym replacement and con- textual insertion (Zhang et al., 2015; Kobayashi, 2018), keeping the semantic of the original sen- tences in the artificial data. Coulombe (2018) pro- posed error injection to be more robust the noise found in social media posts. Rule-based methods can also be used to add, replace or swap words in a sentence (Wei and Zou, 2019), avoiding us- ing costly language models. Recently, generative language models such as GPT3 have also been employed to generate data and augment existing datasets in a few/zero-shot scenario by using the class label and dedicated prompts as cue for the model (Anaby-Tavor et al., 2020; Yoo et al., 2021; Chia et al., 2022: Ocampo et al., 2023).
Classifiers are trained in a supervised way with either traditional fcature- learning algorithms (Alam et al., 2021; Li et al., 2018; Kaufhold et al., 2020) or deep learning architectures (Caragea et al., 2016; Neppalli et al., 2018; Kersten et al., 2019; Kozlowski et al., 2020; Chowdhury et al., 2020b; Liu al., 2021; Wang et al., 2021; Dusart et al., 2021). Overall, results show that intent detection is the most challenging task due to the extremely imbalanced nature of crisis datasets: urgent mes- sages are minority (e.g., about 3.87% and 1.93% for infrastructure/huaan damages respectively in CrisisFACTS).
The imbalanced issue has however received less attention in the literature. Among existing works, Bayer et al. (2021) used text generation, Kruspe et al. (2018) 18) relied on automatic round-trip trans- lation augmenting under-represented classes in the TREC-IS dataset. Ramachandran and Ramasubra- manian (2021) proposed a synonym and hypernym replacement strategy based on WordNet. Finally Chowdhury al. (2020a) explored text interpola- tion via Manifold Mixup in a multilingual setting
Model-based approaches on the other hand make use of multi-task learning leveraging auxiliary tasks (Spangher et al., 2021), knowledge distillation that trains smaller model with soft labels (Tan et al., 2022), and text interpolation that mixes up original and perturbed samples in the hidden space (Chen et al., 2022).
htt des. gla. ac www.rriicardn//REC..... "https://crisisfacts github.io.
In this paper, we relied on advances in data

---

Text
Gold label
Predicted Label
(I) Want to help support the thousands of residents displaced by
Damage
Social
the Forra massive wildfire Don t get scammed
(2) Athens is either under wildfires under storms
Not Useful
Social
hate this summer PrayForGreece
(3) Fort McMurray Trudeau arrives to view wildfire damage
Other
Damage
(4) Alberta Canada declared a state of emergency after
Advice_Wannnn Warning
Damage
a wildfire forced more than numero numero people to evacuate
Table 6: Examples of misclassification by RoBERTa on HUMAID merge
merge dataset.
with BERT. The use of super-resolution in the field is novel.
French but the method and the evaluation protocol for this case can be extended to datasets in other languages since Wikipedia pages available in many languages.
7 Conclusion
We think that it can be also easily transfered to other classification use cases for which low- resolution datasets and high-resolution comparable external resources are available: for example, hate speech detection by augmenting datasets of tweets with posts discussion forums; political opinion detection by augmenting datasets from social me- dia with political speeches; medical diagnosis by augmenting patient records with medical publica- tions, etc.
By drawing an analogy between image and text processing, we have proposed SRL4NLP, a new approach based on super-resolution learning to aug- ment low-resolution imbalanced training data (for example, tweets) with similar texts having higher resolution (paragraphs).
We have evaluated its effectiveness on urgency detection from tweets posted in crisis situations, relying on imbalanced benchmark datasets both English and French. We have compared the classifi- cation performances over these datasets augmented with few data obtained with several data augmenta- tion methods. The results show statistically signifi- cant F-scores in out-of-type evaluation setting, out- performing existing data-based and model-based augmentation approaches. SRL4NLP has also very low computational cost since it avoids the generation of artificial data and their manual anno- tation.
Ethies Statement
The data that was used for conducting the experi- ments is composed of texts from the public domain taken from either datasets publicly available to the research community, or Wikipedia. The datasets are anonymized and contain no offensive or abu- sive language. They were collected before Twitter changed to X and conform to the Twitter Developer Agreement and Policy that allows unlimited distri- bution of either the numeric identification number or the textual content of each tweet.
For future work, we plan to rely on other external sources for augmenting our data (c.g., news, gov- ernment reports, return of operating experiences) well experiment with other use cases (see below).
References
Acknowledgements
Firoj Alam, Umair Qazi, Muhammad Imran, and Ferda Ofli. 2021. Humaid: Luman-annotated disaster in- cidents data from twitter with deep learning bench- marks. Proceedings of the International AAAI Conference on Web and Social Media, volume 15, pages 933-942.
The research of Farah Benamara is partially sup- ported by DesCartes: The National Research Foun- dation, Prime Minister's Office, Singapore under its Campus for Research Excellence and Techno- logical Enterprise (CREATE) program.
Nilani Algiriyage, Rangana Sampath, Raj Prasanna. Emma EH Doyle, Kristin Stock, and David John- ston. 2021. Identifying disaster-related tweets: a large-scale detection model comparison. In Social Media in Crises and Conflicts, Proceedings of the 18th ISCRAM Conference, pages 731-743.
Limitations
this paper, we have evaluated SRL4NLP on cri- sis management benchmark datasets in English and

---

Ateret Anaby-Tavor, Boaz Carmeli, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020. Do not have enough data? deep learning to the rescue! In AAAI, pages 7383- 7390. AAAI Press.
Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si. 2022. RelationPrompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction. In Findings of the Association for Compu- tational Linguistics: ACL 2022, pages 45-57, Dublin, Ireland. Association for Computational Linguisties.
Gunjan Ansari, Muskan Garg, and Chandni Saxena. 2021. Data augmentation for mental health classifi- cation social media. In Proceedings of the 18th International Conference on Natural Language Pro- cessing (ICON), pages 152-161, National Institute of Technology Silchar, Silchar, India. NLP Association of India (NLPAI).
Jishnu Chowdhury, Comella Caragea, and Doina Caragea. 2020a. Pross--lingual disaster-related multi- label tweet classification with manifold mixup. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics: Student Re- search Workshop, pages 292-298, Online. Associa- tion for Computational Linguistics.
Markus Bayer, Marc-André Kaufhold, Bjöm Buchhold, Marcel Keller, Jörg Dallmeyer, and Christian Reuter. 2021. Data augmentation in natural language pro- cessing: A novel text generation approach for long and short text classifiers. CoRR, abs/2103. 14453.
Jishnu Ray Chowdhury, Cornelia Caragea, and Doina Caragea. 2020h. On identifying hashtags in disaster twitter data. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI).
Nils Bourgon, Farah Benamara, Alda Mari, Véronique Moriceau, Gaetan Chevalier, and Laurent Leygue. 2022. Are Sudden Crises Making me Collapse? Mea- suring Transfer Learning Performances on Urgency Detection. In 19th International Conference on Infor- mation Systems for Crisis Response and Management (ISCRAM 2022).
Claude Coulombe. 2018. Text data augmentation made simple by leveraging nlp cloud apis. ArXiv, abs/1812.04718.
Alexis Dusart, Karen Pinel-Sauvagee Sauvagnat, and Gilles Hu- bert. 2021. Issumset: A tweet summarization dataset hidden in trec track. In Proceedings of the 36th An- nual ACM Symposium on Applied Computing, SAC 221, page 665-671, New York, NY, USA. Association for Computing Machinery.
Tom Brown Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Πlya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. A survey of data augmentation approaches for NLP. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 968-988, Online. Association for Computa- tional Linguistics.
Kai Fukami, Koji Fukagata, and Kunihiko Taira. 2023. Super-resolution analysis via machine learning: sur- vey for fluid flows. Theoretical and Computational Fluid Dynamics, pages 1-24.
Mateusz Buda, Atsuto Maki, and Maciej A. Mazurowski. 2018. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249-252
Tianyu Gao. Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2019. FewRel 2.0: Towards more challenging few-shot relation classi- fication. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNN.P-IJCNTPP) pages 6250-6255 Hong Kong, China, Association for Com- putational Linguistics.
Cornelia Caragea, Adrian Silvescu, and Andrea Tapia. 2016. Identifying Informative Messages in Disas- ters Events using Convolutional Neural Networks. In 13th International Conference on Information Sys- tems for Crisis Response and Management (ISCRAM 2016), pages 1-7.
Carlos Castillo. 2016. Big crisis data: social media in disasters and sime-critical situations. Cambridge University Press.
Sophie Henning, William Beluch, Alexander Fraser. and Annemarie Friedrich. 2023. A survey of meth- ods for addressing class imbalance in deep-learning based natural language processing. In Proceedings of the 17th Conference of the European Chapter of the A Asociation for Computational Linguástics, pages 523-540, Dubrovnik, Croatia. Association for Com- putational Linguistics.
Hui Chen, Wei Han, Diyi Yang, and Soujanya Poria. 2022. DiubleMix: Simple interpolation-bait data augmentation for text classification, In Proceedings of the 29th International Conference on Computa- tional Linguistics, pages 4622-4632, Gyeongju, Re- public of Korea. International Committee on Compu- tational Linguistics.
Muhammad Imran, Shady Mamoon Elbassuoni, Carlos Castillo, Fernando Diaz, and Patrick Meier. 2013.

---

Extracting information nuggets from risaster-related messages in social media. Proc. of ISCRAM, Baden- Baden, Germany.
Anna Kruspe, Jens Kersten, Matti Wiegmann, Benno Stein, and Friederike Klan. 2018. Classification of incident-related tweets: Tackling imbalanced training data using hybrid enns and translation- data aug- mentation. In Text REtrieval COnference (TREC).
Gunilla Elleholm Jensen. 2012. Key criteria for in- formation quality in the use of online social media for emergency management in New Zealand. Ph.D. thesis, Open Access Te Herenga Waka- Victoria Uni- versity of Wellington.
Hang Le. Loic Vial, Jibril Frej, Vincent Segonne, Max- imin Coavoux, Benjamin Lecouteux, Alexandre Al- lauzen, Benoit Crabbé, Laurent Besacier, and Didier Schwab. 2020. FlauBERT: Unsupervised language model pre-training for French. In Proceedings of the Twelfth Language Rexources and Evaluation Confer- ence, pages 2490,, Marseille, France. European Language Resources Association.
Ehsan Kamalloo, Mehdi Rezagholizadeh, and Ali Gh- odsi. 2022. When chosen wisely, more data is what you need: A universal sample-efficient strategy for data augmentation. In Findings of the Association for Computational Linguistics: ACL 2022. pages 1048- 1062, Dublin, Ireland. Association for Computational Linguisties.
Hongmin Li, Doina Caragea, Cornelia Caragea, and Nic Herndon. 2018. Disaster response aided by tweet classification with a domain adaptation approach. Journal of Contingencies and Crisis Management. 26(1):16-27.
Marc-André Kaufhold, Markus Bayer, and Christian Reuter. 2020. Rapid relevance classification social media posts in disasters and emergencies: A system and evaluation featuring active, incremental and on- line learning. Information Processing Management, 122133.
Xiaoya Li, Xiaofei Sun, Yuxian Meng. Junjun Liang, Fei Wu, and Jiwei Li. 2020. Dice loss for data- imbalanced NLP tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 465-476, Online. Association for Computational Linguistics.
Jens Kersten, Anna Kruspe. Matti Wiegmann, and Friederike Klan. 2019. Robust Filtering of Crisis- related Tweets. In ISCRAM 2019 conference proceedings- 16th international conference on infor- mation systems for crisis response and management.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár, 2020. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318-327.
Sarthak Khanal and Doina Caragea. 2021. Multi task learning to enable location mention identification in the early hours of a crisis event. In Findings of the Association for Computational Lingwisties: EMNLP Punta Cana, Dominican Re-
Bo Liu, Jiupeng Tang, Haibo Huang, and Yi-Yun Lu. 2020. Deep learning methods for super-resolution reconstruction of turbulent flows. Physics of Fluids, 32(2). 025105.
2021, pages public. Association for Computational I Linguisties.
Junhua Liu, Trisha Singhal, Lucienne T.M. Blessing. Kristin L. Wood, and Kwan Hui Lim. 2021. Crisis- bert: A robust transformer for crisis classification and contextual crisis embedding. In Proceedings of the 32nd A ACM Conference on Hyperiext and Social Media, HT '21, page 133-141, New York, NY, USA. Association for Computing Machinery.
Sosuke Kobayashi. 2018 Contextual augmentation: Data augmentation by words with paradigmatic re- lations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 2 (Short Papers), pages 452-457, New Orleans, Louisiana. Association for Computa- tional Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint 1907.11997..1999
Rani Koshy and Sivasankar Elango. 2022, Multimodal tweet classification in disaster response systems us- ing transformer-hasee bidirectional attention model. Neural Computing and Applications, 35.
Sreenivasulu Madichetty, Sridevi Muthukumarasamy. and Sreekanth Madisetty. 2023. A roberta based model for identifying the multi-modal informative tweets during disaster. Multimedia Tools and Appli- cations.
Michat Koziarski and Boguslaw Cyganek. 2018. Impact of low resolution on image recognition with deep neu- ral networks: An experimental study. International Journal of Applied Mathematics and Computer Sci- ence, 7854)7335-744
Tobias Mayer, Santiago Marro, Elena Cabrio, and Serena Villata. 2021. Enhancing evidence-based medicine with natural language argumentative anal- ysis of clinical trials. Artificial Intelligence in Medicine, 102:09.
Diego Kozlowski, Elisa Lannelongue, Frédéric Saude- mont, Farah Benamara, Alda Mari. Véronique Moriceau, and Abdelmoumene Boumadane. 2020. A three-level classification of french tweets in ecologi- cal crises. Information Processing & Management, 57(5): 102284.
Richard McCreadie and Cody Buntain. 2023, Crisis- facts: Building and evaluating crisis timelines.

---

Richard McCreadie, Cody Bumtain, and Ian Soboroff. 2019. TREC Incident Streams: Finding Actionable Information on Social Media. In Proceedings of the 16th International Conference on Information Sys- lems for Crisis Response and Management, València, Spain, May 19-22, 2019. ISCRAM Association.
An evaluation and analysis of crisis informatics re- search. International Journal of SumannCoompun Interaction, 34(4))280-294,
Laria Reynolds and Kyle McDonell. 2021. Prompt pro- gramming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Com- puting Systems, CHI EA '21, New York, NY, USA. Association for Computing Machinery.
Richard McCreadie, Cody Buntain, and Ian Soboroff. 2020. Incident streams 2019: Actionable insights and how to find them. In Proceedings of the 17th ISCRAM Conference.
Roberta Rocca, Nicolò Tamagnone, Selim Fekih, Xi- mena Contla, and Navid Rekabsaz. 2023. Natural language processing for humanitarian action: Oppor- tunities, challenges, and the path toward humanitar- ian nlp. Frontiers in Big Data, 6.
Elias Nehme, Lucien E. Weiss, Tomer Michaeli, and Yoav Shechtman. 2018. Deep-storm: super- resolution Langle-molecul microscopy by deep learn- ing. Optica, 5(4):458-464
Venkata Kishore Neppalli, Cornelia Caragea, and Doina Caragea. 2018. Deep Neural Networks versus Naive Bayes Classifiers for Identifying Informative Tweets during Disasters. In Proceedings of the 15th Interna- tional Conference on Information Systems for Crisis Response and Management, ISCRAM'20 2018.
Cinthia Sánchez, Hernan Sarmiento, Andres Abeliuk. Jorge Pérez, and Barbara Poblete. 2023. Cross- lingual and cross-domain crisis classification for low- resource scenarios. In Proceedings of the Interna- tional AAAI Conference on Web and Social Media, volume 17, pages 754-765
Rajvardhan Oak, Min Du, David Yan, Harshvardhan Takawale, and Idan Amit 2019, Malware detection on highly imbalanced data through sequence model- ing. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, AlSec' 19, page 37-48, New York, NY, USA. Association for Com- puting Machinery.
Efsun Sarioglu Kayi, Linyong Nan, Bohan Qu, Mona Diab, and Kathleen McKeown. 2020. Detecting ur- gency status of crisis tweets: A transfer learning approach for low resource languages. In Proceed- ings of the 28th International Conference on Com- putational Linguistics, pages 4693-4703, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Nicolas Ocampo, Elena Cabrio, and Serena Villata. 2023. Playing the Part of the Sharp Bully: Generat- ing Adversarial Examples for Implicit Hate Speech Detection. In.
Anna Schmidt and Michael Wiegand. 2017. A survey hate speech detection using natural language pro- cessing. In Proceedings of the Fifth International Workshop on Natural Language Processing for So- cial Media, pages 1-10. Valencia, Spain. Association for Computational Linguistics.
Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Sarah Vieweg. 2014, Crisislex: A lexicon for collect- ing and filtering microblogged communications in crises. Proceedings of the International AAAI Con- ference on Web and Social Media, 8(1):36--35.
Connor Shorten and Taghi Khoshgoftaar. 2019. A sur- vey on image data augmentation for deep learning. Journal of Big Data, 6.
Alexandra Olteanu, Sarah Vieweg, and Carlos Castillo. 2015. What to expect when the unexpected happens: Social media communications across crises. In Pro- ceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW '15, page 994-1009, New York, NY, USA. Association for Computing Machinery.
Alexander Spangher, Jonathan May, Sz-Rung Shiang, and Lingjia Deng. 2021. Multitask semi-supervised learning for class-imbalanced discourse classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 498- 517, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Xingchao Peng, Judy Hoffman, Stella X. Yu, and Kate Sacnko. 2016. Fine-to-coarse knowledge transfer for low-res image classification. In 2016 IEEE Interna- tional Conference on Image Processing (ICIP), pages 3683-3687.
Qingyu Tan, Ruidan He, Lidong Bing, and Hwee Tou Ng. 2022. Document- level relation extraction with adaptive focal loss and knowledge distillation. In Findings of the Association for Computational Lin- guisticx; ACI. 2022, pages 1672-1681, Dublin, Ire- land. Association for Computational Linguistics.
Dharini Ramachandran and Parvathi Ramasubramanian. 2021. A novel domain and event adaptive tweet augmentation approach for enhancing the classifi- cation of crisis related tweets. Data Knowl, Eng.,
135:101913.
Rui Tian, Zuxuan WU, Qi Dai, Han Hu, Yu Qiao, and Yu-Gang Jiang. 2023. ResFormer: Scaling ViTs with RustiiReeooution Training. In IEEE/CVF Confer- ence on Computer Vision and Putlern Recognition.
Christian Reuter, Amanda Lee Hughes, and Marc-André Kaufhold, 2018. Social media in crisis management:

---

Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS- MT building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479-480, Lisboa, Portugal. European Associa- tion for Machine Translation.
from spatially limited data using enhanced super- resolution generative adversarial network. Physics of Fluids, 33( 2211:222211211 1251 19.
Adams Wei Yu, David Dohan, Minh- Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09: 09541.
Cagri Toraman, Izzet Emre Kucukkaya, Oguzhan Ozce- lik, and Umitcan Sahin. 2023. Tweets under the rub- ble: Detection of messages calling for help in earth- quake disaster. arXiv preprint arXiv: 134033
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems, volume 28. Curran Associates, Inc.
Ruud J.G. van Sloun, Oren Solomon, Matthe Bruce, Zin Z. Khaing, Yonina C. Eldar, and Massimo Mischi. 2019. Deep learning for super-resolution vascular ultrasound imaging. In ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1059.
Wilman W. W. Zou and Pong C. Yuen. 2012. Very low resolution face recognition problem. IEEE Transac- lions on Image Processing, 21(1):327-340.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Bengio. 2019, Manifold mixup: Better rep- resentations by interpolating hidden states, In Pro- ceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Ma- chine Learning Research, pages 6438 03336444. PMLR.
A French Lexicon
Damage: Bilan, Victimes, Répercussion, Dom- mage, Impact, Conséquences, Dégâts, Infrastruc- ture, Humain, Panne, Séquelle, Pertes, Rescapés, Dévastation, Effet, Destruction
Advice Ware_Wnnnn: Précédant, Repère, Chronolo- gie, Précurseur, Préparatifs, Contexte, Situation, Préparation, Prémice, Prévision, Evolution, Plan Social: Réactions, Secours, Reconstruction, Presse, Communiqué, Judiciaire, Controverse, Complo- tiste, Instrumentalisation, Solidarité, Sociale, Aide, Restauration, Hommage, Commémorationn Com- munauté
Congcong Wang, Paul Nulty, and David Lillis. 2021. Transformer- based multi-task learning for disaster tweet categorisation. In International Conference on Information Systems for Crisis Response and Man- agement.
Jason Wei and Kai Zou. 2019. EDA: Easy data augmen- tation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Con- ference Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP- (JCLPP), pages 6382-6385 Hong Kong, China. As- sociation for Computational Linguistics.
B English Lexicon
Damage: Effects, Fatalities, Impact ,Damage, Casualties, Outbreak, Destruction, Victim, Death, Result, Outcome, Rupture, Aftermath, Incident, Losses, Outages, Injuries, Trapped, Body
Wenming Yang, Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, and Qingmin Liao. 2019. Deep learning for single image super-resolution. brief review. Trans. Multi.,
C Scrapper and Automatic Annotation
Wei Ye. Bo Li. Rui Xie, Zhonghao Sheng, Long Chen, and Shikun Zhang. 2019. Exploiting entity BIO tag embeddings and multi-task learning for relation ex- traction with imbalanced data. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages -361- Florence, Italy, Association for Computational Linguistics.
Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Wor Woo Lee, and Woomyoung Park. 2021. GPT3Mix: Lever- aging large-scale language models for text augmen- tation. In Findings of the Association for Computa- tional Linguistics: EMNLP 2021, pages 2225-2239, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Mustafa Yousif, Linqi Yu, and Hee-Chang Lim. 2021. High-fidelity reconstruction of turbulent flow

---

Hurricane
AA 32 languages
Meteorological history
(..)
Preparations
(..)
(mpacts in the United States
The widespread and catastrophic effects Hurricane Harvey resulted one the costlest natural disasters United States history. An estimated 300,000 000 structures and 500,000 vehicles were damaged or destroyed in Texas alone. The storm also spawned 53 foraades across six states. The National Oceanic and Atmospheric
Administration estimated total damage $125 billion, 90% confidence interval $90-160 billion. The foooding in areas with low National Flood Insurance Program (NFIP) participation lends to the large uncertainty damage
Harvey
tropical cyclone
country increases since
alongside Hurricane
However, accounting inflation
Hurricane Center considers Harvey the second costliest [19) Harvey was the costiiest natural disaster recorded Texas at the time, (!] until it was surpassed February 2021 by a severe winter storm that crippted the state's power grid, which was estimated to have cost least $195 billion (2021 USD) damages in Texas.
Satelitee loop of Harvey making landfall Texas August 26
Nationwide, 107 people died storm-related incidents: 103 Texas, Arkansas, Tennessee, and Kentucky. Of the deaths Texas, 68 were from direct effects of Harvey, the highest such number in the state since 1919 (16)
Lexicon for the label DAMAGE:
Legend:
Effects, Fatalities,Impac,
Crisis type in the page title
Damage, Casualties, Outbreak, Destruction, Victim, Death, Result, Outcome, Rupture, Aftermath, Incident, Losses, Outages, Injuries, Trapped, Body
Word in the lexicon and in the section title
Labelled high-resolution paragraph
Example of the automatic annotation of Wikipedia paragraphs (https: htttp:://en.wikiiedia.rro////// Hurricane ca
Figure 2:


Reviews:
[{"review_id": "fS1zecuu3K", "reviewer": "Reviewer_tS2X", "summary": "", "questions": "Question A: In L179, how is the high-resolution text mixed with tweets for training? How in details is the augmentation implemented (e.g., the example shown in Table 2)?\n\nQuestion B: What are the 7 classes of intent in Table 1?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you\ninvested in reviewing our submission\n\nRegarding novelty, compared to existing approaches, ours does not generate any artificial data and relies on external high quality sources. Our results demonstrate  that mixing data with different resolutions works well, confirming that super-resolution, initially designed for image processing, is quite effective for text classification as well.\n\nWe will improve the explanation of the methodology, pointing out its novelty. In addition, we will rewrite the algorithm to be clearer, illustrating it with an example.\n\n\nAnswer Question A :  \nHigh resolution-text and all other data-based data augmentation approaches we used follow the same technique: First, we use a given augmentation method (SR4NLP, generative, ...) to create a new set of data, then, when we split the tweets from the original dataset between training set and test set, we add our augmented data on the training set. Therefore we train on both the original data and the augmented data and only test on the original data.\n\nAnswer Question B : \nThis is indeed a misprint: there are 5 classes (Damage, Social, Advice-Warning, Other message, Not Useful).\n\nWe would like to thank you again for your time and\nconsideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The idea of using high-resolution data to augment low-resolution data is interesting. The experimental results are good.", "reasons_to_reject": "The method outlined in Section 2.2 is not explained clearly, which makes it hard to distinguish how SRL4NLP advances the previous methods and thus seems incremental. It would be better if the authors could expand the explanation of the methodology, clarifying how it works, how it's different from existing methods, and why it's effective. More details about the novelty can also be added, such as the specific challenges of adapting super-resolution learning to text classification tasks and how they overcame them.", "questions_to_authors": "Question A: In L179, how is the high-resolution text mixed with tweets for training? How in details is the augmentation implemented (e.g., the example shown in Table 2)?\n\nQuestion B: What are the 7 classes of intent in Table 1?", "paper_topic_and_main_contributions": "This paper focuses on the problem of imbalanced class distribution and the scarcity of high-quality training data in text classification tasks, particularly for urgent crisis situation tweets. The authors propose to augment short noisy twitter text with the long high-quality wikipedia data. The main contributions of this paper is the proposal of SRL4NLP, a data augmentation method inspired from super-resolution learning in image processing.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yr7KF32CAQ", "reviewer": "Reviewer_p7BT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you\ninvested in reviewing our submission.\nWe will improve the explanation of the methodology and rewrite the algorithm to be clearer, illustrating it with an example. \n\nConcerning the use of LLM, we used ChatGPT as one of the data augmentation methods. \n\nRegarding the classification task, we do not report ChatGPT results for three reasons. First, our aim is to design a simple approach that requires very low computational cost while avoiding manual annotation. Second, as we used ChatGPT to generate the new augmented data and so learned from these data, we decided to not use it for classification because results may be biased. Finally,  we have experimented with ChatGPT on a small subset of the French test data but ChatGPT was unable to distinguish “Other” and “Not useful” messages, achieving therefore low performances compared to FlauBert (e.g. ChatGPT was only able to correctly classify 35% of messages into intention to act categories). We will report these experiments in the camera ready as well.\n\nWe would like to thank you again for your time and\nconsideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) Innovative Data Augmentation Approach: SRL4NLP is an innovative, simple, yet effective adaptation of a data augmentation strategy from image processing applied to text classification, expanding the potential of data augmentation in natural language processing.\n(2) Versatile Performance: The paper presents a study of this strategy's robust performance in disaster management on social media, using English and French benchmark datasets, showcasing the technique's versatility across languages and domains.\n(3) Competitive Out-of-Type Performance: The paper demonstrates the technique's competitive performance in out-of-type situations, where classifiers encounter unseen event types during training, indicating its strong generalization and high performance even in challenging scenarios.", "reasons_to_reject": "(1) Need for Comparative Analysis with Mainstream Models: While the paper presents compelling research, it would be beneficial to include a comparative analysis with existing mainstream large language model or large multimodal models, such as ChatGPT, etc. Such comparisons could enhance the credibility and persuasive power of the study.\n(2) Clarity of Methodological Description: The methodology employed in the paper, while intriguing, could benefit from a more detailed and lucid explanation. The current presentation required multiple readings for comprehension, suggesting room for improvement in readability. Providing additional descriptions and clarifications around the methodology may help readers better understand the approach and its implications.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors present SRL4NLP, an innovative technique for data augmentation in Natural Language Processing (NLP), drawing inspiration from the concept of super-resolution learning prevalent in image processing. Conventionally employed to enhance low-resolution or noisy images, this method has not been hitherto utilized in text analytics. Adapting this approach for text classification, the authors validate its effectiveness on the challenging task of detecting urgency in crisis-related tweets, an area marked by scarcity of data and a considerable imbalance. The study substantiates that SRL4NLP surpasses existing leading-edge data augmentation methodologies on a variety of benchmark datasets, spanning two languages.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zSMhXj5gZE", "reviewer": "Reviewer_SHwF", "summary": "", "questions": "- How are the class keywords decided and how much efforts requires? Any experiments to measure the sensitivity against different keyword selection/combination?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the time and effort you\ninvested in reviewing our submission.\n\n As adapting super resolution from image classification to text classification is new, we wanted to experiment it on a challenging use case, namely crisis management, a highly imbalanced task focusing on  two datasets in French and English. Now we have shown that super-resolution works well, we plan to test it on other tasks as mentioned in the conclusion\n\nRegarding the effort needed to use SRL4NLP, the alignment between long texts from structured documents and short texts is easy when section titles and class labels match (directly or via synonyms). As mentioned in the Limitations section, the only difficulty may be the lack of such high-resolution texts. This is why, as a first step, we used the keyword protocol to gather long text data in crisis management.\n\nKeywords list has also been relatively easy to build as our classes are self explanatory. Indeed, we used a synonym dictionary and arrived at a relatively short list per class (around 15 words per class). We will better explain the keyword selection procedure in the camera-ready paper if accepted. \n\nAbout the use of model-based data augmentation approaches, we experimented with two model-based methods: Manifold mixup which is a text interpolation method and multitask learning, following (Ye et al., 2019). We only report the results obtained by Manifold mixup because this method has already been used in state of the art in crisis management. However we did try Multi-task learning and were still behind SRL4NLP (F1-score of 53.81 for the French corpus vs. 54.19 for SRL4NLP). Due to space limitation, we choose to include only one model-based method, and we choose Manifold Mixup as this method is the most popular in crisis management. We will add the multitask results in the camera ready paper for a better comparison. \n\nWei Ye, Bo Li, Rui Xie, Zhonghao Sheng, Long Chen, and Shikun Zhang. Exploiting entity\nBIO tag embeddings and multi-task learning for relation extraction with imbalanced data.\nIn Proceedings of ACL, pages 1351–1360, 2019\n\nWe would like to thank you again for your time and\nconsideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper proposes a new data augmentation approach SRL4NLP that adapts super-resolution learning from computer vision to NLP in a novel way. It draws an innovative analogy between images/texts and high/low resolution data.\n\n- The paper is clearly structured and well written overall. The motivation and proposed approach are explained clearly. It provides sufficient background on data augmentation in NLP and crisis tweet classification to contextualize their work.\n\n- The author shows statistically significant improvements in F-score over no augmentation and competitive results vs other augmentation techniques. In addition, the experiments covers both English and French, making it more convincing.", "reasons_to_reject": "- While the proposed approach is similar to the super-resolution, its requires more efforts on different tasks, making it less generalizable. The reliance on availability of suitable external high resolution texts as well as a way to align low-resource and high-resource task are both needed.\n\n- While the proposed SRL4NLP is easy to understand, the method may not be fully explored, especially the key-word for each class. In addition, This work did not compare with other model-based augmentation approaches, which may yield better improvement as comparing to SRL4NLP.", "questions_to_authors": "- How are the class keywords decided and how much efforts requires? Any experiments to measure the sensitivity against different keyword selection/combination?", "paper_topic_and_main_contributions": "In this paper, the author proposes SRL4NLP, a new data augmentation approach for imbalanced text classification by drawing an analogy to super-resolution learning in image processing. While super-resolution uses high-resolution images to be mixed with low-resolution images. SRL4NLP uses long, well-written texts from wikipedia as \"high resolution\" to augment short, noisy texts like tweets as \"low resolution\".\n\nThe author evaluates SRL4NLP on urgency detection in crisis tweets, a highly imbalanced task. Additional \"high resolution\" texts are extracted from Wikipedia articles related to disasters. The experiment results on benchmark English and French datasets show SRL4NLP performs competitively compared to state-of-the-art augmentation techniques. SRL4NLP improved F-scores significantly compared to no augmentation, particularly for minority classes.\n\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is about disaster urgency detection. It proposes a data augmentation method inspired by super resolution in image processing. The reviewers highlight the effectiveness of the proposed method (including for unseen disaster events). They also note that the application of super resolution to NLP classification tasks is new and interesting. The reviewers criticize some aspects of novelty, e.g., how the proposed method distinguishes from domain transfer techniques, which parts of the proposed approach are novel in NLP and what the specific challenges of this application were. Two reviewers note that the presentation and formalism requires revision. Finally, this work could benefit from further comparisons to better contextualize the reported results, e.g., by comparing against LLMs or other augmentation methods.
Decision: Accept-Findings