Multi-Task Learning approach to identify sentences with impact and affected location in a disaster news report (2025)

Abstract: The first priority of action in the Sendai Framework for Disaster Risk Reduction 2015-2030 advocates the understanding of disaster risk by collecting and processing practical information related to disasters. A smart collection may be the compilation of relevant and summarized news articles focused on some key pieces of information such as disaster event type, geographic location(s), and impacts. In this article, a Multi-Task Learning (MTL) based end-to-end model has been developed to perform three related tasks: sentence classification depending on the presence of (1) relevant locations and (2) impact information to generate a summary,and (3) identification of the causes or event types in disaster news. Each of the three tasks is formulated as a multilabel binary classification problem. The results of the proposed MTL model have been compared with three popular transformer models: BERT, RoBERTa, and ALBERT. It is observed that the proposed model showed better performance scores than the other models in most cases.

## introduction the first priority of action of the third united nations (un) world conference on disaster risk reduction (wcdrr) foot_0 advocates disaster risk understanding through the collection and processing of relevant and practical pieces of information. news reports published by reputed sources provide fast and reliable information that can be processed and used to keep track of such events @xcite researchers found that @xcite jointly learning multiple related tasks (multitask learning) benefits the learning of each of them. the knowledge gathered through the training of one task is used in learning others. it helps the model improve its generalization ability for all related tasks and reduces model overfitting on training data @xcite experiments have shown that small language models can summarize well @xcite if used on a specific category. note that, the proposed language model foot_1 is light (around 29 million parameters and 115 mb size), simple, and has been designed for a precise category of documents. this work aims to do three tasks: classification of sentences depending on (1) disaster location and (2) impact information ( the rest of the paper is organized as follows: a literature review is presented in section 2. the methodology is covered in section 3.the training and inference of the proposed model are discussed in section 4 and section 5, respectively. a discussion of the results and analysis of the outputs is given in section 6. finally, the article is concluded in section 7. ## related work the proposed model is designed to generate a disaster news extractive summary with location and impact sentences following the multi-task learning (mtl) approach. the methods in @xcite includes the disaster impacts and causes in the generated abstractive summary. the ner (named entity recognition) @xcite @xcite , machine learning téllez @xcite are applied to extract the disaster impacts from tweet and news texts. there are excellent works that extract salient information from text (not limited to disaster-related reports). the mtl based abstractive summarization methods in @xcite @xcite @xcite jointly learn the target summarization task with other language understanding tasks. interestingly, the extractive methods in @xcite applied the graph attention network (gat) and in qiu et al., 2020 used automatic classification based on geoscience-dictionary attention. the mtl model in mulyar et al., 2021 learns eight tasks on clinical notes and huang et al., 2022 learns four tasks across multiple language datasets. there are mtl models identifying event information lv et al., 2022, summarizing legal documents agarwal et al., 2022, efficiently generating sentence embeddings lamsiyah et al., 2023, and processing conversation song et al., 2023. the authors in aguirre and dredze, 2024 dealt with the performance dispar-3 https://dx.doi.org/10.21227/hsdv-2t76 ity in models on different data subpopulations by transferring demographic fairness transfer among related tasks. the literature shows that the mtl-based approach is highly efficient when employed in closely related tasks. the methods targeting summarization have used various language understanding task(s) as auxiliary. however, an extractive summarization method that learns multiple sentence classification tasks on related topics (impacts and relevant location) is rarely present in the literature. the proposed model does the above and also the relevant event identification task together in an end-to-end model. an ner technique may find "flood" disaster in "...complaints flood t.n. police. . . ". however, the event identifier is intended to find none in it. ## methodology the end-to-end model depicted in figure 1 starts with a token embedding layer followed by a layer encoding the sequence of tokens. the encoded sequence is then sent to the event class decoder for event identification. the encoded token sequence in each sentence is averaged and sent to the attention-based decoders that classify each sentence based on the (impact and relevant location) information it carries. recurrent neural network (rnn) is highly efficient when processing sequential data. the long short-term memory (lstm) neural network @xcite is a category of rnns that efficiently addresses the exploding and vanishing gradient issue of rnn training. in this method, the encoder processes a sequence of sentences in a document and the decoder uses the contextual information from the encoder and produce a sequence of labels. therefore, the lstm units are employed to construct the encoder and decoder structure of the model. ## training all the samples are shuffled to properly mix the contents. then they are divided into the train, validation, and test sets (8:1:1) with 6153, 769, and 770 samples. the distribution of the event type labels has been shown in two different procedures are used to calculate the loss, one for the event identification task and another for the sentence labeling tasks. the loss (l 1 ) of the event identification task is calculated in the following steps. require: v , enchidd, enccell ensure: sentence_labels hidd ← mean(enchidd 1 , enchidd 2 ) ▷ last time step hidden outputs from 2-layer encoder for each i in m do ▷ @xmath0, v j )) ▷ weights[j] ∈ r 1 end for attention ← sof tmax(weights) for each a j and v j in attention and v do attn_applied[j] ← a j × v j ▷ a j -scalar and v j -vector end for attn_input ← relu(f c_apply_attn(attention_applied finally, the loss quantities from the document classification and two sentence classifications are averaged to get the loss of the mtl model (m t l model @xmath1, where event, impact related sentence and location related sentence identification losses are l 1 , l 2 , and l 3 , respectively). at the time of data preparation, after going through the sentence and token frequencies, the sentences per document and tokens per sentence are fixed at m (40) and n (20). there are shorter documents having fewer sentences than m. the variable sc represents sentence count (equation 6) that carries the number of sentences for shorter documents and m for bigger documents. so, the loss calculation in equation 6 is done only on the actual length of the sentence, and it helps to avoid calculating loss for the padded sentence entries. due to its relevance to classification tasks, the f-measure scores are considered for judging the best architecture among the models with 1-layer, 2-layer, and 3-layer lstm encoder and decoder units. each of the above three generated low precision and high recall values, which means a high false-positive ratio. on the basis of the results found in ## inference the trained mtl model and the component classifiers are separately applied to the test dataset with 770 samples. at first, the tokens in a sample are converted into embeddings (ref. section 3.1). then the sequence encoder generates token encodings from the embeddings (ref. section 3.2). the encoded sequence is then passed through the event class decoder to identify the probable events in the sample document (ref. section 3.3). the encoded sequence of tokens is then converted to an encoded sequence of sentences (ref. section 3.4). after that, the encoded sequence of sentences is used in both the attention decoders (ref. section 3.5). finally, the predicted labels for the event classes and the sentence classifications are used to map the actual event names and the actual sentences in the test set. the results found after comparing the predicted and ground truth labels are elaborated in section 6. 6 results and discussion this section shows results obtained after using the custom disaster news dataset to train and test the proposed mtl model, each of the component classifiers (for the ablation study) in the proposed mtl model, and three pretrained popular transformer models: bidirectional encoder representations from transformers (bert) devlin et al., 2019, a lite bert (albert) lan et al., 2019, and robustly optimized bert pretraining approach (roberta) liu et al., 2019. all the above three transformers are pre-trained on bookcorpus and english wikipedia datasets, whilst roberta is further trained on commoncrawl-news, openweb-text, and stories datasets. all these pre-trained transformer models can be fine-tuned for downstream language understanding tasks. in this work, the pre-trained bert-base-uncased, roberta-base, and albert-base-v2 models (from the huggingface library wolf et al., 2020), each with 12 layers, 12 heads, and 768 hidden dimensions (@xmath5, @xmath6, and @xmath7, have been fine-tuned and tested on the event identification and sentence classification tasks. the dice-coefficient (equation 7) finds the overlap or similarity between each pair of values in the 0 to 1 range between two equal-length vectors @xcite a score close to '1' indicates high similarity in them. the event class label of a sample consists of nine bits. the first bit signifies the presence/absence of an event. it is used in the loss calculation of event identification task (ref. section 4) depending on the context of the document it may also identify a crisis event that is not present in the list of events ("fire", or "lightning"). the sentence identification accuracy ( to binary values to calculate accuracy. tables 6 and 8 show the average accuracies. calculated scores in the first and second tables are all above 0.6 and 0.8, which is good. the proposed mtl model has shown a good precision score in event class prediction and moderate scores in the other two. the model has also shown good recall for all three tasks. the bert model for location sentence labeling has shown a little better f1 score than the proposed mtl model. all transformers have shown balanced precision and recall scores. overall, in most of the scores, the proposed mtl model has shown the best performance among all the experimented models. ## conclusion this article introduces an mtl based model that jointly learns identifying (1) disaster event types and the sentences containing the (2) disaster lo-cations and (3) impacts in a disaster news article. the union of the set of extracted sentences forms a summary. eight frequent disaster events are identified from the corpus and used as the target labels. three component classifiers of the proposed mtl model and three transformer models are tested on the same data to compare the performances. the mtl model has performed well in comparison to the component models and transformer models (ref. section 6). hopefully, the model can perform better if it is trained with a larger amount of samples. a relevant dataset in multilingual news would be prepared, and the generalization ability of the proposed model on this dataset would be tested in future.