TDAC , The First Corpus in Time-Domain Astrophysics: Analysis and First Experiments on Named Entity Recognition (2022)

Abstract: The increased interest in time-domain astronomy over the last decades has resulted in a substantial increase in observation reports publication leading to a saturation of how astrophysicists read, analyze and classify information. Due to the short life span of the detected astronomical events, the information related to the characterization of new phenomena has to be communicated and analyzed very rapidly to allow other observatories to react and conduct their follow-up observations. This paper introduces TDAC: the first Corpus in Time-Domain Astrophysics, based on observation reports. We also present the NLP experiments we made for named entity recognition based on annotations we made and annotations from the WIESP NLP Challenge.

## introduction time-domain astrophysics consists in observing and studying transient cosmic phenomena, i.e. unpredictable, short-lived, and the most violent phenomena occurring in the universe, such as supernovae explosions or gamma-ray bursts (grbs), which are highly energetic explosions lasting from milliseconds to a few hours or days only @xcite . the short life span of these events requires a rapid sharing, analysis and synthesis of the information disseminated in observation reports. however, the increased interest in time-domain astronomy has led to a significant increase in observation reports, leading to a saturation of how astrophysicists analyze and classify information in observation reports. as the current manual reading and analyzing of available information is approaching saturation, new ways of handling information are necessary. one of the most promising approaches is to build natural language processing (nlp) methods that tackle the challenges of extracting and summarizing information on observation reports by detecting, for example, named entities. named entity recognition (ner) can identify and extract information about an astrophysical object, such as the date of detection, its coordinates in the universe, and numerous information, such as intensity and magnitude, to let astrophysicists know if they can trigger a follow-up observation. to train and evaluate an ner system, a corpus must first be created and annotated. this paper presents tdac: a time-domain astrophysics corpus for nlp, based on observation reports. to our knowledge, no existing resources and studies so far are based on time-domain astrophysics observation reports, and therefore there are no studies characterising the discourse used in astrophysics. our objective is twofold: the first objective of our study, with the creation of this corpus, is to highlight differences between astrophysics corpora. what are their properties, and are they all the same? we provide some elements characterizing and revealing the specificity of the formulations used in astrophysics by conducting a corpus analysis (section 4). secondly, we started building an ner system for the domain. section 5 presents our annotations and the first ner experiments we conducted on a sub-corpus of tdac (75 documents). the annotated section of tdac is the first annotated and publicly available foot_0 corpus based on observation reports for named entity recognition in time-domain astrophysics. ## research and language resources in astrophysics the vast majority of the limited research performed so far in nlp for astrophysics studies papers from the astrophysics data system (ads foot_1 ). the ads is a database for researchers in astronomy with more than 15 million records covering publications in astronomy, astrophysics, and general physics. abstracts and full text of astronomy paper publications are indexed and searchable through ads, making it a rich exploitable platform for creating nlp resources. ## discussion and outlook experiment 1 is not comparable to experiments 2 and 3 because the test sample size is not the same. however, it allows us to first appreciate the baseline model's robustness by testing it on our tdac corpus. when tested on the tdac corpus, we noticed a considerable drop in performance (a loss of 0.2166 on the f1 score globally). the results may appear low or moderate. this could be explained by a strict evaluation (identical label and border). with experiments 2 and 3, we notice relatively similar results. overall, the model fine-tuned from scratch performs slightly worse than the baseline model for which we continued the fine-tuning over nine additional epochs. experiment 3 shows that the system performs better on the atels when fine-tuning from scratch. these preliminary results on this first small annotated corpus nevertheless show that the deal corpus is a good starting point for building an entity detection system and can be adapted to other types of documents in the astrophysical domain. however, it is necessary to analyze whether this behaviour is confirmed on a larger scale. while the first annotations have been made by a phd student with a background in astrophysics in order to make a proof-of-concept, we are now experiencing new annotations made by two senior experts, one in astrophysics, the other in nlp. joining the two corpora (deal+tdac) would be complementary because of the distribution of classes in the two corpora (figure 2 ). we observe that certain classes of entities are more present in the tdac corpus than in deal (e.g. formula, celestialobject, observatory, or celestialregion). the tdac corpus thus makes it possible to fill the lack of specific classes and vice versa. therefore, joining these two corpora would thus allow for building a more efficient system for a more significant number of classes. ## conclusion in this paper, we presented the tdac corpus, composed of astrophysics textual content from three sources (atel, gcn circulars, and astronotes). our corpus has been manually annoted in named entity, based on the annotation schema used in the deal corpus. we also presented the experiments we made in order to make it easier the manual annotation process, using a scibert-based model fine-tuned on the wiesp 2022 nlp challenge. we observed that a model trained on the deal corpus is not sufficient since it obtained moderate results, while a quite light fine-tuning (9 additional epochs) on our tdac corpus allows us to improve the performances of our ner system. in the future, we plan to enrich the corpus with morpho-syntactic annotations and relations between named entities. we estimate this corpus would be a useful resource for nlp applications in astrophysics. once the information extraction system we are developing is considered reliable enough, we aim to deploy them in astro-colibri, a real-time platform that evaluates alerts sent by observers regarding transient sources @xcite . the deployment of our nlp models in astro-colibri will allow both professional and amateur astronomers to access the most relevant information disseminated through gcn circulars, atels and astronotes instantaneously.