Do Numbers Matter? Types and Prevalence of Numbers in Clinical Texts (2024)

Abstract: In this short position paper, we highlight the importance of numbers in clinical text. We first present a taxonomy of number variants. We then perform corpus analysis to analyze characteristics of number use in several clinical corpora. Based on our findings of extensive use of numbers, and limited understanding of the impact of numbers on clinical NLP tasks, we identify the need for a public benchmark that will support investigation of numerical processing tasks for the clinical domain.

## introduction numbers comprise a considerable amount of textual content and contribute substantially to conveying meaning in a range of domains including financial and scientific contexts. targeted strategies for representing numbers have been shown to improve general literacy of language models @xcite . numbers pose challenges for natural language processing (nlp) due to their varied representations in text, as digits, words, or numerical expressions. this requires nlp models to handle ambiguity and context effectively in interpreting numerical information @xcite . numerical reasoning is crucial in the generative large language model (llm) era because it underpins data-driven decision-making in many fields, including clinical, where accurate numerical insights are essential. llms often struggle with arithmetic operations and unit conversions, impacting their reliability in quantitative tasks. in the clinical domain, accurate numerical reasoning is vital for analyzing trial data, interpreting results, and making precise treatment recommendations, such as determining appropriate drug dosages based on statistical analyses of patient outcomes. for example, generating a report that states "the mean number of antihypertensive medication classes increased from 1.6 (95% ci, 1.4-1.8) at baseline to 2.2 (95% ci 2.0-2.4) at 6 months" foot_0 requires precise numerical reasoning. in this paper, we characterize the numerical information in clinical nlp corpora. through corpus analysis, we find that numerical information is frequent, but only a small portion is annotated or utilized. our analysis identifies a number of issues concerning numeracy that need further attention from the clinical nlp research community. ## numerical strings and types there is an assumption that numbers may be trivially extracted from text, as they consist of digit sequences or a finite set of numerals. however, numerical information can appear in various lexical surface and semantic contexts @xcite . we identify a multitude of number variants, and further define semantic categories for numerical information. numerical values can be expressed in many forms: (i) digit, including integer ('3', '100,000'), float ('0.5'), and negative ('-1'), (ii) number with unit ('100 mg', '160/90mmhg'), (iii) fraction, written using the division symbol ('5/32') or with a special symbol (½), (iv) number range ('from 0 to 2 years', '1969-77'), (v) numeral, can be alphabetic numbers ('twenty-five', 'two') or combinations of numbers and words (' ## corpus analysis clinical documents are rich with diverse numerical data, expressed in different manners and contexts. ## discussion there are several possible directions to progress treatment of numbers in clinical nlp research. need for benchmarks while there has been some research on extraction of numerical information from different medical data, most work has used their own data and the gold standard has not always been made public. this limitation has made it impossible to compare model performances of different systems @xcite . the performance reported in several past works was very high (@xmath0. this raises the question of whether numerical information extraction is a solved task. we raise two concerns. first, a number of works utilized relatively small data and this may result in the reported accuracy scores lacking statistical significance. second, some works applied 'easy' task formulations, i.e., given a sentence containing only a single number mention, it is trivial to extract most numerical attributes. such spurious patterns in evaluation data may not generalize when we deal with more realistic scenarios for information extraction @xcite . for example, a sentence containing multiple numbers and more than one candidate for entities and attributes (see example of ebm-nlp instance in scope of numerical reasoning recent works on numerical reasoning deal with math and arithmetic problems @xcite @xcite . in fact, complex mathematics are less applicable in the clinical domain. in addition to arithmetic, other types of reasoning are required for clinical decision support. for example, number comparison @xcite and number normalization @xcite . different units of measurement often need conversion, requiring precise calculations to maintain accuracy. for example, hormone levels may be measured using nmol/l, ng/dl and ng/ml in different trials. dealing with various number representation is important to interpret numerical information correctly. on the other hand, contextualizing such numbers with medical background knowledge is another important numeracy skill, as showcased by the example of mednli instance in tokenization challenges how to encode numbers in language models has been discussed in several works @xcite @xcite . encoding numbers is related to the problem of tokenization @xcite . the models struggle to recognize extrapolated numbers that are seldom found in corpora and force them to be tokenized digit by digit @xcite . in the clinical context, when the numbers are grounded with units or appear as ranges, the tokenizer is expected to be more robust. we inspected few samples of tokenlevel annotated data of the ebm-nlp corpus. our finding was that numbers are not fully correctly tokenized (e.g., "95% ci 0.16-1.17" is segmented into multiple individual digit numbers that lack meaning), even in the gold standard. ## conclusions we analyzed well-established clinical nlp corpora, covering a variety of tasks and data sources, and identifying a broad set of types and usage of numbers. our analysis shows that numbers play a major role in medical texts. on the basis of these findings, and the lack of systematic resources in the clinical domain for investigating numerical information extraction and reasoning tasks, we argue for the need for the construction of such resources. numbers contain vital medical information. we strongly encourage clinical nlp researchers to consider how numerical processing may interact with their work.