A Dataset for N-ary Relation Extraction of Drug Combinations (2022)

Abstract: Combination therapies have become the standard of care for diseases such as cancer, tuberculosis, malaria and HIV. However, the combinatorial set of available multi-drug treatments creates a challenge in identifying effective combination therapies available in a situation. To assist medical professionals in identifying beneficial drug-combinations, we construct an expert-annotated dataset for extracting information about the efficacy of drug combinations from the scientific literature. Beyond its practical utility, the dataset also presents a unique NLP challenge, as the first relation extraction dataset consisting of variable-length relations. Furthermore, the relations in this dataset predominantly require language understanding beyond the sentence level, adding to the challenge of this task. We provide a promising baseline model and identify clear areas for further improvement. We release our dataset ( https://huggingface.co/datasets/allenai/drug-combo-extraction ), code ( https://github.com/allenai/drug-combo-extraction ) and baseline models ( https://huggingface.co/allenai/drug-combo-classifier-pubmedbert-dapt ) publicly to encourage the NLP community to participate in this task.

## introduction indeed, combining two or more drugs together has proven to be useful for treatments of various medical conditions, including cancer @xcite @xcite , aids @xcite , malaria @xcite , tuberculosis @xcite , hypertension @xcite and covid-19 @xcite . in this work, we examine the clinically significant and challenging nlp task of extracting known drug combinations from the scientific literature. we present an expert-annotated dataset and baseline models for this new task. our dataset contains 1600 manually annotated abstracts, each mentioning between 2 and 15 drugs. 840 of these abstracts describe one or more positive drug combinations, varying in size from 2 to 11 drugs. the remaining 760 abstracts either contain mentions of drugs not used in combination, or discuss combinations of drugs that do not give a combined positive effect. for the clinical setting, solving the drug combination identification task can help researchers suggest and validate complex treatment plans. for example, when searching for effective treatments for cancer, knowing which drugs interact synergistically with a first line treatment allows researchers to suggest new treatment plans that can subsequently be validated in-vivo and become a standard protocol @xcite @xcite . from an nlp perspective, the drug combination identification task and dataset pushes the boundaries of relation extraction (re) research, by introducing a relation extraction task with several challenging characteristics: variable-length n-ary relations most work on relation extraction is centered on binary relations (e.g. @xcite , see full listing in §5), or on n-ary relations with a fixed n (e.g. @xcite ). in contrast, the drug combination task involves variable-length n-ary relations: different passages discuss combinations of different numbers of drugs. for each subset of drugs mentioned in a passage, the model must predict if they are used together in a combination therapy and whether this drug combination is effective. "we tried adding nifedipine , as labetalol combined to prazosin did not reduce blood pressure. other_comb pos_comb indeed, the addition produced a marked decrease in blood pressure. no reduction of urinary na excretion was observed in our patient during the addition of the nifedipine therapy, suggesting that the decrease in blood pressure was not caused by suppression of na release from pheochromocytoma tissue." "in thailand , artesunate and artemether are the mainly used antimalarials for treatment of no_comb severe or multidrug resistant falciparum malaria ." no type hints as noted by @xcite and @xcite , in many relation extraction benchmarks @xcite @xcite , the argument types serve as an effective clue. however, argument types do not apply naturally to the drug combination task, in which all possible relation arguments are entities of the same type (drugs) and we need to identify specific subsets of them. ## the drug combinations dataset a set of drugs in a biomedical abstract are classified to one of the following labels: positive combination (pos_comb): the sentence indicates the drugs are used in combination, and the passage suggests that the combination has additive, synergistic, or otherwise beneficial effects which warrant further study. ## related work the ddi dataset @xcite is the only work to our knowledge that annotates drug interactions for text mining. however, it fundamentally differs from our dataset in the type of annotations provided: the ddi annotates the type of discourse context in which a drug combination is mentioned, without providing explicit information about combination efficacy. in contrast, our dataset is focused on semantically classifying the efficacy of drug combinations as stated in text. other re datasets exist in the biomedical field @xcite @xcite , but do not focus on drug combinations. similarly, several re datasets tackle the n -arity problem in the scientific domain @xcite @xcite , and in the non-scientific domain @xcite , however, all of them consider a fixed choice of n . ## conclusions we present a new resource for drug combination and efficacy identification. we establish baseline models that achieve promising results but reveal clear areas for improvement. beyond the immediate, application-ready value of this task, this task poses unique relation extraction challenges as the first dataset containing variable-arity relations. we also highlight challenges with documentlevel representation learning and incorporating domain knowledge. we encourage others to partici-pate in this task. our processed dataset foot_7 and best baseline model foot_8 are available on hugging face, and our model training code is available to the public at https://github.com/allenai/ drug-combo-extraction . to the interaction-labeling phase and treat them the same as the other drugs, but, it is not required to mark a word as drug in order to use it in an interaction. if a drug is marked in a wrong manner they should try and fix it, e.g. the span of the drug is incorrect. in order to achieve more consistent and accurate annotations, they are also instructed to annotate all the interactions that they can find in a given sentence. they should always use the accept button even if there are no interactions in the sentence. only in cases where they want to skip a sentence (e.g. when there is an inherent problem with it) or leave it for a future discussion they should use the ignore button. an interaction can occur between more than two drugs, if so they should notice that they don't need each pair from this group to have a marked interaction, as long as they all connect to the same graph. e.g. "drugs a, b and c are synergistic." connecting a to b and b to c is sufficient, no need to connect drug a to drug c. each interaction should be marked with a different tag (pos_comb1, pos_comb2..., other_comb1, other_comb2...).