The Search for Conflicts of Interest: Open Information Extraction in Scientific Publications (2025)

Abstract: A conflict of interest (COI) appears when a person or a company has two or more interests that may directly conflict. This happens, for instance, when a scientist whose research is funded by a company audits the same company. For transparency and to avoid undue influence, public repositories of relations of interest are increasingly recommended or mandated in various domains, and can be used to avoid COIs. In this work, we propose an LLM-based open information extraction (OpenIE) framework for extracting financial or other types of interesting relations from scientific text. We target scientific publications in which authors declare funding sources or collaborations in the acknowledgment section, in the metadata, or in the publication, following editors’ requirements. We introduce an extraction methodology and present a knowledge base (KB) with a comprehensive taxonomy of COI centric relations. Finally, we perform a comparative study of disclosures of two journals in the field of toxicology and pharmacology.

## introduction many modern institutions require inputs from experts prior to decisions that affect people's health, finances, rights, etc. for instance, expert input is needed to approve new drugs or medical procedures, to award contracts or public grants, etc. we call expert a person whose input may be called upon for such a decision, or who may actually take the decision. for public trust in the process, experts should not have conflicts of @xcite , that is: they should not stand to gain from a decision that they can impact. to avoid cois, transparency databases are built, cataloging known relations of interest between individuals and organizations, and consulted to avoid soliciting experts with stakes in the decision. for instance, the us sunshine act requires medical doctors to state their relationships, e.g., with drug and medical device manufacturers; the eu database eurosfordocs harvests such information from different eu countries, etc. in this work, we investigate if reliable and valuable information extraction (ie) can be performed from disclosure texts using llms. in the context of scientific articles, the disclosure text can appear in a variety of forms -from an explicit declaration of coi as manuscript metadata to an acknowledgment, or a funding section at the end of the manuscript. we observed that while llm can produce good quality triples as compared to traditional methods, the extracted triples need further refinement for improving the overall quality and, hence, usability of the collection of triples. to this end, we propose an end-to-end pipeline that relies on in-context learning for raw triple extraction and, further, leverages rule-based and clustering-based methods to transform the triples into a meaningful, well-structured knowledge base (kb). our methodology can be replicated to build transparency kbs in a variety of contexts. as a step forward, we applied our methodology to construct a kb capturing relations of interest from journal articles in the field of toxicology and pharmacology. we selected the biomedical domain due to large impact on the public's health, as witnessed for instance by the "forever pollution" project undertaken by journalists from 16 countries, and motivated by prior research highlighting possible real-world harm resulting from cois in this domain (see section 2). inspired by prior journalistic work which identified experts involved in cois with the food industry, we selected: a journal in which these experts published: "regulatory toxicology and pharmacology" (rtp, in short), and, as a basis for comparison, a second one from the same scientific area, namely "toxicology and ap-plied pharmacology" (tap, in short). our analysis highlights very strong involvement of the industry in one scientific journal, in striking contrast with the other. to summarize, our contributions are as follows: our code and kb are publicly available 1 . ## related work open information extraction open information extraction (oie) seeks to identify the subject, relation, and object of a sentence in a domainindependent manner, unlike closed-domain information extraction, where the extracted information should belong to known domains. a comprehensive review of open information extraction techniques is @xcite . oie has evolved significantly over the years -from rulebased or traditional machine learning approaches @xcite , to neural approaches @xcite @xcite , to recent approaches based on large language models (llms) @xcite . @xcite ; @xcite formulated oie as a sequence-labelling problem, while @xcite ; @xcite cast it as a sequence to sequence generation task. this was further extended to handle named entities by 1 https://gitlab.inria.fr/cedar/coi-openie implementing soft constraints during training and inference @xcite . the emergence of powerful llms (gpt-4 (ope-nai et al., 2024), llama-3 @xcite ) highlight the promise of llms for oie, either through fine-tuning @xcite or through few-shot prompting @xcite . our work extends this line by showing that few-shot prompting alone can be competitive not only for oie but also for relation normalization. at the same time, we demonstrate that purely generation-based methods are sometimes brittle, and that lightweight linguistically motivated preprocessing rules remain useful to enforce prompt adherence. ## extracting relations of interest from disclosure text extracting relations of interest from the disclosure statements involves acquiring and cleaning the data (section 3.1), followed by the extraction of the relations of interest (section 3.2), and finally the creation of a knowledge base (section 3.3). we present an overview of our approach in figure 1 . ## experiments we now describe experiments validating the methodology presented in section 3. dataset from the rtp, respectively, tap journals we retrieved the entire publication set from 1993 to 2022. ber of published articles (papers); number of articles with explicit coi metadata (coi); among these, articles with actual information in that field as opposed to stating "we have nothing to disclose" (∃coi); number of papers containing phrases of interest (phrases) ; number of triples we extracted from paper (#t_paper), respectively, from explicit coi statements (#t_coi). only about 1%of papers contain useful coi metadata, whereas roughly 99% include relevant phrases in the pdf. although this may be slightly overestimated (≈ 10% error; see limitations), such phrases remain far more frequent, underscoring the value of harvesting them in addition to coi metadata, unlike prior studies @xcite . models we test the gemma 2 models @xcite of 2b, 9b, and 27b and gpt, the gpt-3.5-turbo-0125 version. we run the models with temperature of 0. gemma 2 27b is run in 8 bit precision and the other models in full precision. we ran the gemma models on a v100 32g gpu and the gpt model using the openai api. ## comparative analysis of the journals in this section, we investigate if our extracted triples of relations of interest can serve as useful input for computational social science investigations of publishing practices. ## conclusion we have presented an oie approach, based on llm prompting, clustering, and external knowledge bases, for extracting coi triples from disclosure statements. we propose a new kg, and a set of 300+ fine-granularity relationship predicates. finally, we show a vastly stronger industry presence in a journal than in another one on the same topics. our resources should enable more social science studies on this important topic.