Uni-SMART: Universal Science Multimodal Analysis and Research
  Transformer
Authors: name: Hengxing Cai; name: Xiaochen Cai; name: Shuwen Yang; name: Jiankun Wang; name: Lin Yao; name: Zhifeng Gao; name: Junhan Chang; name: Sihang Li; name: Mingjun Xu; name: Changxin Wang; name: Hongshuai Wang; name: Yongge Li; name: Mujie Lin; name: Yaqi Li; name: Yuqi Yin; name: Linfeng Zhang; name: Guolin Ke
Published: 2024-03-15 13:43:47

Abstract: In scientific research and its application, scientific literature analysis is
crucial as it allows researchers to build on the work of others. However, the
fast growth of scientific knowledge has led to a massive increase in scholarly
articles, making in-depth literature analysis increasingly challenging and
time-consuming. The emergence of Large Language Models (LLMs) has offered a new
way to address this challenge. Known for their strong abilities in summarizing
texts, LLMs are seen as a potential tool to improve the analysis of scientific
literature. However, existing LLMs have their own limits. Scientific literature
often includes a wide range of multimodal elements, such as molecular
structure, tables, and charts, which are hard for text-focused LLMs to
understand and analyze. This issue points to the urgent need for new solutions
that can fully understand and analyze multimodal content in scientific
literature. To answer this demand, we present Uni-SMART (Universal Science
Multimodal Analysis and Research Transformer), an innovative model designed for
in-depth understanding of multimodal scientific literature. Through rigorous
quantitative evaluation across several domains, Uni-SMART demonstrates superior
performance over leading text-focused LLMs. Furthermore, our exploration
extends to practical applications, including patent infringement detection and
nuanced analysis of charts. These applications not only highlight Uni-SMART's
adaptability but also its potential to revolutionize how we interact with
scientific literature.