Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text
  Analytics? An Examination on Several Typical Tasks
Authors: name: Xianzhi Li; name: Xiaodan Zhu; name: Zhiqiang Ma; name: Xiaomo Liu; name: Sameena Shah
Published: 2023-05-10 03:13:54

Abstract: The most recent large language models such as ChatGPT and GPT-4 have garnered
significant attention, as they are capable of generating high-quality responses
to human input. Despite the extensive testing of ChatGPT and GPT-4 on generic
text corpora, showcasing their impressive capabilities, a study focusing on
financial corpora has not been conducted. In this study, we aim to bridge this
gap by examining the potential of ChatGPT and GPT-4 as a solver for typical
financial text analytic problems in the zero-shot or few-shot setting.
Specifically, we assess their capabilities on four representative tasks over
five distinct financial textual datasets. The preliminary study shows that
ChatGPT and GPT-4 struggle on tasks such as financial named entity recognition
(NER) and sentiment analysis, where domain-specific knowledge is required,
while they excel in numerical reasoning tasks. We report both the strengths and
limitations of the current versions of ChatGPT and GPT-4, comparing them to the
state-of-the-art finetuned models as well as pretrained domain-specific
generative models. Our experiments provide qualitative studies, through which
we hope to help understand the capability of the existing models and facilitate
further improvements.