LLMFactor: Extracting Profitable Factors through Prompts for Explainable Stock Movement Prediction (2024)

Abstract: Recently, Large Language Models (LLMs) have attracted significant attention for their exceptional performance across a broad range of tasks, particularly in text analysis. However, the finance sector presents a distinct challenge due to its dependence on time-series data for complex forecasting tasks. In this study, we introduce a novel framework called LLMFactor, which employs Sequential Knowledge-Guided Prompting (SKGP) to identify factors that influence stock movements using LLMs. Unlike previous methods that relied on keyphrases or sentiment analysis, this approach focuses on extracting factors more directly related to stock market dynamics, providing clear explanations for complex temporal changes. Our framework directs the LLMs to create background knowledge through a fill-in-the-blank strategy and then discerns potential factors affecting stock prices from related news. Guided by background knowledge and identified factors, we leverage historical stock prices in textual format to predict stock movement. An extensive evaluation of the LLMFactor framework across four benchmark datasets from both the U.S. and Chinese stock markets demonstrates its superiority over existing state-of-the-art methods and its effectiveness in financial time-series forecasting.

heading: 1 Introduction
text: Artificial Intelligence (AI) has become a staple in the financial sector, addressing various challenges such as predicting stock movements (Patel et al., 2015; Henrique et al., 2019), providing roboadvisory services (Xue et al., 2018; Bertrand et al., 2023), and managing risks (Ahbali et al., 2022; Wang et al., 2021). Among these tasks, forecasting stock trends is particularly significant, as it leverages historical data to shape trading strategies and pinpoint opportunities for buying or selling stocks.
*Corresponding Author
The Efficient Market Hypothesis (EMH), proposed by Eugene Fama (Fama, 1970), suggests that stock prices reflect all available information, making it difficult to predict future price movements. However, subsequent research has identified limitations to market efficiency, highlighting how phenomena such as information asymmetry (Timmermann and Granger, 2004) and irrational behaviours (Lo, 2004) can induce departures from perfect efficiency. These observations have paved the way for researchers to seek excess market returns by identifying and leveraging market inefficiencies.
Building on this foundation, there is a growing interest in exploring various data types to enhance prediction capabilities. Several studies underscore the importance of news related to stocks in uncovering fundamental market insights (Vargas et al., 2018; Li et al., 2021), while others highlight the significance of understanding the interconnections between companies and industries (Feng et al., 2019; Hsu et al., 2021). Recent research has empirically demonstrated the impact of public sentiment on market trends, with efforts to extract sentiment and keyphrases from news and social media data (Nguyen et al., 2015; Hao et al., 2021).
However, as illustrated in Figure 1, these meth-
3120
ods encounter various limitations. We propose a novel task to enhance stock movement prediction through the use of “factors.” Figure 1 delineates the distinctions among keyphrases, sentiment words, and factors generated by LLMs. For the given stock-related news, keyphrases may outline the content but do not directly correlate with the stock price. Conversely, sentiments are associated with stock price but lack clarity for human interpretation. Consequently, factors offer superiority in three aspects: news explainability, human readability, and stock price explainability.
To incorporate factors into financial forecasting, we introduce a new framework LLMFactor, which derives factors from LLMs through Sequential Knowledge-Guided Prompting (SKGP) and then explains stock price trends. LLMFactor represents a holistic approach to integrating LLMs into financial applications. Initially, our proposed SKGP strategy prompts LLMs to generate factors related to stocks. SKGP starts with a fill-in-theblank technique to elicit background knowledge (e.g. company relations) related to the stock. Subsequently, we prompt LLMs to identify reliable factors from news articles. Finally, we compile the text-formatted time-series data with these factors and knowledge to predict stock price trends. Experiments across four datasets indicate that LLMFactor yields superior predictive results and can clarify the principles behind these predictions. Additionally, the obtained factors significantly help to dynamically present market changes over time in a format that is accessible and understandable to humans.
The main contributions of this study are summarized as follows:
• New Task - Factor Extraction: We introduce a novel task, factor extraction, aimed at extracting significant factors from textual data to assist in predicting time-series data. This task surpasses traditional keyphrase-based and sentiment-based methods in three key areas: the explainability of news content, the readability for humans, and the ability to explain stock price movements. • New Strategy - SKGP: We propose Sequential Knowledge-Guided Prompting (SKGP) as an innovative strategy for leveraging background knowledge in the process of stock movement prediction. Unlike basic prompting methods, SKGP employs a fill-in-the-blank approach, utilizing minimal background knowledge to enhance the
richness of the prompt template. • New Framework - LLMFactor: The LLMFac-
tor framework, developed in this study, utilizes factors derived from LLMs to elucidate the dynamic temporal changes. This framework has proven to be a valuable tool in financial applications, providing deep insights into market trends and facilitating factor analysis.

heading: 2 Related Work

heading: 2.1 Stock Movement Prediction using Textual Data
text: With the advancement of natural language processing (NLP) techniques, many researchers leverage textual data to forecast stock market trends. (Xu and Cohen, 2018) uses tweets and historical prices to make temporally dependent predictions from stock data. (Luo et al., 2023) models the multimodality between financial text data and causalityenhanced stock correlations. These efforts aim to develop sophisticated models to enhance the prediction accuracy and consider the textual data as a whole.
Furthermore, certain studies focus on extracting more granular insights from textual data. (Zhou et al., 2021) identifies corporate events as the driving force behind stock movements while (Wan et al., 2021) demonstrates the significant association between strong media sentiment and market return. (Wang et al., 2020) enhances stock movement prediction by including expert opinions aggregated from various sources. These insights offer greater clarity to the understanding of market dynamics.

heading: 2.2 Time-series Forecasting with LLMs
text: LLMs, such as variants of the Generative Pretrained Transformer (GPT), are constructed from wide-ranging and diverse datasets, significantly enriching their extensive knowledge base (Brown et al., 2020). Yet, their foundational structure, grounded in the transformer architecture, is not intrinsically designed for analyzing time-series data (Gruver et al., 2023). This limitation has sparked increased interest among scholars in exploring methods to tailor LLMs for proficient time-series prediction.
(Xue and Salim, 2023) proposes a prompt-based approach to transform numerical inputs and outputs into textual prompts, framing the forecasting task as a sentence-to-sentence conversion. This method allows language models to be directly applied to
forecasting tasks. For financial time series forecasting, (Yu et al., 2023) employs prompts to generate summaries and keyphrases from LLMs, incorporating multiple data sources to enhance time-series forecasting. Their approaches offer valuable insights into stock market trends. However, their prompts contain excessive information, resulting in responses from LLMs that are relatively lacking in detail.

heading: 2.3 Prompt Engineering
text: Prompt engineering allows LLMs to address a variety of tasks effectively through carefully crafted prompts. However, identifying the optimal prompt for each specific task is a challenge (Liu et al., 2023). This challenge leads to the exploration of diverse prompting strategies. For instance, (Wei et al., 2022) introduces a chain of thought (CoT) approach, which involves a sequence of intermediate reasoning steps to enhance the capability of LLMs in complex reasoning tasks. Meanwhile, (Liu et al., 2022) proposes a generated knowledge prompting technique, which involves generating knowledge through a language model and then using this generated knowledge as supplementary input for answering questions. Another popular technique, known as Retrieval-Augmented Generation (RAG) (Lewis et al., 2020), incorporates external knowledge into LLMs through retrieve-augment-generate flows. Inspired by these techniques, we propose a novel Sequential Knowledge-Guided Prompting, designed
to enhance the accuracy of financial forecasts.

heading: 3 LLMFactor
text: The proposed LLMFactor is a comprehensive framework to predict and explain stock market trends as shown in Figure 2.

heading: 3.1 Task Definition
text: For a given stock, denoted as stocktarget, we consider its associated news published on the target prediction date datetarget, referred to as newstarget, and its historical stock price sequence P = {P1, P2, ..., Pt}, where t represents the window size. The task of predicting stock movement is formulated as a binary classification problem, where the stock price sequence is transformed into a series of stock movements, P̂ = {P̂1, P̂2, ..., P̂t}. In this series, P̂i = 1 indicates a rise in the stock price from Pi−1 to Pi, whereas P̂i = 0 indicates a fall. Our objective is to predict P̂t+1 given datetarget, newstarget, and P̂ .

heading: 3.2 Sequential Knowledge-Guided Prompting

heading: 3.2.1 Matching and Acquiring News Background Knowledge
text: The foundation of our approach is the Sequential Knowledge-Guided Prompting (SKGP) strategy, which comprises three primary stages. Consider Nvidia (NVDA) as our example stocktarget, the initial phase involves matching the stock with relevant news and acquiring background knowledge.
Let S = {(Ci, Ti, Ii) | i ∈ N, i ≤ n} be the stock list, where each tuple (Ci, Ti, Ii) consists of a company Ci, its ticker symbol Ti, and its industry Ii. These tuples are collected from the NYSE and NASDAQ exchanges, with n being a positive integer representing the total number of such tuples. We match S with newstarget to obtain stockmatch = {S ∩ newstarget}. Following this, we prompt LLMs to obtain relations between stocktarget and stockmatch. The prompting method is defined as LLM : String → String, where the input is a RelationTemplate “Please fill in the blank: stocktarget and stockmatch are most likely in a ___ relationship,” and the output is the type of relationship, denoted as relation.
This method of deriving background knowledge about newstarget significantly improves our understanding of the news content. For example, if stocktarget and stockmatch are identified as competitors, information about stockmatch could potentially have a negative impact on stocktarget. Furthermore, numerous studies have underscored the critical role of company relationships in predicting stock market movements (Hsu et al., 2021; Ang and Lim, 2022). Therefore, our fill-in-the-blank technique aims to constrain the response format, facilitating a direct and unambiguous identification of the relationship.

heading: 3.2.2 Generating Factors that May Affect Stock Prices
text: The next step for SKGP involves generating factors from newstarget. The importance of these factors is threefold: 1) They are more closely associated with stock movements than keyphrases, sentiments, news summaries, or entire news articles, thus providing a higher likelihood of profitable market trend predictions. 2) Factors derived from news texts offer more immediate and detailed insights into stock price fluctuations compared to those obtained from other sources. 3) They improve the explainability of stock price trends and the rationale behind LLMs’ predictions.
To generate reliable factors, we instruct LLMs to analyze the news content and identify factors that could influence the stock price. This approach fully utilizes the intrinsic knowledge of LLMs. Considering the top k factors we aim to extract for the given stocktarget and newstarget, the prompting method is described as LLM(FactorTemplate) = factor, where FactorTemplate is a structured sentence: “Please
extract the top k factors that may affect the stock price of stocktarget from the following news,” followed by newstarget, and the output is the factors generated by the LLMs. The factors produced by LLMs are not restricted to the words found in the news; instead, LLMs consider the content of the news and its potential impact on stock movement, often summarizing the significant elements in the content. For example, the factor “Nvidia stock gain in January” mentioned in Figure 2 suggests that the event was Nvidia’s past stock performance, and the impact was a gain, which could positively influence Nvidia’s future stock price.

heading: 3.2.3 Predicting Stock Price Movements
text: To predict stock movement, we integrate news background knowledge and factors to guide LLMs. Meanwhile, we transform time-series data into a textual format for LLMs to understand. In a sequence of stock movements denoted by P̂ = {P̂1, P̂2, ..., P̂t}, we introduce a function where f(P̂i = 0) is assigned the value “fell” and f(P̂i = 1) is assigned “rose,” thereby converting P̂ into a sequence of outcomes TextMovement = {f(P̂i) | i ∈ {1, 2, . . . , t}, f(P̂i) = “fell” if P̂i = 0 and “rose” if P̂i = 1}. Given the textual stock movement sequence TextMovement and its date series date = {date1, date2, ..., datet}, the past stock price movements are transformed into a TimeTemplate, structured as “On datei, the stock price of stocktarget f(P̂i).”
Subsequently, we construct a PriceTemplate that includes an initial instruction, “Based on the following information, please judge the direction of the stock price as rise or fall, fill in the blank and give reasons,” followed by a concluding instruction, “On datei, the stock price of stocktarget will ___.” By integrating the relation, factor, TimeTemplate, and PriceTemplate, we articulate the prompting method as LLM(relation, factor, T imeTemplate, PriceTemplate) = prediction. The prediction outcome specifies whether the stock price will “rise” or “fall,” along with the rationale for this inference. More details of templates are in Appendix A.

heading: 3.3 Factor Analysis in the Stock Market
text: The SKGP presents a powerful technique for predicting stock movements, and the factors derived from SKGP offer additional insights into stock market trends. For instance, as illustrated in Figure 2,
factor analysis can be applied to the stock market. Taking Nvidia’s stock price trend as an example, after a consistent increase over the past five days, the day highlighted with a blue box also exhibits an upward movement. To explain this occurrence, our LLMFactor identifies a concise set of factors, such as “Nvidia stock gain in January, new product announcements, and selection of Nvidia Drive Thor by EV makers.”

heading: 4 Experiments
text: By conducting experiments on benchmark datasets, we aim to address the following research question: Does LLMFactor offer improved prediction accuracy and more insightful explanations for stock market movements?

heading: 4.1 Datasets
text: We evaluate LLMFactor on four benchmark datasets, in alignment with the current SOTAs: a) StockNet (Xu and Cohen, 2018) includes 87 stocks from 9 industries, accompanied by stock-related tweets and historical price data from 2014-01-01 to 2016-01-01, in the US stock market; b) CMINUS (Luo et al., 2023) comprises the top 110 stocks along with their tweets and historical price data from 2018-01-01 to 2021-12-31, in the US stock market; c) CMIN-CN (Luo et al., 2023) consists of the 300 stocks in the CSI300 index 1, with their tweets and historical price data from 2018-01-01 to 2021-12-31, in the Chinese stock market; d) EDT (Zhou et al., 2021) includes 54,080 news articles from 2020-03-01 to 2021-05-06, along with related stock and stock price information in the US stock market. Unlike StockNet, CMIN-US, and CMIN-CN, which focus on time-series forecasting, the EDT dataset is centred on news content. The
1A market capitalization-weighted index crafted to mirror the performance of the leading 300 stocks traded on the Shanghai Stock Exchange and the Shenzhen Stock Exchange.
statistics of all datasets are summarized in Table 1.

heading: 4.2 Evaluation Metrics
text: Following previous studies by (Xu and Cohen, 2018; Luo et al., 2023; Zhou et al., 2021), we apply Accuracy (ACC) and Matthews Correlation Coefficient (MCC) as the evaluation metrics. Given the confusion matrix [ tp fp fn tn ] , ACC and MCC are defined as: ACC = tp+ tn
tp+ tn+ fp+ fn (1)
MCC = tp× tn− fp× fn√
(tp+ fp)(tp+ fn)(tn+ fp)(tn+ fn) (2)

heading: 4.3 Baselines
text: Keyphrase-based. We employ several models to identify keyphrases in text for prediction purposes:
• PromptRank(Kong et al., 2023): An unsupervised approach that utilizes pre-trained language models (PLMs) with a focus on leveraging prompt-based techniques for keyphrase extraction. • KeyBERT(Grootendorst, 2020): Leverages BERT embeddings to identify key sub-phrases in documents through cosine similarity, emphasizing the most representative phrases. • YAKE(Campos et al., 2020): A lightweight unsupervised method for keyword extraction, employing statistical features from individual documents to pinpoint the most pertinent keywords. • TextRank(Mihalcea and Tarau, 2004): An unsupervised approach identifies keyphrases based on the co-occurrence graph of words in a text. • TopicRank(Bougouin et al., 2013): Groups similar expressions into topics and ranks these topics to identify keyphrases. • SingleRank(Wan and Xiao, 2008): An extension of TextRank that weights edges in the cooccurrence graph based on the number of cooccurrences.
• TFIDF(Sparck Jones, 1972): A traditional unsupervised approach that evaluates the importance of a word within a document in a corpus.
Sentiment-based. We employ various models to analyze sentiment in texts and predict stock market movements:
• EDT(Zhou et al., 2021): Utilizes a bi-level event detection framework based on BERT to classify corporate events impacting stock prices. • FinGPT(Zhang et al., 2023): Employs instruction tuning on LLMs to enhance financial sentiment analysis capabilities. • GPT-4-turbo: Builds upon GPT-4’s capabilities, enhancing efficiency and performance, especially for processing longer texts. • GPT-4: Represents a significant leap forward from previous iterations, with improved processing speeds and enhanced handling of extended text inputs. • GPT-3.5-turbo: Known for its comprehensive understanding and human-like text generation, with optimized processing for multiple tasks. • RoBERTa: Applies Financial RoBERTa2 for sentiment analysis in English texts and a finetuned sentiment-specific RoBERTa model(Wang et al., 2022) for Chinese texts. • FinBERT: A BERT model fine-tuned for financial sentiment classification, available in both English (Araci, 2019) and Chinese versions3.
Time-based. We employ models that leverage both textual and time-series data to predict stock movement:
• CMIN(Luo et al., 2023): An end-to-end deep neural network that models the multimodality between financial text data and causality-enhanced stock correlations. • StockNet(Xu and Cohen, 2018): A deep generative model that jointly exploits textual and price signals for stock prediction.

heading: 4.4 Implementation Details
text: For keyphrase-based methods, the datasets are divided into two subsets based on labels: one containing data labeled with 0 (indicating a fall) and the other with data labeled with 1 (indicating a rise). Keyphrase extraction is applied to each subset to
2https://huggingface.co/soleimanian/ financial-roberta-large-sentiment
3https://huggingface.co/bardsai/ finance-sentiment-zh-base
identify negative and positive keyphrases for each stock. The score for a given text is calculated as follows: Let POS be the set of positive keyphrases and NEG be the set of negative keyphrases. For a piece of text T , the score S is computed as:
S = σ
( ∑
p∈POS I(p ∈ T )−
∑
n∈NEG I(n ∈ T )
) (3)
where σ(x) is the sigmoid function defined as σ(x) = 1
1+e−x , and I(·) is an indicator function that equals 1 if the condition is true and 0 otherwise. The condition p ∈ T (or n ∈ T ) denotes the presence of a positive (or negative) keyphrase in the text T . For sentiment analysis, we predict the sentiment of the text as either positive or negative and anticipate a rise if the text is positive and a fall if it is negative, in alignment with baseline settings. However, since the EDT dataset contains only textual data, we do not employ time-based methods on it. Additionally, for our factor-based methods, we execute the entire SKGP process on benchmark datasets but exclude the use of the TimeTemplate when assessing the EDT dataset. For a fair comparison, we ensure consistency in the processing of baselines according to their original settings. We employ gpt-3.5-turbo-1106, gpt-4, and gpt-4-1106preview through the API4 for factor-based methods. The parameters are set as follows: 1) the window size t is 5; 2) the number of keyphrases and factors k is 5; 3) the batch size is 64 for BERT series models and 5 for GPT series models. Our experiments are conducted with an NVIDIA RTX A6000 GPU. The details of the pre-trained models used for the baselines are summarized in Appendix B.

heading: 4.5 Results
text: Table 2 presents the results for LLMFactor and various baselines across four datasets. Among all evaluated approaches, our factor-based methods surpass the others, with time-based, sentiment-based, and keyphrase-based methods following in performance. Notably, LLMFactor achieves superior performance over the SOTAs, with improvements of 2.9%, 0.4%, 11%, and 4.8% in MCC across the four datasets, respectively.
The minimal performance differences among keyphrase-based models suggest their limited effectiveness in forecasting stock movements using keyphrases, as keyphrases primarily summarize the main content of the text rather than provide actionable insights on stock prices. Addition-
4https://platform.openai.com/docs/models
ally, sentiment-based methods demonstrate varying performance across different models. LLMs like GPT-4 and GPT-4-turbo excel in identifying sentiment within the text. Although the EDT model achieves the highest accuracy on its specific dataset, its low MCC indicates imbalanced model performance. Unlike sentiment-based methods geared towards efficiently extracting relevant sentiment, time-based methods focus on integrating multiple data types. Since time-based methods consider the text in its entirety without detailed analysis, their performance is on par with that of sentiment-based methods, which may lack temporal context but include detailed textual information. However, our LLMFactor model stands out among other SOTAs due to the SKGP technique, which not only identifies significant factors influencing stock prices from textual data but also incorporates relation and temporal information, thereby enhancing its ability to filter out irrelevant content and provide a more comprehensive analysis.
Analyzing average scores from diverse datasets with the use of three factor-based models, our LLMFactor demonstrates an average accuracy of over 63% and an average MCC of over 0.2 for both the StockNet and CMIN-US datasets, tailored to the US market. However, its performance slightly declines on the CMIN-CN dataset, focused on the CN market, with an average accuracy of 58% and an MCC of 0.19. This variation indicates that the GPT series models are potentially more proficient in processing English-language text. The EDT dataset, which consists solely of stock-related news without historical price data, poses additional challenges, resulting in a marginally lower average accuracy of 59% and an MCC of 0.1 for the LLMFactor. The absence of historical price information in the EDT dataset likely diminishes the LLMFactor’s effectiveness, underscoring the importance of comprehensive data for financial market analysis.

heading: 4.6 Data Analysis

heading: 4.6.1 Ablation Study
text: We conduct an ablation study on the LLMFactor to demonstrate the improvement contributed by each layer. The first layer utilizes only the TimeTemplate, which incorporates past price movements. The second layer incorporates additional factors extracted, and the third layer further includes relations obtained. We average the scores for ACC and MCC from LLMGPT−4−turbo,
LLMGPT−4, and LLMGPT−3.5−turbo, as shown in Table 3. The price layer accounts for approximately 86% and 32% of the total performance in ACC and MCC, respectively, while the factor layer contributes an improvement of 9% in ACC and 46% in MCC. The relation layer leads to a 5% and 22% enhancement in ACC and MCC, respectively. These results imply that the factor layer contributes most significantly to the overall performance of the proposed LLMFactor.
We also conduct experiments on different types of FactorTemplate, as shown in Appendix C.

heading: 4.6.2 Case Study
text: In this section, we highlight the practical effectiveness of LLMFactor through an illustrative application. Figure 3 presents a case study where we apply factor analysis to selected stocks in the US and CN markets. For instance, on September 17, 2019, Apple Inc.’s stock (AAPL) experienced an increase, attributed to significant developments such as a $250 million investment in its supplier, Corning Incorporated, and the positive initial demand for the iPhone 11. Conversely, on May 8, 2018, Tesla Inc.’s stock (TSLA) experienced a decline, mainly due to a record quarterly loss, growing negative concerns among investors, and the increasing competition in the autonomous and electric vehicle markets. This case study demonstrates LLMFactor’s ability to effectively integrate background knowledge of company affiliations with both historical news and price data. By analyzing these diverse data sources, LLMFactor significantly enhances the interpretability of stock market dynamics.

heading: 5 Conclusion
text: In this study, we introduce LLMFactor, an innovative framework designed to predict and elucidate stock market trends accurately. At the core of LLMFactor is the Sequential Knowledge-Guided Prompting (SKGP) strategy, which integrates background knowledge, stock-related factors, and temporal data to forecast stock movements. Through rigorous testing on four benchmark datasets, LLMFactor has proven its superiority over SOTAs that rely on keyphrases, sentiment analysis, and multimodal data inputs. The application of factor analysis highlights LLMFactor’s novelty and effectiveness, establishing it as a powerful tool for financial analysis. This research represents a significant advancement in leveraging LLMs for transparent and explainable financial forecasting.

heading: 6 Limitations
text: We recognize three main limitations:
• Although LLMFactor’s foundation relies on extensively discussed factors, converting temporal data into text format remains vital for financial forecasting. Future studies could explore additional methods for this conversion and their integration into LLMs. • Due to the variable nature of LLM responses, replicating our experimental results precisely is challenging. In future research, we aim to explore further methodologies to improve reproducibility. • Evaluating LLMFactor against benchmark datasets enables comparison with leading models, but it’s essential to carefully assess the quality of factors extracted from texts of various lengths and types. This research marks the beginning of a novel approach to applying factors in finance. Moving forward, we will concentrate on enhancing the quality of these factors.

heading: Acknowledgements
text: We sincerely thank the anonymous reviewers for their reviews. This work was supported by JST SPRING, Grant Number JPMJSP2108, and JSTPRESTO, Grant Number JPMJPR2267.

heading: A Templates of SKGP
text: The fundamental templates employed at each step of SKGP are detailed in Table 5 for English and Table 6 for Chinese. In the final step of SKGP, we compile the responses from previous steps to formulate prompts for LLMs. Nonetheless, due to the variable nature of LLM responses, the exact reproducibility of results cannot be guaranteed. Instead, we demonstrate the overall superior performance of the LLMFactor.

heading: B Details of Pre-trained Models
text: We employ the pke (Boudin, 2016) and pke_zh (Xu, 2023) toolkits to access keyphrase-based methodologies, ensuring alignment with established baselines. Additionally, configuration details for sentiment-based models are outlined in Table 4, provided by HuggingFace. The models specified at the top are designated for English texts, whereas the ones at the bottom cater to Chinese texts.

heading: C Comparative Experimental Results of Factor Templates
text: The experimental results for various factor templates, utilizing three factor-based models: LLMFactorGPT−4−turbo, LLMFactorGPT−4, and LLMFactorGPT−3.5−turbo, are showcased in Table 7 for English templates and in Table 8 for Chinese templates. The datasets tested include CMINUS and CMIN-CN. Within the English templates, the last template achieves the highest performance, while among the Chinese templates, the initial template emerges as the most effective. The consistent effectiveness of LLMFactor across different templates underscores its robustness in predicting stock market movements.

heading: Future Work
text: Future studies could explore additional methods for this conversion and their integration into LLMs. • Due to the variable nature of LLM responses, replicating our experimental results precisely is challenging. In future research, we aim to explore further methodologies to improve reproducibility. • Evaluating LLMFactor against benchmark datasets enables comparison with leading models, but it’s essential to carefully assess the quality of factors extracted from texts of various lengths and types. This research marks the beginning of a novel approach to applying factors in finance. Moving forward, we will concentrate on enhancing the quality of these factors