## 2 Foundational Concepts and Technologies
### Large Language Models (LLMs)
#### 2.2.1 Capabilities and Applications in the Financial Domain  
LLMs represent a significant advancement in the field of AI and NLP. These models are designed to understand, generate, and interact with human language in a way that is increasingly indistinguishable from human performance. Examples of LLMs include OpenAI's GPT-4 and GPT-4o [20], Google's Gemini [21], and Claude and the LLaMA series [18]; their releases have been shown in Figure 5. They have been trained on vast amounts of text data and leverage advanced transformer architectures to perform a wide range of language-related tasks, from translation and summarization to question answering and creative writing [18]. Figure 4: Horizontal lines show Fibonacci retracement levels of 23%, 38%, 50%, 62%, and 78%, measured from the top of the uptrend; screenshot from Ref. [5]. Applications of LLMs could be used in various domains [22]. Some of the most common applications include text generation, translation, question answering, summarization, and sentiment analysis. LLMs can generate human-like text based on a given prompt, which is used in creative writing, automated content creation, and chatbots. For example, GPT-4 [20] can write essays, poems, and even code snippets based on a few lines of input. LLMs can translate text from one language to another with high accuracy. Models like Gemini and GPT-4 have been fine-tuned for translation tasks, enabling them to understand and generate text in multiple languages. LLMs can answer questions based on a given context. This is used in applications like virtual assistants, customer support, and educational tools. Summarization models can extract the key points from a document and present them in a coherent manner. LLMs can analyze the sentiment of a given text, determining whether it is positive, negative, or neutral. This is used in social media monitoring, market research, and customer feedback analysis. Sentiment analysis models can help businesses understand customer opinions and make informed decisions. Despite their impressive capabilities, LLMs face several challenges and limitations [23]. One of the main challenges is the computational cost of training and running these models. LLMs require significant computational resources, including powerful GPUs and large amounts of memory. This makes them expensive to train and deploy. Another challenge is the issue of bias. LLMs are trained on large datasets that may contain biased or unrepresentative data. As a result, the models can learn and propagate these biases, leading to biased or unfair outputs. LLMs also struggle with understanding context and common sense reasoning [23]. While they can generate coherent text, they may not always produce factually accurate or contextually appropriate responses. This is because LLMs rely on patterns in the training data rather than genuine understanding. For instance, an LLM might generate a plausible-sounding but incorrect answer to a factual question. Time series prediction plays a fundamental role in financial forecasting. Classical approaches, such as ARIMA [24] and exponential smoothing, analyze data by breaking them down into trend, seasonal, and residual components to uncover patterns and forecast future values. However, these models are limited by their assumption of linear relationships, making them less effective at capturing the intricate non-linear dependencies characteristic of financial markets. LLMs leverage deep learning and extensive datasets to understand and predict sequential data [25, 26]. They excel at capturing trends and seasonality, especially in datasets with clear patterns. Studies show LLMs like GPT-4 perform well on datasets with strong periodicity [23]. Techniques such as natural language paraphrasing and incorporating external knowledge via prompts can further enhance their performance [27]. However, LLMs face challenges with multi-period datasets, struggling to recognize distinct periods [28], similar to other methods. They are computationally demanding but often perform comparably to simpler models.