# Exploring Decentralized Warehouse Management Using Large Language Models: A Proof of Concept
## 2 Theoretical Background
### Large Language Models  
Large Language Models (LLMs), a specialized subset of AI technologies, have revolutionized Natural Language Processing (NLP) by enabling machines to understand and generate human-like language with unprecedented fluency and contextual awareness [25]. Built on the Transformer architecture, models like OpenAI's GPT-4 and Google's PaLM use a self-attention mechanism, defined in Equation (10), to process input sequences in parallel and capture long-range dependencies [15]. \[\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{QK^{\top}}{\sqrt{d_{k}}} \right)V, \tag{10}\]  
where \(Q\), \(K\), and \(V\) represent the query, key, and value matrices derived from the input embeddings, and \(d_{k}\) is the dimensionality of the key vectors. Pre-trained on diverse corpora, including web texts and academic papers, LLMs encode rich semantic representations and emergent reasoning abilities through self-supervised learning [15]. In decentralized systems such as warehouse management, decision-making usually involves the evaluation of complex, multidimensional data under uncertainty. LLMs are particularly suitable for this area as they are capable of interpreting structured and unstructured inputs, including natural language prompts and symbolic data such as prices, capacities, and time constraints; generating contextual responses, making them effective agents for negotiation, scheduling, or exception handling; and learning and adapting to historical interaction data, enabling cumulative performance improvements without explicit reprogramming. For example, in our framework, LLMs are prompted with a structured format that includes background knowledge, the current system state, and previous decision outcomes. Based on these inputs, the LLM issues a decision (e.g., to accept, reject, or forward an offer) and thus acts as a strategic decision engine within a multi-agent system, as detailed in Section 3 (System Architecture). Two primary prompting strategies are used in LLM applications: stateless prompting and stateful prompting. In stateless prompting, each interaction is independent, so the entire context (background (B), current situation (C), and historical data (S)) must be included in the prompt. This ensures reproducibility and is useful when system memory is not preserved between decisions. In contrast, stateful prompting allows the LLM to retain a kind of memory in which only incremental updates are provided. This is computationally more efficient and supports long-term strategic consistency, making it ideal for persistent agent roles in a distributed environment. Mathematically, the prompt formulation can be expressed as shown in Equation (11):  
\[\mathcal{P}_{k}= \begin{cases}\mathcal{P}(B_{k}\cup C_{k}\cup S_{k})&\text{if Stateless Mode, for all $k$,}\\ \mathcal{P}(B_{0}\cup C_{0}\cup S_{0})&\text{if Stateful Mode, at $k=0$,}\\ \mathcal{P}(C_{k})&\text{if Stateful Mode, for $k>0$.}\end{cases}  \tag{11}\]  
In the stateless mode, the prompt explicitly includes the background \(B_{k}\), current context \(C_{k}\), and historical data \(S_{k}\) at each step, ensuring independence between iterations. In contrast, the stateful mode differentiates the initial prompt (\(k=0\)) by explicitly including\(B_{0}\), \(C_{0}\), and \(S_{0}\). For subsequent prompts (\(k>0\)), \(B\) and \(S\) are implicitly inherited, requiring only the addition of \(C_{k}\) to maintain continuity across interactions. Each method has its advantages and is suited to different scenarios. Stateless prompting is preferred in environments where reproducibility and independence between decisions are critical, while stateful prompting is useful for systems requiring long-term contextual awareness and cumulative decision-making. In this study, stateless prompting is employed to ensure reproducibility in the simulation environment (Section 4), achieving sub-second inference latencies suitable for real-time warehouse operations [14].