# Modeling Semantic-Aware Prompt-Based Argument Extractor in Documents
## 5 Analysis  
### Comparative Analysis of BERT and GPT Performance on D-EAE Tasks  
To assess the capabilities of different pre-trained language models on D-EAE, we compare the performance of GPT-series models with that of our proposed BERT-based model, SPARE. The results for GPT-3.5 are adopted from [PERSON] et al. [52], while the GPT-4 results are based on findings reported by [PERSON] et al. [53]. Table 5 presents experimental results on the RAMS dataset, illustrating how these models perform on two key metrics: argument identification (Arg-I) and argument classification (Arg-C).  
As shown, SPARE significantly outperforms both GPT-3.5 and GPT-4. Specifically, it achieves an Arg-I score of 57.3 and an Arg-C score of 52.9, while GPT-4 records 50.4 and 42.8, respectively. GPT-3.5 exhibits an even lower performance. These results underscore the advantages of integrating BERT's architectural characteristics with targeted enhancements tailored for D-EAE tasks.  
SPARE builds upon the BERT framework, which inherently benefits from a bidirectional encoding mechanism. Unlike GPT models--which employ a unidirectional, autoregressive architecture that processes input from left to right--BERT is trained through masked language modeling, allowing simultaneous attention to both preceding and succeeding tokens. This bidirectional context modeling is especially advantageous in D-EAE, where identifying arguments often relies on long-range dependencies and semantic cues that span multiple sentences.  
D-EAE inherently involves multi-sentence reasoning and discourse-level understanding. Argument spans may appear far from their corresponding event triggers, requiring the model to resolve coreference, infer implicit relations, or track entity roles across a document. SPARE leverages BERT's holistic context representation to better capture these complex inter-sentential relationships, providing it with a clear edge over GPT models that struggle to incorporate future context during inference.  
In addition, SPARE is explicitly fine-tuned on the argument extraction task, enabling it to learn structural regularities and semantic roles specific to event arguments. In contrast, GPT models are typically applied in zero-shot or few-shot paradigms without specialized task-specific training. This reliance on generic pre-training makes them less effective in structured prediction tasks like D-EAE, where precision in boundary detection and role classification is critical. Consequently, the performance gap observed reflects both architectural differences and task adaptation strategies.  
Furthermore, we provide a brief analysis of the computational complexity of SPARE to complement the empirical evaluation. The inference complexity of SPARE, which is built upon a BERT-based encoder, is approximately \(O(n^{2}\cdot d)\), where \(n\) denotes the sequence length and \(d\) is the hidden dimensionality. This reflects the quadratic complexity characteristic of the self-attention mechanism [19]. However, due to its fully parallelizable architecture, SPARE achieves efficient inference in practice, particularly when leveraging GPU acceleration. In contrast, GPT's autoregressive decoding introduces sequential constraints, leading to a linear time dependency on the number of output tokens during  
\begin{table}
\begin{tabular}{l c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**RAMS**} \\ \cline{2-3}  & **Arg-I** & **Arg-C** \\ \hline GPT3.5 & 46.2 & 40.4 \\ GPT4 & 50.4 & 42.8 \\ \hline SPARE & 57.3 & 52.9 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results on the RAMS dataset.  
generation. This can significantly slow down inference for long sequences [14]. These observations further underscore the advantages of encoder-only architectures like SPARE for structured extraction tasks that demand efficient and precise boundary-level predictions.