## 4 Experiments
### Overall Performance  
We compare our EMRE2 llm model against several baselines and recent models for MRE. These models can be categorized into text-only models and image-text models, to demonstrate the superiority of our model.  
First, text-only traditional models are used to demonstrate the indispensability of visual information in multimodality. We select two representative text RE models: PCNN [53] and MTB [54], a BERT-based RE pre-training model.  
Second, we select previous SOTA models on image-text datasets. UMT [55] uses multimodal Transformers with an auxiliary entity span detection module. BERT + SG [17] connects BERT-derived text representations with scene image features. VisualBERT [56] is a single-stream pretrained visual language model. MEGA [17] employs dual image alignment to capture correlations between textual and visual objects.  
Finally, we conduct a comparison with the latest SOTA models. EEGA [21] first proposes the joint extraction task JMERE of multimodal entities and relations, representing text and images as two images and aligning visual and textual objects. HVPNeT [20] proposes a dynamic gated aggregation strategy to realize multi-level multi-scale visual features as fused visual prefixes. MKGformer [22] uses pure Transformer architecture with PGI and CAF cross-attention techniques to jointly learn textual and visual information.  
Table 2 shows the results of our EMRE2 llm model and other models on the MNRE dataset, also contrasting our enhanced small model EMRE2 llm_nollm. The EMRE2 llm_nollm model is our enhanced small model in the first stage. Its output is processed simply based on the confidence scores, which means the relation labels are directly assigned according to the predicted probability values for each category.  
In order to better utilize the prompt information from small models and guide GPT-3.5, we adopt confidence screening strategies and context guiding strategies. Specifically, the confidence screening strategy divides the confidence scores of the relations predicted by the small model into intervals using the \(\gamma\) hyperparameter to obtain data with uncertain small model correctness. This ensures the effectiveness of the small model on data without excessive external knowledge bases while avoiding the hallucination problem of large models as much as possible, and dealing more with data with diffuse factors. For the context guiding strategy, we select context example data consisting of part of the training  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Modality**} & \multirow{2}{*}{**Methods**} & \multicolumn{3}{c}{**MNRE**} \\ \cline{3-5}  & & **Precision** & **Recall** & **F1** \\ \hline \multirow{2}{*}{Text} & PCNN [53] & 62.85 & 49.69 & 55.49 \\ \cline{2-5}  & MTB [54] & 64.46 & 57.81 & 60.86 \\ \hline \multirow{5}{*}{Text} & UMT [55] & 62.93 & 63.88 & 63.46 \\ \cline{2-5}  & BERT + SG [17] & 62.95 & 62.95 & 62.80 \\ \cline{2-5}  & VisualBERT [56] & 57.15 & 59.48 & 58.30 \\ \cline{2-5}  & UMGF [2] & 77.51 & 76.01 & 76.75 \\ \cline{2-5}  & MEGA [17] & 64.51 & 68.44 & 66.41 \\ \cline{2-5}  & EEGA [21] & 78.27 & 78.91 & 78.59 \\ \cline{2-5}  & HYPNeT [20] & 83.64 & 80.78 & 81.85 \\ \cline{2-5}  & MKGformer\({}^{\star}\)[22] & 80.83 & 79.06 & 79.94 \\ \cline{2-5}  & EMRE2 llm\_nollm(ours) & 84.34 & 80.78 & 82.52 \\ \cline{2-5}  & EMRE2 llm(ours) & **85.06** & **80.94** & **82.95** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance comparison of various baseline models on the MNRE dataset. Here, MKGformer\({}^{\star}\) is our re-implemented result. Even compared to the 81.95 F1 of MKGformer in the original paper, our EMRE2 llm has noticeable gains. The best scores are bolded.  
data, with specific implementation methods including manually selected fixed numbers and random sampling. Both help guide GPT-3.5's understanding of the relation extraction task to some extent.