## 4 Experiments
### Extended Experimentation
#### 4.4.2. Mistral 7B  
The fine-tuned Mistral 7B model exhibited an impressive leap in performance during the generation of domain-specific ontologies for SAR operations. Unlike the earlier experiments, where Mistral 7B struggled compared to GPT-4, this iteration not only matched but surpassed GPT-4 in both depth and contextual accuracy. Leveraging domain-specific data, the model elevated the interaction among the three roles based on the HCOME OE methodology [17]. The iterative discussions, enriched by the model's ability to simulate nuanced domain knowledge, produced more insightful and well-rounded outputs, with enhanced clarity and purpose in defining and linking key concepts relevant to wildfire SAR scenarios. The ontology metrics further validate this success. Mistral 7B generated an ontology comprising 65 classes and 51 object properties, surpassing GPT-4 in capturing relationships and semantic depth (Tables 5 and 6). More specifically, for class identification, the Mistral 7B basic model achieved a precision of 68.4%, a recall of 15.25%, and an F1 score of 0.2626. After fine-tuning, precision decreased to 60%, reflecting a slight 12.3% decrease, but recall improved significantly to 48.75%, corresponding to an improvement equal to 219.7%. As a result, the F1 score rose to 0.5379, marking an overall improvement of 104.8%. This highlights a substantial enhancement in the model's ability to identify relevant classes, despite the slight decline in precision. For object property identification, the basic Mistral 7B model achieved a precision of 23.5%, a recall of 6.66%, and an F1 score of 0.1038. The new fine-tuned model improved precision to 52.94%, a 125.3% increase, and recall to 45%, reflecting an exceptional 575.7% improvement. This significant recall gain was the main contributor to the F1 score rising to 0.4864, an overall improvement of 368.4%. Notable additions included classes and properties that detailed relationships between incidents, services, and environmental observations. The model introduced richer subclass hierarchies and properties, reflecting its improved grasp of the domain-specific complexities. This performance underscores the model's ability to semantically integrate knowledge, connecting heterogeneous data sources such as sensor observations and real-time weather conditions, and translating these into meaningful ontological structures. The fine-tuned Mistral 7B also excelled in leveraging discussions to optimize the ontology iteratively. The elevated level of role-play discussions facilitated not just the coverage of competency questions but also the generation of more context-aware and operationally useful ontologies. Mistral's ability to encapsulate operationally grounded and semantically rich outputs signifies its potential as a cost-efficient alternative to larger models, capable of yielding high-quality domain-specific ontologies tailored to real-world SAR challenges. The performance of Mistral 7B exhibited significant inconsistencies across different fine-tuning sessions. Initial training attempts (as presented in Section 4.2.2, Tables 3 and 4) displayed notable deficiencies, such as class duplication, misclassification of object properties, and inconsistencies in relationship assignments. These issues primarily stemmed from early-stage hyperparameter settings, dataset preprocessing inconsistencies, and limitations in the initial prompt structures guiding the model. However, later iterations (Section 4.4.2, Tables 5 and 6) demonstrated a marked improvement, even surpassing GPT-4 in certain cases. Several key factors contributed to this transition from suboptimal to high-performance results:  
1. Data Preprocessing Adjustments: Early dataset formatting included overlapping or redundant entity descriptions, leading to class duplication and property assignment errors. By refining the dataset through more structured entity extraction and deduplication techniques, later training runs exhibited more consistent outputs. 2.