## 4 Experiments
### Results
#### 4.2.2. For the Mistral 7B experiments, our methodology departed from the approach detailed in Section 3.4.1. While we continued to leverage the same foundational OE datasets, the operational workflow required a more hands-on, iterative process. This shift was necessitated by the distinct platform architecture and configuration requirements of the Mistral environment, which differ significantly from the streamlined setup provided by OpenAI. Unlike with the GPT-4 runs--where fine-tuning could be performed iteratively within the same ecosystem--adopting Mistral 7B involved re-uploading the entire datasetand re-establishing the training parameters each time a new fine-tuning run was initiated. This ensured a rigorously controlled refinement process but also introduced additional logistical overhead, as every iteration had to be executed from a fresh baseline tailored to the Mistral platform's unique constraints. In this first fine-tuning iteration with Mistral 7B, we used the same OE dataset as before, but the process of adjusting hyperparameters was carried out manually because the platform did not have the option of automated selection. Despite the model successfully recognizing and maintaining the role-playing framework established in the prompt--distinguishing between a Knowledge Engineer, Domain Expert, and Knowledge Worker--it did not produce the requested ontology on the first attempt. Although the prompt explicitly instructed the model to generate the ontology, it initially provided only the iterative discussion without the final ontology construction. Consequently, we had to prompt it again, which resulted in the eventual creation of the ontology in Turtle (TTL) format (Figure 5). Upon reviewing the ontology that Mistral 7B generated (Table 3), we observed that it produced a total of 21 classes. However, closer inspection revealed that 16 of these were duplicates, effectively leaving us with approximately 8 distinct classes. A plausible reason for this duplication could be that the model latched onto certain terms or phrase structures repeatedly, due to pattern replication rather than conceptual understanding. For instance, the model may have recognized the lexical pattern of a class name and recreated variations of it without introducing genuinely new semantic content. This behavior might stem from a combination of factors: the complexity of the prompt, the model's relative inexperience with the specific domain content (compared to more extensively fine-tuned models), or subtle ambiguities in the training data that led to repetitive outputs. Additionally, Mistral 7B, being a smaller model than GPT-4 and using a more manually handled fine-tuning process, may have fewer internal parameters to leverage for nuanced differentiation among closely related concepts. While the model did understand the narrative setup and roles, it struggled to follow through with the ontology generation on the first attempt and produced a partially redundant set of classes on the second. These initial results suggest that, at this stage, Mistral 7B may require more careful prompt engineering, possibly more domain  
Figure 5: A side-by-side comparison of the baseline Mistral 7B model output (left) versus the fine-tuned version (right). Although the model understood the role-based scenario from the outset, it did not produce the requested ontology in response to the initial prompt and required an additional prompt before the ontology was finally generated in Turtle format. specific training data, or further iterative refinements to reduce redundancy and ensure that each generated class represents a truly distinct and meaningful ontological concept. Compared to the Mistral Basic Model's initial performance on object properties, the first fine-tuned iteration presents a clear regression (Table 4). The baseline model identified some object properties correctly, registering a few true positives alongside several false positives.