## 4 Experiments
### Results  
#### 4.2.1. GPT-4 Fine-Tuning  
As an initial step in our experimentation, we fine-tuned a GPT-4 model using a dataset derived exclusively from the book \"Semantic Web for the Working Ontologist\" by [PERSON] and [PERSON] (2018). This dataset, extracted and structured as Q&A pairs, aimed to capture the core ontological engineering principles, classes, relations, and foundational concepts detailed within the text. For this first fine-tuning run, we opted to rely on the automated configuration of hyperparameters to evaluate the out-of-the-box adaptability of the approach. Specifically, we trained the model for 3 epochs at a batch size of 1, utilized a learning rate (LR) multiplier of 2, and set the temperature to 1. After approximately 53 min, the fine-tuned model was ready for testing. Figure 3: A screenshot from GPT-4 creating the first batch of 20 question-answer pairs for the datasets. To thoroughly assess performance, we prompted both the newly fine-tuned model and a baseline GPT-4 version (here referenced as GPT-4o) with a scenario involving OE requirements, role-playing elements to simulate an iterative discussion, and a set of competency questions relevant to the domain. By running both models in parallel (Figure 4), we aimed to establish a comparative benchmark. The attached table (first row of the fine-tuning results--Tables 1 and 2) provides quantitative insights: While the fine-tuned model did not achieve the breadth of classes or properties identified by the reference standard, it did present different patterns of true positives, false positives, and false negatives compared to the basic GPT-4 baseline model. Although the fine-tuned model exhibited a higher precision--reflecting its tendency to produce fewer incorrect identifications--it came at the cost of lower recall, suggesting that it recognized fewer relevant concepts overall. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & **Number** & **True** & **False** & **False** & **Precision** & **Recall** & **F1 Score** \\  & **of Classes** & **Positive** & **Positives** & **Negatives** & & & \\ \hline Reference Ontology & \multicolumn{3}{c}{80} & & & & & & \\ \hline GPT-4 Basic Model & 13 & 9 & 4 & 71 & 69\% & 11.25\% & 0.1935 \\ \hline GPT-4 Fine-Tuned Model 1 & 5 & 4 & 1 & 76 & 80\% & 5\% & 0.0941 \\ \hline GPT-4 Fine-Tuned Model 2 & 12 & 10 & 2 & 70 & 83\% & 12.5\% & 0.2173 \\ \hline GPT-4 Fine-Tuned Model 3 & 16 & 13 & 3 & 67 & 81.25\% & 16.25\% & 0.2708 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance comparison of the baseline GPT-4 model and successive fine-tuned iterations in generating ontological classes, evaluated against a human expert reference ontology. The table highlights changes in true positives, false positives, false negatives, as well as in precision, recall, and F1 score, over multiple refinement steps. Figure 4: Running the same OE prompt side-by-side on both a baseline and a fine-tuned GPT-4 model, allowing for direct comparison of their role-based reasoning, conceptual coverage, and quality of generated ontologies. In this early stage, the results highlight a critical tension in OE tasks: precision versus recall. The fine-tuned model, somewhat more conservative in its output, yielded fewer spurious results but also missed some relevant ones. These findings underscore the importance of iterative refinements and additional data augmentation. They also suggest that automated hyperparameter settings, while convenient, may not fully optimize performance for nuanced, domain-specific tasks, but that in our research is not the case as we try to automate the process as much as possible. In the second fine-tuning iteration, we introduced additional training data from the book \"A Semantic Web Primer\" by [PERSON], [PERSON], [PERSON], and [PERSON] [20], building upon the previously fine-tuned model trained solely on Semantic Web for the Working Ontologist.