# The Specialist's Paradox: Generalist AI May Better Organize Medical Knowledge
## 4 Discussion  
The central finding of this study is the clear stratification of sentence-transformer models according to their ability to organize standardized MeSH concepts. The results establish a performance hierarchy that challenges the assumption that greater domain specialization invariably improves every downstream task [42; 43]. A top tier--composed of the general-purpose MPNet and RoBERTa models and the domain-adapted BioBERT checkpoint used here--produced well-separated clusters. These encoders yielded tight, specialty-specific clusters, corroborated by high external scores (Table 3), with \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Misclassified** & **Error Rate** & **Purity** \\ \hline MPNet & 5 & 7.14\% & 92.86\% \\ RoBERTa & 5 & 7.14\% & 92.86\% \\ BioBERT & 6 & 8.57\% & 91.43\% \\ PubMed & 19 & 27.14\% & 72.86\% \\ MiniLM & 19 & 27.14\% & 72.86\% \\ Bioclin & 27 & 38.57\% & 61.43\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary of clustering purity and error rates for each embedding model. ARI = 0.835 for both MPNet and RoBERTa and a purity above 91% for all three models. The BioBERT variant evaluated (pritamdeka/BioBERT-mnli-snil-scinli-scitail-mednli-stsb) had been fine-tuned with a large, contrastive sentence-pair objective in addition to its biomedical masked-language modeling (MLM) [44]. That extra stage explicitly draws semantically related phrases together, possibly explaining why BioBERT rivals the best general encoders despite being domain-adapted rather than domain-exclusive. MiniLM forms a middle tier. Its distilled architecture--384 hidden dimensions and six transformer layers--reduces latency but sacrifices the representational granularity required to separate closely related biomedical descriptors, yielding a modest ARI of 0.512 and a purity of 72.86%. The model still benefits from contrastive training but cannot match the capacity of its larger peers, resulting in it being quite inadequate to carry out complex semantic tasks in the medical field. In contrast, encoders pre-trained only with MLM on specialized corpora performed poorly. The PubMedBERT checkpoint we selected (pritamdeka/S-PubMedBert-MS-MARCO) was optimized for passage retrieval rather than semantic similarity and never exposed to a sentence-pair loss. It achieved a low ARI of 0.506 and scattered terms from every specialty across multiple clusters. This outcome shows that in-domain vocabulary alone does not guarantee globally coherent embeddings; the supervisory signal must match the end task. BioClinicalBERT performed worse still (ARI = 0.350). Two factors contribute: its continued MLM on abbreviation-rich hospital notes shifts the embedding space toward clinical shorthand, and its revised WordPiece vocabulary splits several multitoken MeSH descriptors, fragmenting their semantics [35]. Lacking any contrastive correction, the model delivers a noisy semantic map in which only Oncology forms a stable island, while Cardiology and Dentistry disintegrate. MPNet's top Normalized Mutual Information of 0.902 can be traced to its billion-sentence contrastive training, which directly optimizes cosine distance for semantic similarity [31; 32]. RoBERTa achieves the same by leveraging a more exhaustive base pre-training schedule that remains robust once a sentence-pair loss is layered on by the sentence-transformer framework [32]. BioBERT blends those strengths, pairing domain MLM with an NLI-style objective that realigns biomedical synonyms after pre-training [22]. One ostensibly paradoxical result demands emphasis. BioClinicalBERT registers a Silhouette coefficient of 0.152--better than that of MiniLM (0.071) and PubMedBERT (0.104)--yet returns the poorest ARI and purity. The model produces geometrically \"tight\" clusters that are semantically meaningless. This disconnect demonstrates why internal indices alone cannot certify the quality of a biomedical embedding space; without ground-truth labels, spurious structure can be mistaken for meaningful organization.