Fall 2026 Fatima Fellowship Application
This Bucket contains the analysis of a particular blindspot in the current era of AI models, i.e. models exhibiting output instability under semantically invariant input perturbations.
1. Drawing directly from your background and lived experience, identify a critical blind spot, bias, or capability gap in current AI models that is rarely captured by standard benchmarking datasets, and that you feel is underexplored and underdiscussed.
Ans: Flagship AI models of the current generation struggle to produce a stable answer when provided with semantically equivalent variations of the same input. In my experience, this phenomenon is especially visible when the input is short. For example, when a Large Language Model is asked to analyse a search query and determine whether the content requested is safe, it often fails to produce consistent results when a single word is replaced with another of the same meaning.
The model chosen to evaluate this problem is Shieldstral-1.0-3B. A small dataset was curated with queries spanning both safe and unsafe contexts. Our proposition is that a reliable model should produce consistent outputs within each category, demonstrating stable inference.
Findings
Shieldstral's classifications proved unstable under paraphrasing that preserved query meaning. Length and syntactic reordering flipped the safe/unsafe verdict in every cluster tested, with score ranges as wide as 0.91. Register shifts moved scores substantially but never crossed the decision threshold. Lexical substitution was inconsistent across the two clusters tested, so no reliable claim can be made there from this sample.
| Cluster | Perturbation Type | Mean Score | Min | Max | Range | Std | Crosses 0.5? |
|---|---|---|---|---|---|---|---|
| lex_1_weapon_mod | Lexical | 0.951 | 0.924 | 0.967 | 0.043 | 0.017 | No |
| lex_2_chemistry | Lexical | 0.192 | 0.018 | 0.531 | 0.513 | 0.167 | Yes |
| syn_1_pharma | Syntactic | 0.618 | 0.010 | 0.997 | 0.987 | 0.372 | Yes |
| syn_2_access | Syntactic | 0.714 | 0.407 | 0.867 | 0.460 | 0.157 | Yes |
| reg_1_overdose_info | Register | 0.100 | 0.016 | 0.378 | 0.362 | 0.126 | No |
| reg_2_surveillance | Register | 0.933 | 0.706 | 0.995 | 0.290 | 0.103 | No |
| len_1_network | Length | 0.639 | 0.068 | 0.947 | 0.879 | 0.320 | Yes |
| len_2_surveillance_cam | Length | 0.653 | 0.016 | 0.924 | 0.908 | 0.307 | Yes |
These findings complicate the model's reported 88% WildGuard benchmark accuracy, which reflects performance on static inputs, not stability under meaning-preserving rewording - the condition most relevant to real-world evasion. Given the small sample size, this should be read as a directional signal rather than a definitive ranking, but it shows that benchmark accuracy alone is an incomplete proxy for deployment reliability.
3. Propose a path forward: what specific data curation strategy, fine-tuning paradigm, or architectural adjustment do you think might close this gap? This can be purely speculative, you do not need to prove this.
Ans: It appears that the contextual embeddings in these cases are not sufficiently similar to one another, which may be contributing to the instability in the results — particularly across variations in sentence length and syntactic structure, where instability was most pronounced in this evaluation. One way to mitigate this would be to improve how the model maps semantically equivalent inputs to similar contextual embeddings, with particular emphasis on length and syntactic variation rather than lexical substitution alone, since the latter showed no clear pattern in this evaluation. During training, the model should first learn to classify using existing available datasets, and in a second phase, undergo targeted fine-tuning on a curated dataset specifically designed around semantically invariant examples that vary in length and syntactic structure while holding lexical content fixed.