Several linguistic phenomena at the semantics-pragmatics interface could provide insights into language model capabilities beyond scope ambiguities. Some of these phenomena include:
Anaphora Resolution: Language models' ability to correctly identify and interpret pronouns and their referents in a sentence can reveal their understanding of discourse coherence and the relationships between entities in a text.
Presupposition and Entailment: Models' handling of presuppositions (assumptions made by speakers) and entailments (implications of a statement) can shed light on their ability to capture implicit meanings and context in language.
Lexical Semantics: Understanding how models deal with polysemy (words with multiple meanings) and homonymy (words that are spelled the same but have different meanings) can provide insights into their lexical knowledge and semantic disambiguation capabilities.
Modality and Mood: Examining how models interpret modal verbs (e.g., can, must) and mood markers (e.g., subjunctive mood) can reveal their understanding of certainty, possibility, and speaker attitudes in language.
Speech Acts: Language models' performance in recognizing speech acts (e.g., requests, promises, apologies) can demonstrate their pragmatic understanding of language use and communicative intentions.
By exploring these linguistic phenomena, researchers can gain a more comprehensive understanding of language model capabilities in various aspects of semantics and pragmatics.