Probing GPT -3’s Linguistic Knowledge on Semantic Tasks (2022)

Abstract: GPT-3 has attracted much attention from both academia and industry. However, it is still unclear what GPT-3 has understood or learned especially in linguistic knowledge. Some studies have shown linguistic phenomena including negation and tense are hard to be recognized by language models such as BERT. In this study, we conduct probing tasks focusing on semantic information. Specifically, we investigate GPT-3’s linguistic knowledge on semantic tasks to identify tense, the number of subjects, and the number of objects for a given sentence. We also experiment with different prompt designs and temperatures of the decoding method. Our experiment results suggest that GPT-3 has acquired linguistic knowledge to identify certain semantic information in most cases, but still fails when there are some types of disturbance happening in the sentence. We also perform error analysis to summarize some common types of mistakes that GPT-3 has made when dealing with certain semantic information.

## introduction gpt-3 @xcite is a large neural language model (nlm) released in 2020, and it has realized state-of-the-art performance on various language tasks. disregarding its achievement in recent years, however, few pieces of literature interpret well what would happen inside gpt-3, as well as the knowledge it has acquired or represented. this is also true for understanding linguistic phenomena, which represent all features and grammar that a linguist should study @xcite . based on recent studies, like the task-agnostic methodology named checklist @xcite , it is revealed that nlp models have a high failure rate in testing linguistic phenomena, though they may perform well in many other language tasks. this paper contributes to the existing literature by making an effort on understanding gpt-3's knowledge of the linguistic phenomenon, especially with a fo-cus on semantic information including tense and singularity or plurality of the number of subject and object of a given sentence. the senteval probing tasks @xcite introduce 10 probing tasks covering the aspects of surface information, syntactic information, and semantic information. in our study, we want to evaluate gpt-3's knowledge and understanding of linguistic phenomena, thus we focus on the aspect of semantic information. specifically, we apply the semantic tasks (tense, subjnum, and objnum) to test gpt-3's linguistic knowledge to understand tense and singularity or plurality of the number of subject and object, which do not involve any replacement or inversion of source corpus. (see section 3.1 for details of these semantic tasks.) for our experiment, we design zero-shot and few-shot prompts separately, which means different numbers of examples from the dataset appear in the prompt. we also set the binary choice question like "is the number of the subject of the sentence singular or plural?" as the default prompt style, while designing another general prompt that allows gpt-3 to give its own answer to the general question. more details about prompt design can be found in based on our experiment result, we find that gpt-3 has acquired some linguistic knowledge to understand semantic information like tense and singularity or plurality of subject and object, though it may be disturbed in some cases. besides, we notice that designing the prompt with the general question might lead to model performance degradation. the model tends to provide irrelevant answers, since no expected choice is provided as in the default prompt style. we also find that variation in temperature has a minor impact on gpt-3's performance. further, it is unexpected that more examples in the few-short prompts confuse and hurt the model in some tasks, rather than providing more hints. our work contributes to the stream of the work on probing the large language models, which helps us better understand what linguistic properties the model has acquired or represented. specifically, we provide better insights on gpt-3's linguistic knowledge of certain semantic information. ## related work in recent years, although pre-trained language models like bert @xcite have achieved state-of-the-art performance in many nlp tasks, it is still difficult to figure out what linguistic information is learned by the language representations. probing tasks are designed to test whether language models have encoded linguistic phenomena in learned representations by training a probing classifier on these representations. in an early study of machine translation, @xcite convert source sentences into encoded representations by the neural machine translation model, and train a logistic regression model on these representations to predict syntactic labels. in another study, @xcite design tasks to measure what extent the sentence representation from cbow @xcite and lstm auto-encoder encodes its length, the identities of words within it, and word order. their results indicate that the probing task is an effective way to evaluate the language model's ability to learn linguistic information. thus, many recent works have made some efforts to profile neural language models (nlms) @xcite @xcite . for example, @xcite test whether the language model assigns a higher probability to the grammatical sentence than the ungrammatical ones, showing that the performance of the language model lags behind the human performance in recognizing the grammaticality of the sentence. warstadt et al. (2019) also assess the nlm's ability on learning grammatical knowledge and show that the bert has significant knowledge of some grammatical features in sentences. @xcite test the model's ability to understand linguistic features, such as sentence length and part-of-speech tagging (pos tagging). it reveals that "the more nlm stores readable linguistic information of a sentence, the stronger its predictive power". many other works also focus on understanding the attention mechanism of nlms @xcite @xcite . for example, @xcite conduct an analysis on bert's attention and show that "certain attention heads correspond well to linguistic notions of syntax and coreference". previous work has provided evidence of nlm's ability to learn linguistic knowledge from the data. some work tries to understand whether the learned linguistic knowledge has a particular structure (e.g. hierarchical structure) @xcite . these works have developed important probing tasks that profile the different aspects of the linguistic knowledge of nlms. we follow the approach to conduct our own experiments of probing tasks on exploring gpt-3's ability to understand linguistic phenomena. ## experiment we test the semantic tasks (tense, subjnum, and objnum) from senteval @xcite on gpt-3 with combinations of different prompt designs and temperatures, and calculate accuracy for each type of linguistic phenomena in the probing task. ## error analysis we perform the error analysis on records that gpt-3 does not answer correctly, which indicates the response is either adverse or irrelevant with label 2 and label 3 separately. we manually go through some records that get incorrect responses from gpt-3 to mark them with potential reasons and categorize them into several types of mistakes. however, this process requires a large amount of human annotations for each scenario to identify mistake types precisely, which may not be feasible in our case. thus, this analysis may not exhaust all possibilities that gpt-3 might make mistakes when identifying certain linguistic phenomena and is not able to quantify the corresponding proportions, but it still provides some insights into how gpt-3 understands linguistic knowledge to some extent. below is a brief summary of some common types of mistakes that gpt-3 has made when returning the response. examples of each mistake type can be found in disturbance of quotation mark for sentences that have partial content inside quotation marks as part of the dialogue, if the tense of the main verb is in past but the tense of the content inside quotation marks is in present, then gpt-3 will predict the tense as "present" incorrectly.