Patent application EP21169044A (EP), filed 2021-04-19
Publication number: EP3929826A1
Title: CONTRASTIVE SELF-SUPERVISED MACHINE LEARNING FOR COMMONSENSE REASONING
Abstract: In an example embodiment, a self-supervised learning task is used for training commonsense-aware representations in a minimally supervised fashion and a pair level mutual-exclusive loss is used to enforce commonsense knowledge during representation learning. This helps to exploit the mutual-exclusive nature of the training samples of commonsense reasoning corpora. Given two pieces of input where the only difference between them are trigger pieces of data, it may be postulated that the pairwise pronoun disambiguation is mutually exclusive. This idea is formulated using a contrastive loss and then this is used to update the language model.
IPC codes: G06N   3/    08            A I
Claims: number: 1
text: A system comprising:
number: 2
text: The system of claim 1, wherein the operations further comprise:
number: 3
text: The system of claim 2, wherein each grouping of data is a grouping of text sentences.
number: 4
text: The system of claim 2 or 3, wherein the ambiguous piece of data is a pronoun and the plurality of pieces of data from which the pronoun needs to be resolved are nouns in the grouping of text sentences.
number: 5
text: The system of any one of the preceding claims, wherein the machine learning algorithm is a neural network,
wherein the neural network may utilize a Bidirectional Encoder Representations from Transformers (BERT) sentence encoder.
number: 6
text: The system of any one of the preceding claims, further including a chatbot, wherein the chatbot includes a model trained to take the input grouping of data and the prediction of the resolution of the ambiguous piece of data from the commonsense machine learned model and output a text output to a user responsive to the input grouping of data.
number: 7
text: A method comprising:
number: 8
text: The method of claim 7, wherein the operations further comprise:
number: 9
text: The method of claim 8, wherein each grouping of data is a grouping of text sentences.
number: 10
text: The method of claim 8 or 9, wherein the ambiguous piece of data is a pronoun and the plurality of pieces of data from which the pronoun needs to be resolved are nouns in the grouping of text sentences.
number: 11
text: The method of any one of claims 7 to 10, wherein the machine learning algorithm is a neural network.
number: 12
text: The method of claim 11, wherein the neural network utilizes a Bidirectional Encoder Representations from Transformers (BERT) sentence encoder.
number: 13
text: The method of any one of claims 7 to 12, further including a chatbot, wherein the chatbot includes a model trained to take the input grouping of data and the prediction of the resolution of the ambiguous piece of data from the commonsense machine learned model and output a text output to a user responsive to the input grouping of data.
number: 14
text: A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
number: 15
text: The non-transitory machine-readable medium of claim 14, wherein the operations further comprise the operations of any one of claims 7 to 13.
Office action: search_report, dated 2021-12-29
Rejection reasons: novelty_epc_54
Prior art cited: ref_id: - TASSILO KLEIN ET AL: "Contrastive Self-Supervised Learning for Commonsense Reasoning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 2 May 2020 (2020-05-02), XP081657164
ref_type: npl
severity: background
ref_id: - SHANE STORKS ET AL: "Commonsense Reasoning for Natural Language Understanding: A Survey of Benchmarks, Resources, and Approaches", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 2 April 2019 (2019-04-02), XP081163657
ref_type: npl
severity: background
ref_id: - TASSILO KLEIN ET AL, "Contrastive Self-Supervised Learning for Commonsense Reasoning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20200502), XP081657164 [X] 1-5,7-12,14,15 * abstract * * sections 1 to 5;; figure 1 * [I] 6,13
ref_type: npl
severity: novelty_destroying
ref_id: - SHANE STORKS ET AL, "Commonsense Reasoning for Natural Language Understanding: A Survey of Benchmarks, Resources, and Approaches", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20190402), XP081163657 [A] 1-15 * the whole document *
ref_type: npl
severity: background
Final disposition: pending (2021-12-29)