Patent application EP19206110A (EP), filed 2019-10-30
Publication number: EP3696711A1
Title: CONTEXTUALIZED TEXT DESCRIPTION
Abstract: Various examples are directed to systems and methods for classifying text. A computing device may access, from a database, an input sample comprising a first set of ordered words. The computing device may generate a first language model feature vector for the input sample using a word level language model and a second language model feature vector for the input sample using a partial word level language model. The computing device may generate a descriptor of the input sample using a target model, the input sample, the first language model feature vector, and the second language model feature vector and write the descriptor of the input sample to the database.
IPC codes: G06F  40/   284            A I
G06F  40/   289            A I
G06F  40/    30            A I
Claims: number: 1
text: A computerized system for classifying text, comprising:
number: 2
text: The computerized system of claim 1, wherein the target model is a partial word level model, and wherein the descriptor of the input sample describes a first partial word of the input sample.
number: 3
text: The computerized system of claim 2, wherein generating the first language model feature vector comprises generating a first word level feature vector using the word level language model and a first word of the input sample, the first word including the first partial word, and wherein, preferably, the first language model feature vector comprises the first word level feature vector.
number: 4
text: The computerized system of claim 3, wherein generating the first language model feature vector comprises:
number: 5
text: The computerized system of any one of the preceding claims, wherein the target model is a word level model and wherein the descriptor of the input sample describes a first word of the input sample.
number: 6
text: The computerized system of claim 5, the operations further comprising generating a first partial word feature vector using the partial word level language model and a first partial word, wherein the first word comprises the first partial word, and
the operations, preferably, further comprising:
number: 7
text: The computerized system of claim 6, wherein the partial word level language model is a forward model, wherein the first partial word is an initial partial word of the first word, the operations further comprising selecting the first partial word feature vector as the second language model feature vector.
number: 8
text: The computerized system of claim 6, the operations further comprising executing an attention mechanism to select either the first partial word or the second partial word feature vector as the second language model feature vector.
number: 9
text: The computerized system of any one of the preceding claims, wherein the partial word level language model is a sub-word level model and the operations further comprising generating a third language model feature vector for the input sample using a character level model, wherein the generating of the descriptor of the input sample is also based on the third language model feature vector.
number: 10
text: The computerized system of any one of the preceding claims, the operations further comprising:
number: 11
text: A method for classifying text, the method comprising:
number: 12
text: The method of claim 11, wherein the target model is a partial word level language model, and wherein the descriptor of the input sample describes a first partial word of the input sample.
number: 13
text: The method of claim 12, wherein generating the first language model feature vector comprises generating a first word level feature vector using the word level language model and a first word of the input sample, the first word including the first partial word, and wherein, preferably, the first language model feature vector comprises the first word level feature vector, and
wherein, preferably, generating the first language model feature vector comprises:
number: 14
text: The method of any one of claims 11-13, wherein the target model is a word level model and wherein the descriptor of the input sample describes a first word of the input sample.
number: 15
text: A non-transitory machine-readable medium comprising instructions thereon that, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 11-14.
Office action: search_report, dated 2020-08-19
Prior art cited: ref_id: - YONGHUI WU ET AL: "Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation", 8 October 2016 (2016-10-08), XP055542980, Retrieved from the Internet <URL:https://arxiv.org/pdf/1609.08144.pdf> [retrieved on 20190116]
ref_type: npl
severity: background
ref_id: - CICERO NOGUEIRA DOS SANTOS ET AL: "Boosting Named Entity Recognition with Neural Character Embeddings", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 19 May 2015 (2015-05-19), XP081338669
ref_type: npl
severity: background
ref_id: - JORGE A BALAZS ET AL: "Gating Mechanisms for Combining Character and Word-level Word Representations: An Empirical Study", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 11 April 2019 (2019-04-11), XP081167951
ref_type: npl
severity: background
ref_id: - GUILLAUME LAMPLE ET AL: "Neural Architectures for Named Entity Recognition", PROCEEDINGS OF THE 2016 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES, 1 January 2016 (2016-01-01), Stroudsburg, PA, USA, pages 260 - 270, XP055498130, DOI: 10.18653/v1/N16-1030
ref_type: npl
severity: background
ref_id: - MINH-THANG LUONG ET AL: "Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models", PROCEEDINGS OF THE 54TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (VOLUME 1: LONG PAPERS), 7 August 2016 (2016-08-07), Stroudsburg, PA, USA, pages 1054 - 1063, XP055691128, DOI: 10.18653/v1/P16-1100
ref_type: npl
severity: background
ref_id: - ZHILIN YANG ET AL: "Words or Characters? Fine-grained Gating for Reading Comprehension", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 6 November 2016 (2016-11-06), XP080729645
ref_type: npl
severity: background
ref_id: - YASUMASA MIYAMOTO ET AL: "Gated Word-Character Recurrent Language Model", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 6 June 2016 (2016-06-06), XP080706108
ref_type: npl
severity: background
ref_id: - PIOTR BOJANOWSKI ET AL: "Enriching Word Vectors with Subword Information", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 July 2016 (2016-07-15), XP080716101
ref_type: npl
severity: background
ref_id: - JAN A BOTHA ET AL: "Compositional Morphology for Word Representations and Language Modelling", PROCEEDINGS OF THE 31ST INTERNATIONAL CONFERENCE ON MACHINE LEARNING, 2014, Beijing, China, XP055691138, Retrieved from the Internet <URL:http://proceedings.mlr.press/v32/botha14.pdf> [retrieved on 20200504]
ref_type: npl
severity: background
ref_id: - ZHILIN YANG ET AL, "Words or Characters? Fine-grained Gating for Reading Comprehension", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20161106), XP080729645 [I] 1-15 * abstract *
ref_type: npl
severity: background
ref_id: - YASUMASA MIYAMOTO ET AL, "Gated Word-Character Recurrent Language Model", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20160606), XP080706108 [I] 1-15 * abstract *
ref_type: npl
severity: background
ref_id: - PIOTR BOJANOWSKI ET AL, "Enriching Word Vectors with Subword Information", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20160715), XP080716101 [A] 1,11,15 * abstract *
ref_type: npl
severity: background
ref_id: - Jan A Botha ET AL, "Compositional Morphology for Word Representations and Language Modelling", Proceedings of the 31st International Conference on Machine Learning, Beijing, China, (2014), URL: http://proceedings.mlr.press/v32/botha14.pdf, (20200504), XP055691138 [A] 1,11,15 * abstract *
ref_type: npl
severity: background
Final disposition: pending (2020-08-19)