Patent application EP19206455A (EP), filed 2019-10-31
Publication number: EP3736726A1
Title: AUTOMATED CHATBOT LINGUISTIC EXPRESSION GENERATION
Abstract: Linguistic expressions for training a chatbot can be generated in an automated system via linguistic expression templates that are associated with intents. The pre-categorized linguistic expressions can then be used for training and validation. Chatbot development can thus be improved by having a large number of expressions for development, leading to a more robust chatbot. In practice, the process can iterate with modifications to the templates until a suitable benchmark is met. The technique can be applied across human languages to generate chatbots conversant in any number of languages and is applicable to a variety of domains.
IPC codes: G06F  16/  9032            A I
G06F  40/   216            A I
G06F  40/    30            A I
G06F  40/    56            A I
Claims: number: 1
text: A computer-implemented method of automated linguistic expression generation comprising:
number: 2
text: The method of claim 1 wherein:
number: 3
text: The method of claim 1 or 2 further comprising:
augmenting the generated linguistic expressions before training, wherein optionally the augmenting comprises:
number: 4
text: The method of any one of the proceeding claims further comprising:
number: 5
text: The method of any one of the proceeding claims wherein:
the benchmark value comprises one or more selected from the group consisting of:
number: 6
text: The method of any one of the proceeding claims further comprising:
selecting a first subset of the generated linguistic expressions for training, wherein optionally selecting a second subset of the generated linguistic expressions for use in validation.
number: 7
text: The method of any one of the proceeding claims wherein:
the syntax supports regular expressions.
number: 8
text: The method of any one of the proceeding claims wherein:
number: 9
text: The method of any one of the proceeding claims further comprising:
repeating the training and validation over multiple iterations as the linguistic expression generation templates are modified and the benchmark value converges on an acceptable value.
number: 10
text: The method of any one of the proceeding claims wherein the respective pre-categorized intents are according to an intent group of a given linguistic expression generation template from which the generated linguistic expressions were generated.
number: 11
text: One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed, cause a computing system to perform the method of any one of the proceeding claims.
number: 12
text: A computing system comprising:
number: 13
text: The computing system of claim 12 wherein the operations further comprise:
validating the chatbot.
number: 14
text: The computing system of claim 12 or 13 wherein:
number: 15
text: The computing system of any one of the claims 12 to 14 wherein the operations further comprise:
filtering the generated linguistic expressions based on an edit distance of the generated linguistic expressions, wherein optionally the filtering comprises:
removing at least one linguistic expression that has a close edit distance to one or more other linguistic expressions.
Office action: search_report, dated 2020-11-11
Prior art cited: ref_id: US2016196312A1
ref_type: patent
severity: background
ref_id: - JANA SCHWARZ ET AL: "Automatic Analysis of Real Dialogues and Generation of Training Corpora", vol. 3, pages 2201, XP007004823
ref_type: npl
severity: background
ref_id: - BINGFENG LUO ET AL: "Marrying up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 May 2018 (2018-05-15), XP080878301
ref_type: npl
severity: background
ref_id: KR101851785B1
ref_type: patent
severity: background
ref_id: - DESOT THIERRY ET AL: "Towards a French Smart-Home Voice Command Corpus: Design and NLU Experiments", 8 September 2018, 12TH EUROPEAN CONFERENCE ON COMPUTER VISION, ECCV 2012; [LECTURE NOTES IN COMPUTER SCIENCE], SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 509 - 517, ISBN: 978-3-642-36948-3, ISSN: 0302-9743, XP047526978
ref_type: npl
severity: background
ref_id: - MISHAKOVA ANASTASIIA ET AL: "Learning Natural Language Understanding Systems from Unaligned Labels for Voice Command in Smart Homes", 2019 IEEE INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND COMMUNICATIONS WORKSHOPS (PERCOM WORKSHOPS), IEEE, 11 March 2019 (2019-03-11), pages 832 - 837, XP033558252, DOI: 10.1109/PERCOMW.2019.8730721
ref_type: npl
severity: background
ref_id: - JOHN WIETING ET AL: "Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 6 June 2017 (2017-06-06), XP080767922, DOI: 10.18653/V1/D17-1026
ref_type: npl
severity: background
ref_id: - CHAO WANG ET AL: "Automatic induction of language model data for a spoken dialogue system", COMPUTERS AND THE HUMANITIES, KLUWER ACADEMIC PUBLISHERS, DO, vol. 40, no. 1, 8 November 2006 (2006-11-08), pages 25 - 46, XP019447930, ISSN: 1572-8412, DOI: 10.1007/S10579-006-9007-3
ref_type: npl
severity: background
ref_id: - CHRIS BROCKETT ET AL: "Support Vector Machines for Paraphrase Identification and Corpus Construction", 2005, XP055692677, Retrieved from the Internet <URL:https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/I05-50015B15D.pdf> [retrieved on 20200507]
ref_type: npl
severity: background
ref_id: - UDIT CHANDNA ET AL: "System for Semi-Automated Chatbots Query Classification Training Corpus Generation", PROCEEDINGS OF 24TH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING AND COMMUNICATIONS (ADCOM 2018), 22 September 2018 (2018-09-22), XP055691719, Retrieved from the Internet <URL:https://accsindia.org/downloads/ADCOM-2018-papers/ADCOM_2018_paper_18.pdf> [retrieved on 20200505]
ref_type: npl
severity: background
ref_id: - ELENA MANISHINA ET AL: "Automatic Corpus Extension for Data-Driven Natural Language Generation", PROCEEDINGS OF THE TENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC'16), May 2016 (2016-05-01), pages 3624 - 3631, XP055691893, Retrieved from the Internet <URL:https://www.aclweb.org/anthology/L16-1575.pdf> [retrieved on 20200506]
ref_type: npl
severity: background
ref_id: - ANONYMOUS: "Training, test, and validation sets - Wikipedia", 29 November 2016 (2016-11-29), XP055458148, Retrieved from the Internet <URL:https://en.wikipedia.org/w/index.php?title=Training,_test,_and_validation_sets&oldid=752127502> [retrieved on 20180309]
ref_type: npl
severity: background
ref_id: KR101851785B
ref_type: patent
severity: background
ref_id: - DESOT THIERRY ET AL, Towards a French Smart-Home Voice Command Corpus: Design and NLU Experiments, 12TH EUROPEAN CONFERENCE ON COMPUTER VISION, ECCV 2012; [LECTURE NOTES IN COMPUTER SCIENCE], SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 509 - 517, (20180908), ISSN 0302-9743, ISBN 978-3-642-36948-3, XP047526978 [I] 1-15 * the whole document *
ref_type: npl
severity: background
ref_id: - MISHAKOVA ANASTASIIA ET AL, "Learning Natural Language Understanding Systems from Unaligned Labels for Voice Command in Smart Homes", 2019 IEEE INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND COMMUNICATIONS WORKSHOPS (PERCOM WORKSHOPS), IEEE, (20190311), doi:10.1109/PERCOMW.2019.8730721, pages 832 - 837, XP033558252 [A] 2,14 * abstract * * section II. C. 1) *
ref_type: npl
severity: background
ref_id: - JOHN WIETING ET AL, "Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20170606), doi:10.18653/V1/D17-1026, XP080767922 [A] 3 * abstract *
ref_type: npl
severity: background
ref_id: - CHAO WANG ET AL, "Automatic induction of language model data for a spoken dialogue system", COMPUTERS AND THE HUMANITIES, KLUWER ACADEMIC PUBLISHERS, DO, (20061108), vol. 40, no. 1, doi:10.1007/S10579-006-9007-3, ISSN 1572-8412, pages 25 - 46, XP019447930 [A] 4,15 * abstract * * section 4.2 *
ref_type: npl
severity: background
ref_id: - Chris Brockett ET AL, "Support Vector Machines for Paraphrase Identification and Corpus Construction", (2005), URL: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/I05-50015B15D.pdf, (20200507), XP055692677 [A] 4,15 * abstract * * section 3.2 *
ref_type: npl
severity: background
ref_id: - Udit Chandna ET AL, "System for Semi-Automated Chatbots Query Classification Training Corpus Generation", Proceedings of 24th International Conference on Advanced Computing and Communications (ADCOM 2018), (20180922), URL: https://accsindia.org/downloads/ADCOM-2018-papers/ADCOM_2018_paper_18.pdf, (20200505), XP055691719 [A] 1,11,12 * abstract *
ref_type: npl
severity: background
ref_id: - Elena Manishina ET AL, "Automatic Corpus Extension for Data-Driven Natural Language Generation", Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), (201605), pages 3624 - 3631, URL: https://www.aclweb.org/anthology/L16-1575.pdf, (20200506), XP055691893 [A] 1,11,12 * abstract *
ref_type: npl
severity: background
Final disposition: pending (2020-11-11)