Patent application EP21177789A (EP), filed 2021-06-04
Publication number: EP3920102A1
Title: MACHINE LEARNING METHOD AND MACHINE LEARNING SYSTEM INVOLVING DATA AUGMENTATION
Abstract: A machine learning method includes steps of: (a) obtaining initial values of hyperparameters and hypernetwork parameters; (b) generating first classification model parameters according to the hyperparameters and the hypernetwork parameters, and updating the hypernetwork parameters according to a classification result based on the first classification model parameters relative to a training sample; (c) generating second classification model parameters according to the hyperparameters and the updated hypernetwork parameters, and updating the hyperparameters according to another classification result based on the second classification model parameters relative to a verification sample; and (d) repeating the steps (b) and (c) for updating the hypernetwork parameters and the hyperparameters.
IPC codes: G06N  20/    00            A I
G06N   3/    04            A I
G06N   3/    08            A I
G06N   5/    00            A I
G06N   3/   045            A I
G06N   3/    09            A I
G06N   3/  0985            A I
G06N   5/    01            A I
Claims: number: 1
text: A machine learning system (100), characterized by comprising:
number: 2
text: The machine learning system of claim 1, wherein the operation (a2) executed by the processing unit comprises:
number: 3
text: The machine learning system of claim 2, wherein each of the exploring classification models (146e1, 146e2, 146e3, 146e4) comprises a plurality of neural network structural layers (SL1∼SLn), the neural network structural layers are divided into a first structural layer portion (P1) and a second structural layer portion (P2) after the first structural layer portion, each of the exploration classification model parameters for forming the exploring classification models comprises a first weight parameter content and a second weight parameter content, the first weight parameter content is configured to determine operations of the first structural layer portion, and the second weight parameter content is configured to determine operations of the second structural layer portion.
number: 4
text: The machine learning system of claim 3, wherein the second weight parameter contents applied to the second structural layer portions of the exploring classification models are the same, the second structural layer portions of the exploring classification models are operating with the same logic.
number: 5
text: The machine learning system of claim 3, wherein the first structural layer portion in each of the exploring classification models comprises at least one first convolutional layer (SL1), the first convolutional layers among the exploring classification models have different weight parameters from each other, the second structural layer portion in each of the exploring classification models comprises at least one second convolutional layer (SL4) and at least one fully connection layer (SLn), the second convolutional layers and the fully connection layers among the exploring classification models have same weight parameters across the exploring classification models.
number: 6
text: The machine learning system of claim 1, wherein the operation (b) (S230) executed by the processing unit comprises:
number: 7
text: A machine learning method (200), characterized by comprising:
number: 8
text: The machine learning method of claim 7, wherein the step (b2) comprises:
number: 9
text: The machine learning method of claim 8, wherein in the step (b4):
calculating the first losses by a cross-entropy calculation between the first prediction labels of the exploring classification models and the training label respectively.
number: 10
text: The machine learning method of claim 8, wherein each of the exploring classification models comprises a plurality of neural network structural layers, the neural network structural layers are divided into a first structural layer portion and a second structural layer portion after the first structural layer portion, each of the exploration classification model parameters for forming the exploring classification models comprises a first weight parameter content and a second weight parameter content, the first weight parameter content is configured to determine operations of the first structural layer portion, and the second weight parameter content is configured to determine operations of the second structural layer portion.
number: 11
text: The machine learning method of claim 10, wherein the second weight parameter contents applied to the second structural layer portions of the exploring classification models are the same, the second structural layer portions of the exploring classification models are operating with the same logic.
number: 12
text: The machine learning method of claim 10, wherein the first structural layer portion in each of the exploring classification models comprises at least one first convolutional layer, the first convolutional layers among the exploring classification models have different weight parameters from each other, the second structural layer portion in each of the exploring classification models comprises at least one second convolutional layer and at least one fully connection layer, the second convolutional layers and the fully connection layers among the exploring classification models have same weight parameters across the exploring classification models.
number: 13
text: The machine learning method of claim 7, wherein the step (c) comprises:
number: 14
text: The machine learning method of claim 13, wherein the step (c3) comprises:
calculating the second loss by a cross-entropy calculation between the second prediction label and the verification label.
number: 15
text: A non-transitory computer-readable storage medium (120), characterized by storing at least one instruction program executed by a processor (140) to perform a machine learning method (200), the machine learning method comprising:
Office action: search_report, dated 2021-12-08
Prior art cited: ref_id: - FELIPE PETROSKI SUCH ET AL: "Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 17 December 2019 (2019-12-17), XP081560001
ref_type: npl
severity: background
ref_id: - HA DAVID ET AL: "Hypernetworks", 1 December 2016 (2016-12-01), XP055773067, Retrieved from the Internet <URL:https://arxiv.org/pdf/1609.09106.pdf> [retrieved on 20210208]
ref_type: npl
severity: background
ref_id: - MATTHEW MACKAY ET AL: "Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 7 March 2019 (2019-03-07), XP081130759
ref_type: npl
severity: background
ref_id: - FELIPE PETROSKI SUCH ET AL, "Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20191217), XP081560001 [I] 1-15 * abstract; figures 1, 3 * * page  2, line  24  - line  33 * * page  2, line  42  - page  4, line  37 * * page  5, line  7  - line  13 * * page  5, line  20  - page  6, line  8 * * page  6, line  33  - page  7, line  18 *
ref_type: npl
severity: background
ref_id: - Ha David ET AL, "Hypernetworks", (20161201), URL: https://arxiv.org/pdf/1609.09106.pdf, (20210208), XP055773067 [A] 1-15 * abstract; figure 1 * * page  1, line  1  - page  2, line  13 * * page  3, line  3  - page  6, line  39 *
ref_type: npl
severity: background
ref_id: - MATTHEW MACKAY ET AL, "Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20190307), XP081130759 [A] 1-15 * abstract; figure 1; table 3 * * page  2, line  1  - page  3, line  15 * * page  3, line  30  - page  5, line  28 * * page  6, line  22  - page  7, line  16 * * page  9, line  7  - line  26 * * page  20, line  20  - line  41 *
ref_type: npl
severity: background
Final disposition: pending (2021-12-08)