Patent ID: 11972344
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 23:
24. A computer-implemented method for improving a simple model using a confidence profile, the method comprising:
transferring information from a pre-trained complex model including a neural network to the simple model by:
generating, using a linear probe, confidence scores through flattened intermediate representations of the neural network and learning a regularized neural network that inputs same confidence scores; and
wherein the generating generates the confidence scores by:
attaching and training the linear probe on the flattened intermediate representations of a neural network;
training the simple model on an original dataset;
learning the weights for examples in the original dataset, by defining a regularization term in the regularized neural network to keep the weights from all going to zero, as a function of the simple model and the linear probes; and
retraining the simple model on a final weighted dataset while minimizing a loss of the simple model,
wherein the simple model includes a lasso model, and
wherein the linear probe is added at certain layers of the neural network, the certain layers representing logical units; and

computing an area under the curve (AUC) traced by the confidence scores at each linear probe for a given input-output pair, wherein the confidence scores are for a true label of a given example instead of a predicted label.