Patent ID: 11966837
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 0:
1. A method for compressing neural networks, executed by one or more processors, the method comprising:
receiving a neural network, wherein the neural network has been trained on a set of training data;
receiving a compression ratio; compressing the neural network layer-by-layer based on the compression ratio using an optimization model imposed at each layer of the neural network to solve fora set of sparse weights, wherein the optimization model is a two-step algorithm that decomposes a problem into a series of a master problem and subproblems and then solves the master problem by combining an alternating optimization algorithm and variance-reduction stochastic gradient decent algorithm to solve for the set of sparse weights for a maximally allowable number of nonzeros and wherein the set of sparse weights includes more zero values than an original set of weights for the neural network;
re-training the compressed neural network with the set of sparse weights; and
outputting the re-trained neural network.