Patent ID: 11948063
Assignee: D5AI LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A computer system comprising:
one or more processing cores; and
computer memory in communication with the one or more processing cores,
wherein the one or more processing cores are configured, through programming, to:
initially train, at least partially, a base neural network through machine learning with a training data set, wherein:
the base neural network comprises an input layer, an output layer, and one or more hidden layers between the input layer and the output layer;
each layer comprises one or more base network nodes, such that the base neural network comprises multiple base network nodes; and
preliminary activations values computed for the base network nodes on each of the one or more hidden layers for data in the training data set are stored in the computer memory;

after the initial training of the base neural network, merge a new node set into the base neural network to form an expanded neural network, wherein:
the new node set comprises one or more nodes; and
merging the new node set and the base neural network comprises directly connecting each of the one or more nodes of the new node set to one or more base network nodes in the expanded neural network; and

after merging the new node set into the base neural network to form the expanded neural network, train the expanded neural network through iterative machine learning on the training data set using a network error loss function for the expanded neural network, wherein the one or more processing cores are configured, through programming, to train the expanded neural network by imposing a node-to-node relationship regularization for at least one base network node in the expanded neural network, wherein the one or more processing cores are configured, through programming, to impose the node-to-node relationship regularization by adding, during back-propagation of partial derivatives through the expanded neural network for a datum in the training data set, a regularization cost to the network error loss function for the at least one base network node based on a specified relationship between a stored preliminary activation value for the base network node for the datum and an activation value for the base network node of the expanded neural network for the datum, wherein the preliminary activation value for the base network node of the base neural network for the datum was stored in the computer memory.