Patent ID: 11900260
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 10:
11. A method, comprising:
providing an integrated system comprising:
a teacher neural network comprising, in series:
a first teacher sub-network adapted to generate a first intermediate feature map based on a training-mode data input to the teacher neural network; and
a final teacher sub-network adapted to generate teacher inference data based on the first intermediate feature map; and

a first student sub-network adapted to generate first student inference data based on the first intermediate feature map; and

training the integrated system by:
propagating the training-mode data input forward through the teacher neural network to generate the first intermediate feature map and teacher inference data;
propagating the first intermediate feature map forward through the first student sub-network to generate first student inference data;
calculating a first knowledge distillation loss based on a knowledge distillation loss function applied to the first student inference data and the teacher inference data; and
propagating the first knowledge distillation loss backward through the first student sub-network and the first teacher sub-network to train the first student sub-network and the first teacher sub-network.