METHOD AND APPARATUS FOR GENERATING EQUIVALENT NEURAL NETWORKS BY DATA MANAGEMENT

The present disclosure provides method and apparatus for generating equivalent neural network. The apparatus: training a first neural network based on a training data set; generating a second neural network by: embedding the training data set into the first neural network, and (1) removing at least one first element from the training data set to obtain a first data set and embedding the first data set into the first neural network to generate the second neural network, (2) inserting at least one second element into the training data set to obtain a second data set and embedding the second data set into the first neural network to generate the second neural network, or (3) updating the training data set by at least one third element to obtain a third data set and embedding the third data set into the first neural network to generate the second neural network.

TECHNICAL FIELD

The present disclosure relates to a method and apparatus for generating equivalent or approximated neural networks, particularly to a method and apparatus for generating equivalent or approximated neural networks by data management.

BACKGROUND

Neural network, one implementation of a machine learning model, is introduced for a computing device to solve complex problems like the human brain. Before being used, the neural network needs to be trained using a plurality of training data sets.

However, when different data sets are to be inserted, deleted or updated in a trained neural network, the neural network must be re-trained with the new training data set, which may be time and resource consuming. For example, regarding some opt-out policies (e.g., California Consumer Privacy Act, Colorado Consumer Protection Act, European General Data Protection Regulation, etc.), the users may re-decide the use range of the personal data they provided to the enterprise. Accordingly, the enterprise that uses the users' data for training their neural network, may need to re-train the neural network based on the re-decided use range of the personal data, and the cost could be very high.

SUMMARY

Some embodiments of the present disclosure provide a method for generating equivalent or approximated neural networks by data management. The method includes: training a first neural network based on a training data set; receiving a deletion request, an insertion request or a modification request from a network, wherein the deletion request indicates a removal of at least one first element from the training data set (also referred to as the unlearning process), the insertion request indicates an insertion of at least one second element into the training data set, and the modification request indicates a modification of at least one third element of the training data set; generating a second neural network equivalent to or approximated to the first neural network by using the embedding technique which associates data sets to neural networks by (1) removing the at least one first element from the training data set to obtain a first data set according to the deletion request and embedding the first data set into the first neural network to generate the equivalent or approximated second neural network, (2) inserting the at least one second element into the training data set to obtain a second data set according to the insertion request and embedding the second data set into the first neural network to generate the equivalent second neural network, or (3) updating the training data set by the at least one third element to obtain a third data set according to the modification request and embedding the third data set into the first neural network to generate the equivalent or approximated second neural network.

Some embodiments of the present disclosure provide an apparatus for generating equivalent or approximated neural networks by data management. The apparatus includes a transceiver, a processor, and a storage unit. The storing unit stores a program that, when being executed, cause the processor to: train a first neural network based on a training data set; receive, via the transceiver, a deletion request, an insertion request or a modification request from a network, wherein the deletion request indicates at least one first element to be removed from the training data set, the insertion request indicates at least one second element to be added into the training data set, and the modification request indicates at least one third element of the training data set to be updated; generate a second neural network equivalent to or approximated to the first neural network by using the embedding technique which associates data sets to neural networks by (1) removing the at least one first element from the training data set to obtain a first data set according to the deletion request and embedding the first data set into the first neural network to generate the equivalent or approximated second neural network, (2) inserting the at least one second element into the training data set to obtain a second data set according to the insertion request and embedding the second data set into the first neural network to generate the equivalent second neural network, or (3) updating the training data set by the at least one third element to obtain a third data set according to the modification request and embedding the third data set into the first neural network to generate or approximated the equivalent second neural network.

DETAILED DESCRIPTION

A neural network is introduced for the computing device to solve complex problems. However, re-training neural networks is time and resource-consuming. Therefore, to realize the concept of data ownership and data access rights, there is a need to develop new methods and apparatus that can efficiently generate a neural network equivalent or approximated to an existing neural network by manipulating different data sets on the given neural network. Further, updating neural networks through managing data makes it possible to cope efficiently with the opt-out and data minimization principle in data management to retain an identified or almost identified neural network to the original neural network.

FIG. 1A illustrates a block diagram of an apparatus 1 according to some embodiments of the present disclosure. The apparatus 1 includes a processor 11, a storage unit 13, and a transceiver 15. The processor 11, the storage unit 13, and the transceiver 15 are electrically coupled through a communication bus 17.

The communication bus 17 may allow the processor 11 to execute a program PG1 stored in the storing unit 13. When executed, the program PG1 may generate one or more interrupts (e.g., software-interrupt) to cause the processor 11 to perform functions of the program PG1 for generating an equivalent or approximated neural network. Descriptions of the functions of program PG1 are provided hereinafter.

In some embodiments, the apparatus 1 generates a first neural network M11 according to a plurality of training data sets T11. After generating the first neural network M11, the apparatus 1 stores the first neural network M11 in storing unit 13 for later use. In some implementations, the training data set T11 may be collected from users or training data databases. After collection, the training data set T11 may be pre-stored, for training purposes, in the storing unit 13 (as shown in FIG. 1A) or in an external database (not shown) such as external storage or cloud database.

FIG. 1B to 1E are schematic views of generating a neural network equivalent to or approximated to first neural network M11 according to some embodiments of the present disclosure. In FIG. 1B, the apparatus 1 generates a second neural network M12 equivalent to the first neural network M11 by embedding the training data set T11 into the first neural network M11 (i.e., making the first neural network M11 as an explicit function of the training data set T11) to generate the equivalent second neural network.

The method demonstrates adaptability by:

It should be noted that the method's responsiveness is a key feature, as the deletion request 80, the insertion request 82, and the modification request 84 are promptly received from a network via the transceiver 15, ensuring the system's real-time adaptability.

More specifically, although the training data T11 trains the first neural network M11, how the weights in the first neural network M11 are related to the training data set T11 is vague. Therefore, the apparatus 1 makes the relation between the weights in the first neural network M11 and training data T11 transparent (e.g., data embedment, deletion, insertion, and updating.) For example, based on some matrix calculations, the weight matrices of the first neural network M11 are expressed as a function of the training data T11. Based on the function, the apparatus 1 performs data management to generate the second neural network, M12, equivalent to or approximated to the first neural network, M11.

In some embodiments, the first neural network M11 is equivalent to the second neural network M12, which means that a first data outputted from the first neural network M11 by inputting an input data is equivalent to a second data outputted from the second neural network M12 by inputting the same input data. In other words, when the same data is respectively inputted into the first neural network M11 and the second neural network M12, the outputs will be equivalent (e.g., the same or substantially the same,) and thereof, the first neural network M11 and the second neural network M12 have the same performance.

In some embodiments, the first neural network M11 is approximated to the second neural network M12, which means that a first data outputted from the first neural network M11 by inputting an input data is approximated to a second data outputted from the second neural network M12 by inputting the same input data. In other words, when the same data is respectively inputted into the first neural network M11 and the second neural network M12, the outputs will be approximated (e.g., nearly the same,) and thereof, the first neural network M11 and the second neural network M12 have almost the same performances.

Neural Network System

FIG. 2A illustrates a block diagram of an apparatus 2 according to some embodiments of the present disclosure. The apparatus 2 includes a processor 21, a storage unit 23, and a transceiver 25. The processor 21, the storage unit 23, and the transceiver 25 are electrically coupled through a communication bus 27.

The communication bus 27 may allow the processor 21 to execute a program PG2 in the storing unit 23. When executed, the program PG2 may generate one or more interrupts (e.g., software interrupts) to cause the processor 21 to perform functions of the program PG2 for generating an equivalent or approximated neural network. The functions of the program PG2 will be further described hereinafter.

In some embodiments, the apparatus 2 generates a first neural network M21 according to a plurality of training data sets T21. After generating the first neural network M21, the apparatus 2 stores it in storing unit 23 for later use. In some implementations, the training data sets T21 may be collected from users or databases. After collection, the training data set T21 may be pre-stored, for training purposes, in the storing unit 23 (as shown in FIG. 2A) or in an external database (not shown) such as external storage or cloud database.

People skilled in the art of neural networks should easily understand the training procedure of neural networks (e.g., the first neural network M21) based on training data sets (e.g., the training data sets T21). Therefore, the details of the neural network training procedure will not be further described.

FIG. 2B is a schematic view of a system 5 utilizing the apparatus 2 according to some embodiments of the present disclosure. In some embodiments, the system 5 may include user devices 51 (e.g., mobile phone, laptop, personal computer, etc.) and the apparatus 2. The user devices 51 may communicate with the apparatus 2 via a network. Users may control the user device 51. In particular, when the users need to change the usage of the personal data utilized by the apparatus 2, the users may use the user devices 51 to transmit a deletion request 90, an insertion request 92, or a modification request 94 to the apparatus 2 through the network. Then, after receiving the deletion request 90, the insertion request 92, or the modification request 94 from the user devices 51, a curator/operator 21 of the apparatus 2 may perform corresponding data management operations (e.g., data deletion, data insertion, or data modification) to generate a second neural network M22 equivalent to or approximated to the first neural network M21. In some embodiments, the user of the user device 51 may initiate the request, and the operator/curator of the apparatus 2 may manipulate the data management operations.

For example, the apparatus 2 of the present disclosure is an enterprise server. The apparatus 2 may utilize personal data sets provided by the user to train the first neural network, M21. When the user wants to opt out of some of the provided personal data from the apparatus 2, the user transmits the deletion request 90 via the user device 51 to the apparatus 2 through the network. The deletion request 90 demands the apparatus remove indicated personal data from the first neural network M21 by some of the data management operations described below.

Data Embedment

FIG. 3A to 3C are schematic views of generating equivalent or approximated neural networks by embedding the training data set T21 into the first neural network M21 according to some embodiments of the present disclosure. FIG. 3A expresses the goal of replacing weight matrices W in M21 with data-dependent matrices AUT in M22, while FIGS. 3B and 3C demonstrate how to realize the goal.

In some embodiments, apparatus 2 generates a second neural network M22 by embedding the training data set T21 into the first neural network M21. The data embedment aims to express each weight matrix in the first neural network M21 as a function of the training data T21. In general, the weight matrix Wi in the first neural network M21 may be expressed as

where fembed,i and gembed,i are functions a neural network system can execute. The training data set T21 is embedded to express all weight matrices in M21 to obtain the second neural network M22.

In particular, when the first neural network M21 is an n-layer neural network, which may be represented as:

M is the first neural network M21. ρ is a non-linear activation function related to the neural network field. Wi is a trained weight matrix of the ith layer of the first neural network M21. The embedment expresses Wi as

Ai is a sparse matrix. Ui is an input to the (i+1)th layer of the first neural network M21 and an output from the ith layer and, therefore, may be represented as:

In these embodiments, the first neural network M21 is trained based on the training data sets T21, which means that the weight matrices W1 to Wn of layers of the first neural network M21 are known parameters. Further, letting U0=T21, according to formula (3), U1 to Un-1 are obtainable from the weight matrices W1 to Wn-1, and U0. A1 to An in the formula (2) are unknown parameters to be calculated.

More specifically, to generate a neural network equivalent to the first neural network M21 by embedding the training data set T21, the following operations are performed:

Therefore, according to formulas (2) and (3),

Because ρ and W1 are known parameters and U1 is obtained from ρW1X, A1 is calculable,

Because ρ, W2 and U1 are known parameters and U2 is obtained from ρW2U1, A2 is calculable,

Because ρ, W3 and U2 are known parameters and U3 is obtained from ρW3U2, A3 is calculable,

Accordingly, the second neural network, M22, is generated and represented as

N is the second neural network M22. Therefore, M is equal to N, which means that the first neural network, M21, is equivalent to the second neural network, M22. In brief, the training data set T21 is introduced to M (i.e., the first neural network M21) for calculating A1 to An in N so that N with determined A1 to An is equivalent to M.

In some embodiments, calculating Ai is based on the following formula:

The above formula is one possible way to obtain Ai. However, it does not limit the ways to obtain Ai.

The above operations of data embedment may be implemented based on the below pseudo code below.

Algorithm Data embedment

Input: The n-layer neural network M= M21 trained with the training data

5:
end for

Data Deletion

FIG. 4A to 4C are schematic views of generating an equivalent or approximated neural network by removing the data set from the first neural network M21 according to some embodiments of the present disclosure. FIG. 4A expresses the goal of replacing weight matrices Wi in M21 with data-dependent matrices A′HT in M22, while FIGS. 4B and 4C demonstrate how to realize the goal. In some embodiments, after receiving the deletion request 90 from the user device 51, the apparatus 2 removes at least one first element from the training data set T21 to obtain a first data set D21 according to the deletion request 90 and embeds the first data set D21 into the first neural network M21 to generate the equivalent or approximated second neural network M22.

In some embodiments, in response to the deletion request, any trained weight matrix Wi in the first neural network M21 is expressed as the following functions of D21 and Hi-1

fembed,i and gembed,i are functions that a neural network system can execute. The first data set D21 is the dataset after removing the at least one first element from the training data set T21. The first data set D21 is embedded to obtain the second neural network M22.

More specifically, when the first neural network M21 is an n-layer neural network as formula (1). M is the first neural network M21. ρ is a non-linear activation function related to the neural network field. Wi is a trained weight matrix of the ith layer of the first neural network M21. In response to the deletion request, expressing Wi as

A′i is a sparse matrix. Hi is an input to the (i+1)th layer of the first neural network M21 and an output from the ith layer, and therefore may be represented as:

In these embodiments, the first neural network, M21, is trained based on the training data sets T21, which means that the weight matrices W1 to Wn in layers of the first neural network, M21, are known parameters. Further, letting H0=D21, according to formula (5), H1 to Hn-1 are obtainable from weight matrices W1 to Wn-1, and H0. A′1 to A′n in the formula (4) are unknown parameters to be calculated.

More specifically, to generate a neural network equivalent or approximated to the first neural network M21 by embedding the first data set D21, the following operations are performed:

Therefore, according to formulas (4) and (5),

Because ρ and W1 are known parameters and H1 is obtainable from ρW1Y, A′1 is calculable,

Because ρ, W2 and H1 are known parameters and H2 is obtainable from ρW2H1, A′2 is calculable,

Because ρ, W3 and H2 are known parameters and H3 is obtainable from ρW3H2, A′3 is calculable,

Accordingly, the second neural network, M22, is generated and represented as

N′ is the second neural network M22. Therefore, M is equal to N′, which means that the first neural network M21 is equivalent to the second neural M22. In brief, the system embeds H0 (i.e., the first data set D21) to M (i.e., the first neural network M21) to drive A′ to A′n in N, which means that H0 is used for determining A′1 to A′n, so that N′ with determined A′1 to A′n is equivalent to M. If the removed data set is larger than the system's minimum requirement, then N approximates M. Section Data Minimization presents the approximation procedure.

In some embodiments, calculating A′i is based on the following formula:

The above formula is one possible way to obtain A′i. However, it does not limit the ways to obtain A′i.

It should be noted that the above operations of data deletion may be implemented based on the below pseudo code below.

Algorithm Data deletion

Input: The neural network M21, trained by the training data set T21, and

the at least one first element to be removed from T21

1:
Delete the one first element from training data set T21, and the

remaining data set D21 is set to H0

5:
end for

neural network M22

Data Minimization

The principle of data minimization means that the apparatus 2's data controller(s) may collect only the data they need and keep it only for as long as necessary. Applying the principle means that the controller may control the minimum training data to retain the equivalent neural networks.

FIG. 5A to 5C are schematic views of generating equivalent neural network by embedding the minimum data set in the first neural network M21, according to some embodiments of the present disclosure. FIG. 5A expresses the goal of replacing weight matrices W in M21 with data-dependent matrices BVT in M22 while FIGS. 5B and 5C demonstrate how to realize the goal.

To meet the principle of data minimization, in some embodiments, the minimum sub-data set V0 from the training data set T21 is utilized to fulfill the following formula:

W
   i
  
  =
  
   
    B
    i
   
   ⁢
   
    
     V
     
      i
      -
      1
     
     T
    
    .

V0 is the minimum sub-data set of the training data set T21, and the corresponding Bi can be derived to fulfill

W
   i
  
  =
  
   
    B
    i
   
   ⁢
   
    V
    
     i
     -
     1
    
    T

for all 1. In that case, the sub-data V0 can replace the training data set T21 embedding in the second neural network M22 to obtain an equivalent neural network that satisfies the data minimization principle. In particular, expressing Wi as

Bi is a sparse matrix. Vi is an input to the (i+1)th layer of the first neural network M21 and output from the ith layer and, therefore, may be represented as:

In the embodiments, V0 is a sub-data set of the training data set T21 that has corresponding Bi for all i. According to formula (7), V1 to Vn-1 is obtainable from the weight matrices W1 to Wn-1, and V0. B1 to Bn in the formula (6) are unknown parameters to be calculated.

More specifically, to generate a neural network equivalent to the first neural network M21 by embedding the minimum sub-data set of the training data set T21, the following operations are performed:

Therefore, according to formulas (6) and (7),

Because ρ and W1 are known parameters and V1 is obtainable from ρW1X′, B1 is calculable,

Because ρ, W2 and V1 are known parameters and V2 is obtainable from ρW2V1, B2 is calculable,

Because ρ, W3 and V2 are known parameters and V3 is obtainable from ρW3V2, B3 is calculable,

Accordingly, the second neural network, M22, is generated and represented as

N2 is the second neural network M22. Therefore, M equals N2, meaning that the first neural network, M21, is equivalent to the second neural network, M22.

In other words, after embedding the sub-data set V0 into the first neural network M21, the second neural network M22 is equivalent to the first neural network M21. Moreover, V0 is the minimum training data set that can be embedded into the first neural network M21 to generate the second neural network M22 equivalent to the first neural network M21.

In some embodiments, because V1 is the smallest sub-data set of the training data set T21 that can derive the corresponding B1 to fulfill

W
   i
  
  =
  
   
    B
    i
   
   ⁢
   
    V
    
     i
     -
     1
    
    T

for all $i$, there is no sub-data set {circumflex over (V)}i of training data set T21 smaller than Vi to fulfill

In some embodiments, a specific sub-data set Si of the training data set T21 is smaller than Vi that the corresponding Ci can be found with

W
   i
  
  ≅
  
   
    C
    i
   
   ⁢
   
    S
    
     i
     -
     1
    
    T

so that the corresponding second neural network M22 generated based on

C
   i
  
  ⁢
  
   S
   
    i
    -
    1
   
   T

is an approximation to the first neural network M21.

In some embodiments, calculating Ci is based on the following formula:

The above formula is one possible way to obtain Ci. However, it is not intended to limit how the Ci is obtained.

In other words, after embedding the sub-data set S0, which is smaller than the minimum sub-data set in T21, into the first neural network M21, the second neural network M22 is an approximation to the first neural network M21.

Data Insertion

FIG. 6A to 6C are schematic views of generating an equivalent neural network by adding a data set in the first neural network M21 according to some embodiments of the present disclosure. FIG. 6A expresses the goal of replacing weight matrices W in M21 with data-dependent matrices B′V′T in M22 while FIGS. 6B and 6C demonstrate how to realize the goal. In some embodiments, after receiving the insertion request 92 from the user device 51, the apparatus 2 adds at least one second element to the training data set T21 to obtain a second data set D22 according to the insertion request 92 and embeds the second data set D22 into the first neural network M21 to generate the equivalent second neural network M22.

In some embodiments, in response to the insertion request, any trained weight matrix Wi in the first neural network M21 can be represented as a function of V′i-1

fembed,i and gembed,i are functions that a neural network system can execute. The second data set D22 is obtained by adding the at least one second element to the training data set T21. The second data set D22 is embedded to obtain the second neural network M22.

More specifically, when the first neural network M21 is an n-layer neural network, as expressed in formula (1). M is the first neural network M21. ρ is a non-linear activation function related to the neural network field. Wi is a trained weight matrix of the ith layer of the first neural network M21. In response to the insertion request, expressing Wi as

B′i is a sparse matrix. V′i is an input to the (i+1)th layer of the first neural network M21 and an output from the ith layer, and therefore may be represented as:

In these embodiments, the first neural network M21 is trained based on the training data sets T21, which means that the weight matrices W1 to Wn of layers of the first neural network M21 are known parameters. Further, letting

according to formula (9), V′1 to V′n-1 is obtainable from the weights W1 to Wn-1, and

B′1 to B′n in the formula (8) are unknown parameters to be calculated.

More specifically, to generate a neural network equivalent to the first neural network M21 by embedding the second data set D22, the following operations are performed

which is input I to M:

Therefore, according to formulas (8) and (9),

Because ρ and W1 are known parameters and V′1 is obtainable from ρW1Y′, B′1 is calculable,

Because ρ, W2 and V′1 are known parameters and V′2 is obtainable from ρW2V′1, B′2 is calculable,

Because ρ, W3 and V′2 are known parameters and V′3 is obtainable from ρW3V′2, B′3 is calculable,

Accordingly, the second neural network M22 is generated and represented as:

N″ is the second neural network M22. Therefore, M is equal to N″, which means that the first neural network M21 is equivalent to the second neural network M22. In brief, the system embeds V′0 (i.e., the second data set D22) in M (i.e., the first neural network M21) to derive B′1 to B′n of N″, which means that V′0 is used for determining B′1 to B′n so that N″ with determined B′1 to B′n is equivalent to M. In some embodiments, calculating B′i is based on the following formula:

The above formula is one possible way to obtain B′i. However, it does not limit the ways to obtain B′i.

The above operations of data insertion may be implemented based on the below pseudo code below.

Algorithm Data insertion

Input: The n-layer neural network M trained with the training data set

T21 , and U″0 the at least one second element needed to be added into M

1:
Insert U″0 the at least one second element into training data set T21

to obtain the resultant data set V'0=D22

5:
end for

Data Modification

FIG. 7A to 7C are schematic views of generating an equivalent neural network by changing the data set in the first neural network M21 according to some embodiments of the present disclosure. FIG. 7A expresses the goal of replacing weight matrices W in M21 with data-dependent matrices A″KT in M22 while FIGS. 7B and 7C demonstrate how to realize the goal. In some embodiments, after receiving the modification request 94 from the user device 51, the apparatus 2 updates the training data set T21 by at least third element to obtain a third data set D23 according to the modification request 92 and embeds the third data set D23 into the first neural network M21 to generate the equivalent second neural network M22. Note that the request to modify the training data set T21 may render a result insufficient for generating an identical neural network to M21. For example, if many training data sets become the same after modification, then the involved data is useless because they are the same training data. In that case, modification request 94 may be dealt with, like the data deletion, to obtain an approximation of the neural network M22 to the first neural network M21.

In some embodiments, in response to the data modification request, expressing trained weight matrix Wi in the first neural network M21 as a function of Ki-1 as follows:

fembed,i and gembed,i are functions that a neural network system can execute. The third data set D23 is the result of modifying the training data set T21 with the at least one third element. Note that the at least on third element is a subset of training data set T21. The third data set D23 is embedded to obtain the second neural network M22.

More specifically, when the first neural network M21 is an n-layer neural network, it may be represented as the formula (1). M is the first neural network M21. ρ is a non-linear activation function related to the neural network field. Wi is a trained weight of the ith layer of the first neural network M21. In response to the data modification request, expressing Wi as follows

A″i is a sparse matrix. Ki is input to the (i+1)th layer of the first neural network M21 and output from the ith layer and, therefore, may be represented as:

In these embodiments, the first neural network M21 is trained based on the training data sets T21, which means that the weight matrices W1 to Wn of layers of the first neural network M21 are known parameters. Further, letting D23=K0, according to formula (11), K1 to Kn-1 is obtainable from the weight matrices W1 to Wn-1, and K0. A″1 to A″n in the formula (10) are unknown parameters to be calculated.

More specifically, to generate a neural network equivalent to the first neural network M21 by embedding the third data set D23, the following operations are performed:

Therefore, according to formulas (10) and (11),

Because ρ and W1 are known parameters and K1 is obtainable from ρW1Z, A″1 is calculable,

Because ρ, W2 and K1 are known parameters and K2 is obtainable from ρW2K1, A″2 is calculable,

Because ρ, W3 and K2 are known parameters and K3 is obtainable from ρW3K2, A″3 is calculable,

Accordingly, the second neural network M22 is generated and represented as:

N″ is the second neural network M22. Therefore, M is equal to N″, which means that the first neural network M21 is equivalent to the second neural M22. In brief, the system embeds K0 (i.e., the third data set D23) in M (i.e., the first neural network M21) to derive A″1 to A″n of N″, which means that K0 is used for determining A″1 to A″n so that N′″ with determined A″1 to A″n is equivalent to M.

In some embodiments, calculating A″i is based on the following formula (however, it does not limit the ways to obtain A″i):

The above operations of data modification may be implemented based on the below pseudo code below.

Algorithm Data modification

Input: The n-layer neural network M trained with the training data T21 and

the at least one third element indicating the sub-set in T21 to be modified

1:
Update the training data set T21 with the at least one third element to

obtain the resultant data set K0=D23

5:
end for

neural network M2

In some embodiments, updating the training data set T21 with the at least one third element may include replacing element(s) of the training data set T21 by the at least one third element.

Illustrative Processes

Some embodiments of the present disclosure include a method for generating an equivalent or approximated neural network by data management, and a flowchart diagram thereof is shown in FIG. 8. The method of some embodiments is for use in an apparatus (e.g., the apparatus of the aforesaid embodiments). Detailed steps of the method are described below.

Step S801 is executed, by the apparatus, to train a first neural network based on a training data set. Step S802 is executed, by the apparatus, to receive a deletion request, an insertion request, or a modification request from a network. The deletion request indicates at least one first element to be removed from the training data set. The insertion request indicates at least one second element to be added to the training data set. The modification request indicates at least one third element in the training data set to be updated.

Step S803 is executed, by the apparatus, to generate a second neural network equivalent to or approximated to the first neural network by (1) removing the at least one first element from the training data set to obtain a first data set according to the deletion request and embedding the first data set into the first neural network to generate the equivalent or approximated second neural network (the latter appears if the remaining training data set after removing the first data set is smaller than the minimum data set requirement to derive an equivalent neural network), (2) inserting the at least one second element into the training data set to obtain a second data set according to the insertion request and embedding the second data set into the first neural network to generate the equivalent second neural network, or (3) updating the training data set by the at least one third element to obtain a third data set according to the modification request and embedding the third data set into the first neural network to generate the second neural network.

Experimental Data

In an example experience, the data includes forty people's face images and ten face images for each person. The training dataset T21 comprises 330 photos, and the remaining 70 photos are testing images. The original neural network M21 trained by the training data T21 is to recognize whether a person wears glasses by face image(s). Each experimental turn removes two persons' face images from the training data set T21 to obtain training data set D11. For each run, we compared the results of two neural networks. One of them, M22, is according to the present disclosure's method, and the other is for benchmark comparison purposes by training a new neural network, M21, using the rest of the face images, which is now training data D11. According to FIG. 9, the test accuracy versus the number of faces in training according to the method of the present disclosure (presented by line L1) is more precise than that of the compared network (presented by line L2). Besides the accuracy, the compared process re-trained the network from scratch. In contrast, the method of the present disclosure involves only matrix operations for obtaining equivalent neural networks, which are more efficient and require much less time and resources.

It shall be appreciated that the processors mentioned in the above embodiments may be a central processing unit (CPU), other hardware circuit elements capable of executing relevant instructions, or a combination of computing circuits that are well-known by those skilled in the art based on the above disclosures.

Moreover, the storing units mentioned above may include memories, such as ROM, RAM, etc., or storage devices, such as flash memory, HDD, SSD, etc., for storing data. Further, the communication buses mentioned in the above embodiments may include a communication interface for transferring data between the elements, such as the processor, the storing unit, the sensor, and the alert element. They may include an electrical bus interface, an optical bus interface, or even a wireless bus interface. However, such a description is not intended to limit the hardware implementation embodiments of the present disclosure.