Patent Description:
Machine learning functionality is becoming more widely used in many of today's applications. Machine learning algorithms may be classified by how they are trained. Supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning are examples of training techniques. Training data is used for training the machine learning algorithm. A machine learning model is what is learned from training with the training data, and contains a parameter set for the machine learning algorithm. A neural network may be used in machine learning. Neural networks are used in the areas of supervised learning and reinforcement learning. The effectiveness of a machine learning algorithm is influenced by its accuracy, execution time, storage requirements, and the quality of the training data. The expertise and expense required for compiling a representative training set and labelling the data results in the training data and model obtained from the training data are valuable assets. Once an attacker has access, the machine learning model can be relatively easily copied. Black box access to the model may not provide sufficient protection. Also, a black box may not always be available to protect the model. Once an attacker has copied the model, it can be illegitimately monetized.

<NPL>" shows a machine learning model using a mix of different activation functions.

Therefore, a need exists for a method to better protect a machine learning model from being easily copied.

Generally, there is provided, a method for making a machine learning model more difficult to copy. This is accomplished by using a hybrid combination of activation functions for the hidden nodes of a neural network architecture. That is, each hidden node includes at least one of one type of activation function and at least one of another type of activation function. The types of activation functions for each node may be chosen at random and independently of the activation function choices made for the other nodes. The assignment of activation functions may be unknown to an adversary. In one embodiment, a neural network application running on the neural network architecture is implemented in hardware in a hardware secure element. The use of a hybrid combination of activation functions provides better resistance to a cloning attack by reducing the accuracy of the copied model so that the copy will provide a different answer when queried. The use of a hardware secure element makes a device more difficult to illegitimately upgrade with a copy of better machine learning model.

In accordance with an embodiment, there is provided a computer-implemented method for protecting a machine learning model from copying according to claim <NUM>. The neural network application may be implemented in a secure hardware element on an integrated circuit. The integrated circuit may be characterized as being a field programmable gate array. The method may further include providing a different neural network architecture for implementation in each of a plurality of machine learning devices, wherein the different neural network architectures may differ in the plurality of machine learning devices by changing a ratio of the combination of the first and second types of activation functions in the plurality of hidden layers. Assigning a combination of the activation functions may further include randomly assigning the mix of the activation functions. The combination of activation functions may be implemented in a secure hardware element in the integrated circuit with the neural network application and wherein the neural network application may not be changeable. Assigning a combination of the activation functions of the first and second types to the plurality of nodes may further include assigning a sine function as one of the activation functions of the first or the second types, and wherein fewer than fifty percent of the plurality of nodes comprises the sine function.

<FIG> illustrates a plurality of type <NUM> activation functions. Activation functions are used in a neural network to add non-linearity to a machine learning model. An activation function can be any type of function. Activation functions may include Gaussian functions, monotonic functions (increasing or decreasing), periodic functions, trigonometric functions, and sinusoidal functions. There are various sets of activation functions that are categorized by whether the activation function has an increasing slope, a decreasing slope, or both. In this description, some commonly used activation functions are categorized by whether a plot, or graph, of the activation function has segments of increasing or decreasing slope. A type <NUM> activation function includes functions providing a plot having at most one increasing slope segment and/or at most one decreasing slope segment. Some examples of commonly used type <NUM> activation functions are illustrated in <FIG>. In <FIG>, five examples of activation function plots are labeled AF1 - AF5. As illustrated, type <NUM> activation function AF1 is a hyperbolic tangent function where x is an input to a node <MAT> Activation function AF2 is a sigmoid <MAT> Activation function AF3 is a rectified linear unit (ReLU) <MAT> Activation function AF4 is a step function <MAT> Activation function AF5 is a Gaussian function <MAT> There are other examples of type <NUM> activation functions (not shown) such as a leaky ReLU function that can be used. Also, Softmax refers to a complete layer, where individual nodes in the Softmax layer have a sigmoid (type <NUM>) activation function and the output is normalized so that the outputs sum up to <NUM>.

<FIG> illustrates two examples of type <NUM> activation functions. A type <NUM> activation function can be plotted having at least two increasing slope segments and/or at least two decreasing slope segments. <FIG> illustrates two examples of commonly used activation function plots labeled AF6 and AF7. Activation function plot AF6 is a sine function abbreviated sin(x), and activation function plot AF7 is a sine cardinal function abbreviated sinc(x).

<FIG> illustrates neural network architecture <NUM> in accordance with an embodiment. Generally, a neural network includes an input layer, output layer, and between the input and output layers are one or more hidden layers. Each layer can have any number of nodes, or neurons. In <FIG>, an input layer (INPUT LAYER) includes a plurality of nodes <NUM>, <NUM>, <NUM>, and <NUM>. There are two hidden layers (HIDDEN LAYERS). One hidden layer includes nodes <NUM>, <NUM>, <NUM>, and <NUM>. Another hidden layer includes nodes <NUM>, <NUM>, and <NUM>. There can be any number of hidden layers. Each hidden layer can include any number of nodes. One or more output signals are computed based on a weighted sum of the inputs and output from the output nodes. An output layer (OUTPUT LAYER) includes nodes <NUM> and <NUM>. There can be any number of output nodes in the output layer. The input and output connections of the nodes are weighted. In <FIG>, the nodes are connected by weighted arcs having arrows indicating forward propagation. However, back propagation is also possible. Also, the activation functions include non-linear activation functions. The activation functions, the weights, and the input to a node defines the output.

In accordance with an embodiment, the nodes of the hidden layers of neural network architecture <NUM> have a hybrid combination of activation functions. That is, the hidden layers include a combination of different types of activations functions. In accordance with an embodiment, neural network architecture <NUM> includes a plurality of a type <NUM> activation function and a plurality of a type <NUM> activation function. More specifically, hidden layer nodes <NUM>, <NUM>, <NUM>, and <NUM> each have type <NUM> activation function AF1 (tanh function) and hidden layer nodes <NUM>, <NUM>, and <NUM> each have type <NUM> activation function AF6 (sine function). Other combinations of activation functions can be used. In one embodiment, the activation functions are chosen and/or assigned randomly. The activation functions may be chosen to have a predetermined ratio of one activation function type to another activation function type. More specifically, the activation functions for the hidden nodes may be assigned at random and independent of the choices made for the other hidden nodes. For example, in the above example, it may be determined that the AF6 sine function comprises less than fifty percent of the total number of activation functions in the hidden layers. A neural network application is run on the neural network architecture with a training set to generate a machine learning model. As an example, one training set is the Pima-Indians-Diabetes data set.

It has been determined that using a combination of activation functions for the hidden layers makes it more difficult for an attacker to make a high-quality copy of the machine learning model using access provided by an external application programming interface (API). A combination of type <NUM> and type <NUM> activation functions provides better resistance to cloning than a combination of two different type <NUM> activation functions. It has also been determined that the hybrid combination of activation functions, while making the model more difficult to copy, does not significantly harm the accuracy of the model. However, a model cloned from a network having a hybrid combination of activation functions does have a reduced accuracy compared to the original model.

The neural network application may be implemented in a secure hardware element to protect the neural network application from being illegitimately changed or upgraded. The secure hardware may include the neural network application implemented in hardware in a field-programmable gate array (FPGA). Using secure hardware in a device to implement the neural network also causes the combination of activation functions to be hard-coded so that the combination cannot be easily changed. For example, assume that an adversary has a device with the same neural network application as a target device. However, the target device includes a higher-quality more expensive machine learning model that the attacker would like to copy and include on the adversary's device. If the target implements the neural network application in an FPGA as described, the attacker will have difficulty upgrading to the higher-quality machine learning model of the target device. Also, the attacker will have difficulty making a copy of the model if the combination of activation functions has been implemented in the target device.

<FIG> illustrates a portion of system having internet of things (IoT) edge node <NUM> and loT device <NUM> in accordance with an embodiment. The loT device <NUM> and edge node <NUM> may each be implemented on one or more integrated circuits. The loT device <NUM> is bidirectionally connected to edge node <NUM>. The loT device <NUM> produces data that is sent to edge node <NUM>. Edge node <NUM> includes machine learning unit <NUM> and secure element <NUM>. A neural network architecture may be implemented as taught above in the presently described embodiment regarding neural network architecture <NUM>. Secure element <NUM> is tamper resistant and may be used to store a neural network application such as described above. Secure element <NUM> may also include a processor and enough storage to store the neural network architecture including the combination of activation functions. The loT device <NUM> may also have a secure element and the neural network architecture and combination of activation functions as implemented and described for edge node <NUM>. The system may include other portions (not shown) that would be capable of implementing the machine learning unit and secure element as described.

<FIG> illustrates circuit <NUM> for use in either of IoT edge node <NUM> or loT device <NUM> in accordance with an embodiment. Circuit <NUM> may be implemented on one or more integrated circuits and may be used to implement either or both of machine learning unit <NUM> and secure element <NUM>. Circuit <NUM> includes bus <NUM>. Connected to bus <NUM> is processor <NUM>, memory <NUM>, user interface <NUM>, instruction memory <NUM>, and network interface <NUM>. Processor <NUM> may be any hardware device capable of executing instructions stored in memory <NUM> or instruction memory <NUM>. Processor <NUM> may be, for example, a microprocessor, field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or similar devices. The processor may be in the secure hardware element and may be tamper resistant.

Memory <NUM> may be any kind of memory, such as for example, L1, L2, or L3 cache or system memory. Memory <NUM> may include volatile memory such as static random-access memory (SRAM) or dynamic RAM (DRAM), or may include non-volatile memory such as flash memory, read only memory (ROM), or other volatile or non-volatile memory. Also, memory <NUM> may be in a secure hardware element.

User interface <NUM> may be connected to one or more devices for enabling communication with a user such as an administrator. For example, user interface <NUM> may be enabled for coupling to a display, a mouse, a keyboard, or other input/output device. Network interface <NUM> may include one or more devices for enabling communication with other hardware devices. For example, network interface <NUM> may include, or be coupled to, a network interface card (NIC) configured to communicate according to the Ethernet protocol. Also, network interface <NUM> may implement a TCP/IP stack for communication according to the TCP/IP protocols. Various other hardware or configurations for communicating are available for communicating.

Instruction memory <NUM> may include one or more machine-readable storage media for storing instructions for execution by processor <NUM>. In other embodiments, memory <NUM> may also store data upon which processor <NUM> may operate. Memory <NUM> may store, for example, a machine learning model, or encryption, decryption, or verification applications. Memory <NUM> may be in the secure hardware element and be tamper resistant.

<FIG> illustrates method <NUM> for protecting a machine learning model from copying. Method <NUM> begins at step <NUM>, where a neural network architecture is provided. At step <NUM>, a neural network application for running on the neural network architecture is provided. At step <NUM>, a combination of different types of activation functions is assigned to a plurality of nodes in the hidden layers of the neural network architecture. In one embodiment, the combination of activation functions is randomly assigned to plurality of nodes of the hidden layers. At step <NUM>, the neural network application is trained with a training set to generate a machine learning model. The use of the combination of activation functions protects the machine learning model for copying.

A method for protecting a machine learning model from copying is provided. The method includes providing a neural network architecture having an input layer, a plurality of hidden layers, and an output layer. Each of the plurality of hidden layers has a plurality of nodes. A neural network application is provided to run on the neural network architecture. First and second types of activation functions are provided. Activation functions including a combination of the first and second types of activation functions are provided to the plurality of nodes of the plurality of hidden layers. The neural network application is trained with a training set to generate a machine learning model. Using the combination of first and second types of activation functions makes it more difficult for an attacker to copy the machine learning model. Also, the neural network application may be implemented in hardware to prevent easy illegitimate upgrading of the neural network application.

Various embodiments, or portions of the embodiments, may be implemented in hardware or as instructions on a non-transitory machine-readable storage medium including any mechanism for storing information in a form readable by a machine, such as a personal computer, laptop computer, file server, smart phone, or other computing device. The non-transitory machine-readable storage medium may include volatile and non-volatile memories such as read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage medium, NVM, and the like. The non-transitory machine-readable storage medium excludes transitory signals.

Claim 1:
A computer-implemented method for protecting a machine learning model from copying, the method comprising:
providing a neural network architecture having an input layer, a plurality of hidden layers, and an output layer, each of the plurality of hidden layers having a plurality of nodes;
providing a neural network application to run on the neural network architecture;
providing a first type of activation function;
providing a second type of activation function;
assigning a combination of one activation function of the first type and one activation function of the second type to the plurality of nodes, each node being assigned either a first type or a second type of activation function; and training the neural network application with a training set to generate a machine learning model; wherein
the assigning the combination of activation functions further comprises the first type of activation function being assigned from a set of activation functions comprising tanh(x), rectified linear unit ReLU(x), step(x), Gaussian(x), sigmoid(x), and leaky ReLU, and
is characterised in that the second type of the activation functions is being assigned from a set of activation functions comprising sin(x) and sinc(x), wherein x is an input to a node of the plurality of nodes.