Programmable feature extractor with anonymization

A compute instance may be configured to extract a feature of a data instance accessed by the compute instance, generate an anonymized feature value for the feature of the data instance, include the anonymized feature value in a feature vector corresponding to the data instance, and transmit the feature vector to a server-based computing system.

BACKGROUND INFORMATION

Conventional signature-based malware detection systems have difficulty keeping pace with the ever increasing proliferation and complexity of malware attacks. For example, it can be time consuming and resource intensive to identify malicious code associated with a malware attack, generate a malware signature for the malicious code, and update anti-malware software installed on a local compute instance (e.g., a local computing device) with the malware signature so that the anti-malware software can adequately protect the compute instance from the malware attack. Until the anti-malware software is updated with the malware signature, the compute instance may be vulnerable to the malware attack.

Machine learning-based malware detection systems have recently been introduced as an alternative to conventional signature-based malware detection systems. Machine learning-based malware detection systems use machine learning techniques to construct models that recognize malware “features.” These models may consist of calculation parameters, classification rule sets, or decision trees that determine the malicious nature of a particular data instance (e.g., a file, a software program, etc.) based, for example, on the determined features. Features of data instances may be extracted into vectors, which may be presented as inputs to the model. Using such a machine learning model, a malware classification system may, in many cases, more effectively and quickly identify data instances as potentially malicious as compared to conventional signature-based malware detection systems.

A machine learning model may be “trained” using a large number (e.g., hundreds, thousands, millions) of previously-classified data instances (e.g., classified as malicious or non-malicious).

DETAILED DESCRIPTION

Systems and methods for using a programmable feature extractor to generate feature vectors for use with machine learning models are described herein. In some examples, as will be described below, a compute instance (e.g., a computing device used by an end user) may store, within a storage facility of the compute instance, a programmable feature extractor associated with a machine learning model maintained by a server-based computing system configured to communicate with the compute instance by way of a network. The machine learning model may take as input a feature set that includes one or more features. The compute instance may execute the programmable feature extractor to generate a feature vector corresponding to a data instance accessed by the compute instance. As will be described below, the feature vector may include a feature value specific to the data instance for each feature included in the feature set. The compute instance may transmit the feature vector corresponding to the data instance to the server-based computing system. The server-based computing system may apply the feature vector as a training input to construct the machine learning model.

In some examples, the compute instance (using the programmable feature extractor stored on and executed by the compute instance) may anonymize the feature vector to comply with one or more privacy regulations and/or preferences of an end user. For example, the compute instance may extract a feature of a data instance accessed by the compute instance, generate an anonymized feature value for the feature of the data instance, and include the anonymized feature value in a feature vector corresponding to the data instance.

Numerous advantages and benefits are associated with the systems and methods described herein. For example, the systems and methods described herein may provide a server-based computing system with feature vectors associated with data instances, for example, benign data instances. The feature vectors may be used to train a machine learning model that may be used to facilitate improved detection and prevention of malware (e.g., zero-day malware that has not previously been specifically identified as malware, variations of known malware, malware that has been disguised, and so on).

Advantageously, the feature vectors associated with data instances as described herein are generated locally by a compute instance and then transmitted to the server-based computing system. This obviates the need to transmit the data instances themselves to the server-based computing system, which increases efficiency, increases privacy, and reduces bandwidth requirements.

Furthermore, the programmable feature extractor stored and executed by the compute devices described herein to generate the feature vectors is dynamically programmable. In other words, at any given time (e.g., even as the programmable feature extractor is being executed by a compute instance), the server-based computing system (or any other computing device remote from the compute instance) may update the programmable feature extractor (e.g., by transmitting a programming script to the compute instance) to begin extracting additional or alternative features for representation in feature vectors generated by the programmable feature extractor. This obviates the need to deploy and install patches on each of the compute instances that store and execute the programmable feature extractor, as would be performed during conventional software update procedures. As such, the programmable feature extractor may be updated in essentially real-time on each of the compute instances that execute the programmable feature extractor, thus facilitating immediate action by the compute instances and conserving resources and time associated with conventional software update procedures. Also, this flexibility enables the programmable feature extractor deployed on some systems to test different features than the programmable feature extractor deployed on other systems.

Moreover, by anonymizing the feature vectors described herein, the systems and methods described herein may increase compliance with privacy regulations and increase end user privacy. This, in turn, may incent more end users to allow the feature vectors to be generated and transmitted to the server-based computing system, which may provide the server-based computing system with more data to better train a model. In some examples, as will be described below, the anonymization may be performed in accordance with an anonymization parameter that specifies a degree of anonymity along a sliding scale between full fidelity and full privacy. The anonymization parameter may be set by an end user, thus providing the end user with a specific control over how his or her personal data is accessed and used.

These and other benefits and/or advantages that may be provided by the systems and methods described herein will be made apparent by the following detailed description.

FIG.1shows an exemplary configuration100in which a server-based computing system102is selectively and communicatively coupled to a local compute instance104(“compute instance104”) by way of a network106.

Server-based computing system102may be implemented by one or more server-side computing devices configured to communicate with local computing instances (e.g., local compute instance104) by way of a network (e.g., network106). For example, server-based computing system102may be implemented by one or more servers or other physical computing devices each comprising a processor and memory. Additionally or alternatively, server-based computing system102may include or may be implemented by a physical computing device associated with an entity, such as a business or other organization. For example, compute instance104may be implemented by a server, a firewall, a gateway, a data center device, a cloud computing instance, etc. Additionally or alternatively, server-based computing system102may include or may be implemented by one or more virtual machines that emulate a physical computing device.

Server-based computing system102may be configured to perform machine learning-based operations as described herein. For example, server-based computing system102may maintain and train a machine learning model for classifying data instances based on a feature set that includes one or more features. These and other operations that may be performed by server-based computing system102will be described in more detail below.

In some examples, server-based computing system102may be associated with (e.g., managed or maintained by) a security entity that specializes in network and/or computer security and that provides malware detection services to end users and/or enterprises.

Compute instance104may be implemented by a physical computing device associated with (e.g., used by) an end user (“user”). For example, compute instance104may be implemented by a mobile device (e.g., a mobile phone or tablet computer), a personal computer, a gaming device, an Internet of things (“IoT”) device, and/or any other type of physical computing device configured to access data instances (e.g., by way of network106and/or from any other source). Additionally or alternatively, compute instance104may implemented by a physical computing device associated with an entity, such as a business or other organization. For example, compute instance104may be implemented by a server, a firewall, a gateway, a data center device, etc. Additionally or alternatively, compute instance104may be implemented by a virtual machine that emulates a physical computing device.

In some examples, compute instance104may access data provided by server-based computing system102and/or any other computing system (e.g., a content server) by way of network106. For example, compute instance104may be configured to execute an application (e.g., a native application, a mobile application, a web-based application, etc.) and use the application to access (e.g., download, transmit, stream, or otherwise process) one or more data instances by way of network106and/or from any other source. Such data instances may include, but are not limited to, a file, metadata associated with a file, an email or other message, network data, streaming data, and/or any other type of data as may serve a particular implementation.

Network106may include a provider-specific wired or wireless network (e.g., a cable or satellite carrier network or a mobile telephone network), the Internet, a wide area network, a content delivery network, or any other suitable network, or any combination of networks. Data may flow between server-based computing system102and compute instance104using any communication technologies, devices, media, and protocols as may serve a particular implementation.

FIG.2illustrates exemplary components of server-based computing system102. As shown, server-based computing system102may include, without limitation, a storage facility202and a processing facility204selectively and communicatively coupled to one another. It will be recognized that although facilities202and204are shown to be separate facilities inFIG.2, facilities202and204may be combined into a single facility or divided into more facilities as may serve a particular implementation. Server-based computing system102may be implemented by one or more computing devices (i.e., one or more physical or virtual computing devices). Facilities202and204will now be described in more detail.

Storage facility202may maintain (e.g., store within memory of a computing device that implements server-based computing system102) various types of data received, generated, managed, used, and/or transmitted by processing facility204. For example, as shown, storage facility202may maintain machine learning model data206, feature definition data208, and feature vector data210. Machine learning model data206may include data representative of, used by, or associated with a machine learning model maintained by processing facility204. Depending on the type of machine learning model, the machine learning model data206may include, for example, constants, coefficients, and configurations defining a neural network, rules or parameters associated with a random forest or a decision tree, an association rule, or other model. Feature definition data208may include data that defines one or more features included in a feature set used by the machine learning model, for example, to classify data instances. For example, feature definition data208may describe particular features, locations of features, manipulations or calculations to be performed on features, combinations of features, or metadata describing features. Feature vector data210may be representative of feature vectors applied to the machine learning model (e.g., as training inputs), and may vary based on the features included in a given feature vector. For example, feature vector data210may include data based on manipulations or calculations performed on the features. Storage facility202may maintain additional or alternative data as may serve a particular implementation.

Processing facility204may perform various operations related to the machine learning model. For example, processing facility204may be configured to maintain a machine learning model to be used in a classifier for classifying data instances based on a feature set that includes a plurality of features. Processing facility204may be further configured to direct a programmable feature extractor stored on and executed by a compute instance (e.g., compute instance104) to generate a feature vector corresponding to a data instance accessed by the compute instance, receive the feature vector from the compute instance by way of a network (e.g., network106), and apply the feature vector as a training input to create or improve the machine learning model. These and other operations that may be performed by processing facility204will be described in more detail below.

FIG.3illustrates exemplary components of compute instance104. As shown, compute instance104may include, without limitation, a storage facility302and a processing facility304selectively and communicatively coupled to one another. It will be recognized that although facilities302and304are shown to be separate facilities inFIG.3, facilities302and304may be combined into a single facility or divided into more facilities as may serve a particular implementation. As mentioned, compute instance104may be implemented by one or more physical computing devices or virtual machines that emulate a physical computing device.

Storage facility302may maintain (e.g., store within memory of compute instance104) various types of data received, accessed, generated, used, processed, and/or transmitted by processing facility304. For example, as shown, storage facility302may maintain a data instance306, programmable feature extractor data308, feature definition data310, and feature vector data312. Storage facility302may maintain additional or alternative data as may serve a particular implementation.

Data instance306may include, for example, any file, portion of a file, or bit sequence maintained by storage facility302. For example, data instance306may include an executable file, a word processing file, an email or other message, metadata associated with a file, a piece of code, etc. Data instance306may be persistently stored by storage facility302(e.g., in a hard drive or flash memory drive) and/or temporarily stored by storage facility302(e.g., in random access memory). In addition to maintaining data instances, compute instance104may otherwise access a data instance by remote access, receiving, streaming, or otherwise processing the data instance.

Programmable feature extractor data308may include data representative of a programmable feature extractor. For example, programmable feature extractor data308may include one or more files that, when executed by compute instance104, may perform various feature extraction operations, as will be described below. Programmable feature extractor data308may further include other data associated with and/or used by the programmable feature extractor. For example, programmable feature extractor data308may include data representative of a programming script transmitted to compute instance104from server-based computing system102to dynamically program the programmable feature extractor executed by compute instance104.

Feature definition data310may be similar to feature definition data208and may include data that defines a plurality of features included in a feature set used by a machine learning model to classify data instances. In some examples, feature definition data310may be provided (e.g., transmitted to compute instance104) by server-based computing system102.

Feature vector data312may be representative of a feature vector as generated by compute instance104(i.e., by programmable feature extractor being executed by compute instance104). The feature vector is associated with a data instance (e.g., data instance306) and may include one or more distinct feature values specific to the data instance for each feature included in a feature set used by a machine learning model, for example, a model maintained by server-based computing system102.

Processing facility304may perform various feature-related operations as may serve a particular implementation. For example, processing facility304may execute a programmable feature extractor to generate a feature vector corresponding to a data instance accessed by compute instance104. While generating the feature vector, processing facility304may perform one or more anonymization operations with respect to the feature vector. Processing facility304may be further configured to transmit the feature vector to server-based computing system102for use as a training input to a machine learning model maintained by server-based computing system102. For example, processing facility304may upload the feature vector to server-based computing system102by way of network106. These and other operations that may be performed by processing facility304will be described in more detail below.

FIG.4shows an exemplary machine learning model402that may be maintained by server-based computing system102(i.e., processing facility204). Machine learning model402may be configured to classify data instances based on a feature set that includes a plurality of features. For example, machine learning model402may receive as an input a feature vector404-1corresponding to a data instance, analyze the feature vector404-1, and output classification data406for the data instance based on the analysis of feature vector404-1. Classification data406may be representative of a particular classification assigned by machine learning model402to the data instance.

In some examples, machine learning model402is associated with a malware detection system and configured to classify data instances as malicious or benign, provide a classification of suspiciousness or maliciousness, provide a classification of a type of malware, or other suitable classification. Alternatively, machine learning model402may be associated with any other classification or classification system and configured to perform any other suitable type of classification of data instances as may serve a particular implementation. For example, machine learning model402may be used to classify data that passes through a firewall as being associated with a particular network site or location. To illustrate, machine learning model402may be used to classify data sequences that pass through a firewall as to type of application or originating with or directed to a particular website or type of website. However, for illustrative purposes, it will be assumed herein that machine learning model402is associated with a malware detection system and configured to classify data instances as malicious or benign.

Machine learning model402may be actively used by a security system, such as a malware detection system. For example, machine learning model402may be used in conjunction with a malware detection service (e.g., a malware detection software program) provided by a computer security entity to identify and/or remedy actual malware files and/or attacks. Alternatively, machine learning model402may be used in a test or beta mode. For example, machine learning model402may be a model that is intended to undergo testing and/or training before being actively used by a malware detection system.

Machine learning model402may analyze data in any suitable manner. For example, machine learning model402may implement one or more decision tree learning algorithms, association rule learning algorithms, artificial neural network learning algorithms, deep learning algorithms, deep neural networks, and/or any other suitable data analysis techniques as may serve a particular implementation. Exemplary machine learning models and algorithms that may be used in connection with the systems and methods described herein, such as the deep neural networks and other machine learning models described in U.S. Pat. No. 9,690,938.

Server-based computing system102may train one or more machine learning models, such as machine learning model402. This training may be performed prior to and/or concurrently with machine learning model402being actively used to classify data instances. To this end, as shown inFIG.4, a plurality of feature vectors404(i.e., feature vectors404-2through404-n) corresponding to data instances with associated classifications may be applied as training inputs408to machine learning model402. In many cases, the total number of feature vectors404applied to machine learning model402as training inputs408is relatively large (e.g., tens or hundreds of thousands, millions). By using a large amount of data, machine learning model402may be trained to more accurately classify data instances.

Each feature vector404applied as training inputs corresponds to a different data instance. For example, some of feature vectors404applied as training inputs may correspond to data instances that are known or pre-classified as being malicious, while some of feature vectors404applied as training inputs408may correspond to data instances that are known or pre-classified as being benign.

Server-based computing system102may generate and/or receive feature vectors corresponding to data instances that are classified as being malicious in any suitable manner. For example, server-based computing system102may maintain and/or access a database of data instances that are known to be malicious and generate feature vectors for the data instances. Additionally or alternatively, server-based computing system102may receive feature vectors corresponding to data instances that are known to be malicious from an external source (e.g., another server-based computing system and/or one or more compute instances associated with end users).

Server-based computing system102may receive feature vectors corresponding to data instances that are known or pre-classified as being benign in any suitable manner. For example, as will be described in more detail below, server-based computing system102may receive feature vectors from compute instances (e.g., compute instance104) by way of a network (e.g., network106). In some examples, machine learning model402may assume that these feature vectors correspond to data instances that are benign. In most cases, this assumption is correct. However, machine learning model402may occasionally receive, from a compute instance, a feature vector that corresponds to a data instance that is actually malicious. Even though machine learning model402may treat this feature vector as corresponding to a data instance that is benign, the training of machine learning model402may not be adversely affected due to the large quantity of feature vectors that are used to train machine learning model402.

As mentioned, machine learning model402is configured to classify data instances based on a feature set that includes a plurality of features. To this end, server-based computing system102may maintain feature definition data208, which defines the plurality of features included in the feature set. As will be described below, the feature set may be modified at any time to include additional or alternative features. For example, after machine learning model402has been trained for a certain amount of time, an administrator associated with machine learning model402may decide that machine learning model402may be better trained by analyzing a feature not currently included in the feature set. The administrator may provide user input that causes server-based computing system102to update feature definition data208to include the new feature in the feature set. As will be described below, server-based computing system102may dynamically update programmable feature extractors executed by compute instances to begin extracting the new feature from data sets and including a corresponding feature value in the feature vectors that are transmitted to server-based computing system102. In like manner, server-based computing system102may remove a feature from the feature set.

FIG.5illustrates an exemplary feature vector404that corresponds to a data instance. Feature vector404may be of any suitable data structure and/or format. For example, feature vector404may be a single or multi-dimensional array of data.

As shown, feature vector404includes a plurality of feature values502(i.e.,502-1through502-n). Each feature value502corresponds to a particular feature included in the feature set used by machine learning model402and represents a value of a feature of the data instance.

A data instance may have many different features for which feature values may be included in feature vector404. For example, exemplary features of a data instance for which feature values may be included in feature vector404include, but are not limited to, an entropy of the data instance, a specific set of byte codes in the data instance, a relative frequency of occurrence of key byte-patterns in the data instance, header information associated with the data instance (e.g., header information for a portable executable file), an author name associated with the data instance, a size of the data instance, a source associated with the data instance, byte values in the data instance, a string length value associated with one or more strings in the data instance, a string hash value of the one or more strings in the data instance, and a compilation date of the data instance. Other suitable features also may be used.

A given feature vector404may have a set number of feature values502for encoding a particular feature. For example, a relative frequency of key byte-patterns may be encoded into a feature vector404that has 256 feature values502. Other numbers of feature values502, such as 128, 512, or 1024 feature values, or any suitable number of feature values may be used.

Each feature value502may have any suitable format. For example, each feature value502may be a numeric value. Each feature value502may alternatively be an alpha-numeric value, a data string, or of any other suitable format. Exemplary manners in which feature values502may be generated will be described below.

FIG.6shows an exemplary configuration600in which compute instance104includes a programmable feature extractor602to generate and transmit, to server-based computing system102, a feature vector404corresponding to a data instance604. Server-based computing system102may use feature vector404as a training input408to machine learning model402. It will be recognized that when operations are described herein as being performed by compute instance104, it is meant that any combination of compute instance104and programmable feature extractor602performs the operations. Likewise, when operations are described herein as being performed by programmable feature extractor602, it is meant that any combination of compute instance104and programmable feature extractor602performs the operations.

As shown, programmable feature extractor602may reside on (i.e., be stored in memory by) compute instance104. For example, compute instance104may store within memory executable code representative of programmable feature extractor602.

Programmable feature extractor602may be configured to run on compute instance104independently from any interaction with server-based computing system102. Alternatively, programmable feature extractor602may be configured to establish a connection with server-based computing system102and run in accordance with instructions provided by server-based computing system102by way of the connection.

Programmable feature extractor602may be configured to run on compute instance104independent of any malware detection software installed on compute instance104. Alternatively, programmable feature extractor602may be provided as a component of malware detection software installed on compute instance104.

Programmable feature extractor602may be configured to run in a sandbox environment of compute instance104. In this manner, programmable feature extractor602may be prevented from interfering with other processes or applications running on compute instance104. Programmable feature extractor602may alternatively run in a non-sandbox environment of compute instance104.

Programmable feature extractor602may be provided by server-based computing system102to compute instance104. For example, server-based computing system102may transmit data representative of programmable feature extractor602to compute instance104for installation on compute instance104. Alternatively, a different system or device unrelated to server-based computing system102may provide programmable feature extractor602to compute instance104for installation on compute instance104.

Compute instance104may execute programmable feature extractor602in any suitable manner. For example, programmable feature extractor602may run in the background on compute instance104. In some examples, compute instance104may display (e.g., on a display device connected to or a part of compute instance104) a graphical user interface associated with programmable feature extractor602. A user may interact with the graphical user interface to provide user input configured to define one or more settings or parameters of programmable feature extractor602.

Programmable feature extractor602may be dynamically programmable by server-based computing system102and/or any other computing system remote from compute instance104. For example, as shown inFIG.6, server-based computing system102may transmit feature extractor programming instructions606to compute instance104(e.g., by way of network106). Feature extractor programming instructions606may be configured to update programmable feature extractor602to extract additional or alternative features from data instances accessed by compute instance104. For example, server-based computing system102may update feature definition data208to either define one or more new features as being included in the feature set associated with machine learning model402or remove one or more features from being included in the feature set. In response, server-based computing system102may transmit programming instructions606that include the newly updated feature definition data208to compute instance104. Programming instructions606may dynamically update programmable feature extractor602to generate feature vectors for data instances accessed by compute instance104in accordance with the updated feature definition data208. Programming instructions606may be configured to dynamically update any other aspect of programmable feature extractor602as may serve a particular implementation.

Programming instructions606may be of any suitable format. For example, programming instructions606may be in the form of a programming script (e.g., a Lua or Python script). Upon receiving the programming script, programmable feature extractor602may immediately begin operating in accordance with the script. In this manner, programmable feature extractor602may be always up-to-date without compute instance104having to perform a conventional software update process (e.g., patching). Programming instructions may be in the form of one or more executable modules that may be called by the programmable feature extractor. Such programming instructions may be, for example, in the form of executable code. Such programming instructions may be, for example, in the form of one or more libraries of code implemented in a processor-independent language such as Java. Such programming instructions may be, for example, in the form of one or more dynamically loaded or called libraries implemented in native code for the environment of the compute instance104.

Additionally or alternatively, server-based computing system102may dynamically program programmable feature extractor602by maintaining a as an ongoing or intermittent communication channel with programmable feature extractor602while programmable feature extractor602is being executed by compute instance104. Programming instructions606may be transmitted to programmable feature extractor602by way of the communication channel at any time during the execution of programmable feature extractor602by compute instance104. Additionally or alternatively, programming instructions606may reside on server-based computing system102. In this configuration, programmable feature extractor602may be configured to fetch or otherwise access programming instructions606by way of the communication channel.

Although shown as providing one feature vector404for a given data instance604, it should be understood that the programmable feature extractor602may support a variety of different features and feature vectors for a given data instance604. For example, the programmable feature extractor602may provide a first feature vector404based on a given data instance604and a second feature vector (not shown) different from the first feature vector404, for example, with some overlapping and some of the same feature values, or with different feature values. Additionally or alternatively, the feature vector404may include feature vectors derived from multiple features, for example, so that the feature vector404may include multiple feature vectors that may be used by different models.

FIG.7shows various functional modules (i.e., modules702-706) that may be implemented by any suitable combination of compute instance104and programmable feature extractor602and that may be configured to perform various operations to generate feature vector404. Additional examples of feature vector generation are described, for example, in the above-referenced U.S. Pat. No. 9,690,938.

As shown, a feature extractor702extracts a feature of data instance604in accordance with feature definition data (e.g., feature definition data received from server-based computing system102). The feature definition data may specify one or more features that feature extractor702is to extract from data instance604. For purposes of this example, feature extractor702extracts a single feature from data instance604. However, it will be recognized that feature extractor702may extract any number of features from data instance604as specified by the feature definition data.

Feature extractor702may extract a feature of data instance604in any suitable manner. For example, feature extractor702may identify data within data instance604that is representative of the feature and copy the data to a location in memory of compute instance104. For example, if feature is an author name associated with data instance604, feature extractor702may identify data within data instance604that specifies the author name and copy this data to a location in memory of compute instance104. In some examples, data instance604may be compressed or otherwise processed before being analyzed by feature extractor702.

A feature value generator704generates a feature value for the feature extracted by feature extractor702. This may be performed in any suitable manner. For example, feature value generator704may compute a hash value for the feature (i.e., a numeric value that uniquely identifies the feature). As mentioned above, the feature value generated by feature value generator704may alternatively be an alpha-numeric value, a data string, or of any other suitable format.

A feature vector compiler706generate feature vector404by receiving the feature value as an input and processing the feature value together with any other feature value generated by feature value generator704for data instance604. For example, feature vector compiler706may apply one or more compiler operations to the feature value to include the feature value in a suitable data structure for feature vector404. In some examples, feature vector compiler706may concurrently generate multiple feature vectors corresponding to multiple data instances accessed by compute instance104.

Once feature vector404has been generated, compute instance104may transmit feature vector404to server-based computing system102. In some examples, compute instance104transmits feature vector404to server-based computing system102immediately in response to feature vector404being generated. Alternatively, compute instance104may store feature vector404in memory for a period of time before transmitting feature vector404to server-based computing system102. For example, compute instance104may wait to transmit feature vector404to server-based computing system102until other feature vectors are generated, until CPU and/or network usage by compute instance104is below a certain threshold, or until a predetermined time specified by a transmission schedule. In these cases, multiple feature vectors may be concurrently transmitted by compute instance104to server-based computing system102.

Programmable feature extractor602(FIG.6) may be configured to anonymize feature vector404to comply with one or more privacy regulations and/or preferences of a user. As will be described below, programmable feature extractor602may be programmed to generate feature vectors that have customizable degrees of anonymity (e.g., anywhere between full fidelity (minimal or no privacy) and full privacy).

In some examples, programmable feature extractor602may anonymize feature vector404by applying one or more privacy operations to feature values generated by feature value generator704before feature vector404is generated.

For example,FIG.8Ashows that an anonymizer802may be included functionally, for example, in between feature value generator704and feature vector compiler706. Anonymizer802may be implemented by any suitable combination of compute instance104and programmable feature extractor602, or other suitable hardware or software implementation.

InFIG.8A, anonymizer802outputs an anonymized feature value by performing a privacy operation on the feature value output by feature value generator704. Feature vector compiler706receives the anonymized feature value as an input (instead of the non-anonymized feature value output by feature value generator704as described in connection withFIG.7) and includes the anonymized feature value in feature vector404.

FIG.8Bshows an alternative use of anonymizer802in which anonymizer802is used to perform a privacy operation on the feature output by feature extractor702(instead of on the feature value corresponding to the feature). In this configuration, anonymizer802outputs an anonymized feature, which is then processed by feature value generator704to generate an anonymized feature value for the feature. In some alternative examples, anonymizer802may perform one or more privacy operations on both features and feature values.

Anonymizer802may perform a privacy operation on a feature and/or a feature value in any suitable manner. For example, anonymizer802may add noise to the feature and/or feature value in accordance with a differential privacy heuristic. To illustrate, anonymizer802may add noise (e.g., random data) to the feature from a Laplace or Gaussian distribution. This may cause feature value generator704to output a feature value that is not quite exactly representative of the feature, or not strictly deterministic, but that is still sufficient to train a machine learning model402.

As another example, anonymizer802may perform a privacy operation by compressing the feature and/or the feature value. The compression may be performed in accordance with a feature hashing heuristic, an autoencoding heuristic, and/or any other compression heuristic as may serve a particular implementation.

As shown inFIGS.8A-8B, anonymizer802may perform a privacy operation on a feature and/or a feature value in accordance with an anonymity parameter. The anonymity parameter specifies a degree of anonymity that the feature value is to have once the privacy operation is performed on the feature and/or feature value. Hence, compute instance104may maintain data representative of the anonymity parameter and use it to determine the amount of noise and/or compression that is applied to a feature and/or a feature vector.

The degree of anonymity specified by the anonymity parameter may be any degree along a sliding scale having two extremes—full fidelity and full privacy. A feature value that has “full fidelity” is one that has not been anonymized or very minimally anonymized and that may therefore be a practically perfect representation of its corresponding feature. In contrast, a feature value that has “full privacy” is one that has been heavily manipulated (e.g., with noise and/or compression) such that it has fully irreversible anonymity.

It will be recognized there is a tradeoff between fidelity (i.e., efficacy in representing a feature) and privacy and that different degrees of anonymity may be appropriate and/or acceptable for machine learning model402depending on a number of different factors (e.g., the type of the data instance, the purpose for which machine learning model402needs to analyze a feature vector for the data instance, etc.). Moreover, certain entities (e.g., government agencies) may mandate a particular degree of anonymization. Hence, in some examples, server-based computing system102may specify the anonymization parameter based on any of these factors and transmit data representative of the anonymization parameter to compute instance104. Compute instance104may receive the data representative of the anonymity parameter from server-based computing system102and use it to perform the privacy operations on features and/or feature values.

Additionally or alternatively, a user may set and/or adjust the anonymization parameter based on one or more preferences of the user. For example,FIG.9shows an exemplary graphical user interface900that may be associated with programmable feature extractor602and presented by compute instance104(e.g., on a display device connected to or a part of compute instance104). As shown, a user may interact with graphical user interface900to adjust a degree of anonymity used by programmable feature extractor602to generate feature vectors. In particular, the user may adjust a position of a slider902along a sliding scale904between a first end906-1(full fidelity) and as second end906-2(full privacy). The position of slider902may be translated by compute instance104into a corresponding degree of anonymization that programmable feature extractor602uses to generate feature vectors. The user may alternatively specify the anonymization parameter in any suitable manner.

As an example, an administrator at an entity (e.g., a company) may interact with graphical user interface900to specify a degree of anonymization that programmable feature extractor602uses to generate feature vectors for data instances accessed by compute instances owned or otherwise managed by the organization. An administrator can optimize the training of a machine learning model specifically designed to be used to classify data instances accessed by the entity's compute instances against preserving anonymity, and, for example, the administrator may position slider902closer to first end906-1(full fidelity) than to second end906-2(full privacy).

Additionally or alternatively, compute instance104may automatically determine the anonymity parameter based on an attribute of a particular data instance for which a feature vector is to be generated. For example, personal files (e.g., word processing files, emails, etc.) associated with a user may be assigned an anonymity parameter that specifies a relatively high degree of anonymity, while non-personal files (e.g., media files, executable files, etc.) may be assigned an anonymity parameter that specifies a relatively low degree of anonymity. In some implementations, an administrator may select different privacy settings based on attributes of data instances. For example, an administrator may assign a higher anonymity parameter to office document files generated by a user or generated within a company than to software application programs or files downloaded from the internet.

In some examples, compute instance104and/or server-based computing system102may be required to receive consent by a user of compute instance104before programmable feature extractor602performs some or any of the feature vector generation operations described herein. For example, compute instance104may receive consent from the user by way of a graphical user interface. Compute instance104may store the data representative of the consent and/or transmit the data representative of the consent to server-based computing system102and, in response, begin generating and transmitting feature vectors. In some implementations, consent may be required based on attributes of data instances. For example, consent may be required for office documents or other documents created by a user, but not required for software application programs.

Programmable feature extractor602and/or compute instance104may be further configured to generate additional data associated with feature vector404. For example,FIG.10illustrates an exemplary configuration in which a metadata generator1002generates metadata1004associated with feature vector404. Metadata generator1002may be implemented by any suitable combination of compute instance104and programmable feature extractor602. Metadata generator1002may analyze data instance604and generate metadata that identifies an attribute associated with data instance604. The attribute may include a type associated with data instance604(e.g., Word files, image files, emails, etc.), an identity of compute instance104(e.g., an IP address, a computer name, etc.), an identity of a user of compute instance104(e.g., a user name), an identity of a source of data instance604(e.g., a URL of a content provider that provides data instance604, etc.), a geolocation of compute instance104, and/or any other characteristic associated with data instance604.

Compute instance104may transmit metadata to server-based computing system102together with feature vector404. In some examples, compute instance104includes the metadata in feature vector404. Alternatively, compute instance104may transmit the metadata in a data structure separate from feature vector404. Server-based computing system102may use the metadata together with feature vector404to train machine learning model402.

In some examples, one or more of the operations described in connection withFIGS.7-10are performed by compute instance104(i.e., by programmable feature extractor602independently from (e.g., without being specifically instructed by) server-based computing system102. Alternatively, one or more of the operations described in connection withFIGS.7-10are performed by compute instance104at the direction of server-based computing system102. For example, server-based computing system102may transmit instructions to compute instance104for programmable feature extractor602to perform one or more of the operations described in connection withFIGS.7-10.

When machine learning model402has been adequately trained, server-based computing system102may direct a malware detection program stored on and executed by compute instance104to use machine learning model402to classify data instances accessed by compute instance104, for example, as being either malicious or benign. For example, the malware detection program may generate a feature vector corresponding to a data instance that is accessed by compute instance104and transmit the feature vector to server-based computing system102. Server-based computing system102may apply the feature vector to machine learning model402, which may classify the data instance as either malicious or benign. Server-based computing system102may transmit data representative of the classification to the malware detection program, which may take appropriate action corresponding to the classification.

FIG.11illustrates an exemplary method1100that may be performed by a server-side computing system (e.g., server-based computing system102). WhileFIG.11illustrates exemplary operations according to one embodiment, other embodiments may omit, add to, reorder, and/or modify any of the operations shown inFIG.11.

In operation1102, a server-based computing system maintains a machine learning model for classifying data instances based on a feature set that includes a plurality of features. Operation1102may be performed in any of the ways described herein.

In operation1104, the server-based computing system maintains feature definition data that defines the plurality of features included in the feature set. Operation1104may be performed in any of the ways described herein.

In operation1106, the server-based computing system directs a programmable feature extractor program stored on and executed by the compute instance to generate, in accordance with the feature definition data, a feature vector corresponding to a data instance accessed by the compute instance. Operation1106may be performed in any of the ways described herein.

In operation1108, the server-based computing system receives, from the compute instance by way of a network, the feature vector corresponding to the data instance. Operation1108may be performed in any of the ways described herein.

In operation1110, the server-based computing system applies the feature vector corresponding to the data instance as a training input to the machine learning model. Operation1110may be performed in any of the ways described herein.

FIG.12illustrates an exemplary method1200that may be performed by a compute instance (e.g., compute instance104). WhileFIG.12illustrates exemplary operations according to one embodiment, other embodiments may omit, add to, reorder, and/or modify any of the operations shown inFIG.12.

In operation1202, a compute instance stores a programmable feature extractor program associated with a machine learning model maintained by a server-based computing system. The machine learning model is based on a feature set that includes a plurality of features. Operation1202may be performed in any of the ways described herein.

In operation1204, the compute instance executes the programmable feature extractor program to generate a feature vector corresponding to a data instance accessed by the compute instance. The feature vector includes a feature value specific to the data instance for each feature included in the feature set. Operation1204may be performed in any of the ways described herein.

In operation1206, the compute instance transmits the feature vector corresponding to the data instance to the server-based computing system for use as a training input to the machine learning model. Operation1206may be performed in any of the ways described herein.

FIG.13illustrates another exemplary method1300that may be performed by a compute instance (e.g., compute instance104). WhileFIG.13illustrates exemplary operations according to one embodiment, other embodiments may omit, add to, reorder, and/or modify any of the operations shown inFIG.13.

In operation1302, a compute instance extracts a feature of a data instance accessed by the compute instance. Operation1302may be performed in any of the ways described herein.

In operation1304, the compute instance generates an anonymized feature value for the feature of the data instance. Operation1304may be performed in any of the ways described herein.

In operation1306, the compute instance includes the anonymized feature value in a feature vector corresponding to the data instance. Operation1306may be performed in any of the ways described herein.

In operation1308, the compute instance transmits the feature vector to a server-based computing system. Operation1308may be performed in any of the ways described herein.

In certain embodiments, one or more of the systems, components, and/or processes described herein may be implemented and/or performed by one or more appropriately configured computing devices. To this end, one or more of the systems and/or components described above may include or be implemented by any computer hardware and/or computer-implemented instructions (e.g., software) embodied on at least one non-transitory computer-readable medium configured to perform one or more of the processes described herein. In particular, system components may be implemented on one physical computing device or may be implemented on more than one physical computing device. Accordingly, system components may include any number of computing devices, and may employ any of a number of computer operating systems.

A computer-readable medium (also referred to as a processor-readable medium) includes any non-transitory medium that participates in providing data (e.g., instructions) that may be read by a computer (e.g., by a processor of a computer). Such a medium may take many forms, including, but not limited to, non-volatile media, and/or volatile media. Non-volatile media may include, for example, optical or magnetic disks and other persistent memory. Volatile media may include, for example, dynamic random access memory (“DRAM”), which typically constitutes a main memory. Common forms of computer-readable media include, for example, a disk, hard disk, magnetic tape, any other magnetic medium, a compact disc read-only memory (“CD-ROM”), a digital video disc (“DVD”), any other optical medium, random access memory (“RAM”), programmable read-only memory (“PROM”), electrically erasable programmable read-only memory (“EPROM”), FLASH-EEPROM, any other memory chip or cartridge, or any other tangible medium from which a computer can read.

FIG.14illustrates an exemplary computing device1400that may be specifically configured to perform one or more of the processes described herein. As shown inFIG.14, computing device1400may include a communication interface1402, a processor1404, a storage device1406, and an input/output (“I/O”) module1408communicatively connected via a communication infrastructure1410. While an exemplary computing device1400is shown inFIG.14, the components illustrated inFIG.14are not intended to be limiting. Additional or alternative components may be used in other embodiments. Components of computing device1400shown inFIG.14will now be described in additional detail.

Communication interface1402may be configured to communicate with one or more computing devices. Examples of communication interface1402include, without limitation, a wired network interface (such as a network interface card), a wireless network interface (such as a wireless network interface card), a modem, an audio/video connection, and any other suitable interface.

Processor1404generally represents any type or form of processing unit capable of processing data or interpreting, executing, and/or directing execution of one or more of the instructions, processes, and/or operations described herein. Processor1404may direct execution of operations in accordance with one or more applications1412or other computer-executable instructions such as may be stored in storage device1406or another computer-readable medium.

Storage device1406may include one or more data storage media, devices, or configurations and may employ any type, form, and combination of data storage media and/or device. For example, storage device1406may include, but is not limited to, a hard drive, network drive, flash drive, magnetic disc, optical disc, RAM, dynamic RAM, other non-volatile and/or volatile data storage units, or a combination or sub-combination thereof. Electronic data, including data described herein, may be temporarily and/or permanently stored in storage device1406. For example, data representative of one or more executable applications1412configured to direct processor1404to perform any of the operations described herein may be stored within storage device1406. In some examples, data may be arranged in one or more databases residing within storage device1406.

In some examples, any of the facilities described herein may be implemented by or within one or more components of computing device1400. For example, one or more applications1412residing within storage device1406may be configured to direct processor1404to perform one or more processes or functions associated with processing facility204of server-based computing system102and/or processing facility304of compute instance104. Likewise, storage facilities202and302may be implemented by or within storage device1406.

References to items in the singular should be understood to include items in the plural, and vice versa, unless explicitly stated otherwise or clear from the text. Grammatical conjunctions are intended to express any and all disjunctive and conjunctive combinations of conjoined clauses, sentences, words, and the like, unless otherwise stated or clear from the context. Thus, the term “or” should generally be understood to mean “and/or” and so forth.

Recitation of ranges of values herein are not intended to be limiting, referring instead individually to any and all values falling within the range, unless otherwise indicated herein, and each separate value within such a range is incorporated into the specification as if it were individually recited herein. The words “about,” “approximately” or the like, when accompanying a numerical value, are to be construed as indicating a deviation as would be appreciated by one of ordinary skill in the art to operate satisfactorily for an intended purpose. Similarly, words of approximation such as “approximately” or “substantially” when used in reference to physical characteristics, should be understood to contemplate a range of deviations that would be appreciated by one of ordinary skill in the art to operate satisfactorily for a corresponding use, function, purpose, or the like. Ranges of values and/or numeric values are provided herein as examples only, and do not constitute a limitation on the scope of the described embodiments. Where ranges of values are provided, they are also intended to include each value within the range as if set forth individually, unless expressly stated to the contrary. The use of any and all examples, or exemplary language (“e.g.,” “such as,” or the like) provided herein, is intended merely to better illuminate the embodiments and does not pose a limitation on the scope of the embodiments. No language in the specification should be construed as indicating any unclaimed element as essential to the practice of the embodiments.