Patent ID: 12248412

DETAILED DESCRIPTION

At least some aspects of the present disclosure are directed to distributed machine learning with data privacy protection and bandwidth enhancement in general, and more particularly, to distributed artificial neural networks with data privacy protection and bandwidth enhancement. Also, at least some aspects of the present disclosure are directed to a computer network that can be configured to implement using of a feature dictionary for centralized training of an artificial neural network (ANN) and reduction of data transmission.

Disclosed herein is a system that can have multiple computing devices that can host different versions of an ANN. Each computing device of the system can host its own version of the ANN. For example, a first device can host a master version of the ANN, and a second device and other devices of the system can host local versions of the ANN. Also, disclosed herein is a system that can have multiple devices that can host different versions of the ANN as well as different versions of a feature dictionary. In the system, inputs for a master version of the ANN can be encoded in one of the multiple devices hosting a local version of the ANN (such as the second computing device). The encoded inputs (which are encoded inputs which are defined by a local version of the feature dictionary) can then be sent to a master computing device hosting the master version of the ANN (such as the first computing device). In the master computing device, the sent encoded features can be decoded according to a master version of the feature dictionary. This allows for encoded input to be sent to a master version of the ANN over a network instead of an original version of the input which usually includes more data than the encoded input. Thus, by using the feature dictionary for training of a master ANN, there can be reduction of data transmission.

In the system, the computing devices hosting the local versions of the ANN and the local versions of the feature dictionary can extract selected features from user data stored on the devices and use a data compression algorithm to encode the extracted features from the user data. With the encoding, the data compression algorithm can provide pairs of extracted features and corresponding encoded features (such as codewords). The pairs can be included in the feature dictionary as definitions. The pairing of the original extracted data with its corresponding encoded data can be used to create or recreate the feature dictionary. Thus, the local versions of the feature dictionary provide definitions, in the pairings, for the encoded data. These pairings or definitions allow the encoded data to be decoded into its corresponding original data, such as by a master version of the feature dictionary. Also, the local versions of the feature dictionary can be used to update the master version of the feature dictionary. And, the data compression algorithm can be varied in its degree of compression to vary the reduction of data transmission.

In the system, with the sending of the user data from one of the computing devices hosting the local versions of the ANN and dictionary to the computing device hosting the master versions of the ANN and dictionary, instead of sending a full description of a feature of the user data (such as uncompressed user data) a codeword can be sent. Each time the feature is sent to the computing device hosting the master versions of the ANN and dictionary, a codeword for the feature can be defined in the local version of the feature dictionary and sent accordingly, such that only the corresponding codeword is transmitted over the network. In some embodiments, a definition pairing the codeword and the feature is only transmitted to the computing device hosting the master version of the feature dictionary when the definition is newly generated or updated in the local version of the feature dictionary—such as generated or updated via the data compression algorithm. This is when both updates to the feature dictionary and the encoded features are sent together. Otherwise, only the encoded features are sent since the master version of the dictionary includes the same definitions as the local versions.

The computing device hosting the master version of the ANN and the master version of the feature dictionary can look up a feature from the master version of the dictionary using the transmitted codeword. And, the computing device can then decode the codeword according to the master version of the dictionary and use the decoded feature for training the master version of the ANN via machine learning.

In some embodiments, the system described herein can include a first computing device hosting a master version of an ANN and a master version of a feature dictionary as well as a second computing device hosting a local version of the ANN and local version of the feature dictionary. And, the second computing device can be one of many computing devices that can host a different local version of the ANN and a different local version of the feature dictionary. The first computing device can be communicatively coupled to the second computing device, and can include memory configured to store the master version of the ANN, and store the master version of the feature dictionary. The first computing device can also include a transceiver configured to receive encoded features from the second computing device. The first computing device can also include a processor configured to: decode the received encoded features according to the master version of the feature dictionary, and train the master version of the ANN based on the decoded features using machine learning. The second computing device can include memory configured to store user data, store a local version of the ANN, and store the local version of the feature dictionary. The second computing device can also include a processor configured to: extract features from the stored user data, and encode the extracted features according to the local version of the feature dictionary. The second computing device can also include a transceiver configured to transmit the encoded features to the first computing device so that the encoded features are decoded by the master version of the feature dictionary and then used as input to train the master version of the ANN using the machine learning.

Additionally, in some embodiments, such as via the encoding, inputs for centralized training of the master version of the ANN can be obfuscated. For example, during or prior to the encoding of the features, the features can be obfuscated. The obfuscation can occur at any one of the multiple computing devices that host different versions of the ANN such as devices hosting local versions of the ANN. For example, the second computing device in the system can include memory that is configured to store a local version of the ANN and user data for inputting into the local version of the ANN. The second computing device can also include a processor that is configured to extract features from the user data and obfuscate the extracted features to generate obfuscated user data during or prior to the encoding of the extracted features. The second device also can include a transceiver that is configured to transmit the obfuscated/encoded user data such as to the first computing device. The first computing device can include a memory that is configured to store the master version of the ANN, a transceiver that is configured to receive obfuscated/encoded user data transmitted from the second computing device or another device of the system hosting a local version of the ANN, and a processor that is configured to train the master version based on the received obfuscated/encoded user data using machine learning after the received obfuscated/encoded user data has been decoded by the master version of the feature dictionary.

In some embodiments, one of the devices hosting a local version of the ANN can input user data, stored in its memory, into the local version of the ANN to use and train the local version of the ANN. For example, this can occur in a mobile device of the user. User data, which may include sensitive or private information, is not shared with other devices in its use with the ANN. To put it another way, the machine learning for the ANN can occur locally and privately in that the user data is not shared with other devices and is secured in the device of the user. However, when the user data is shared with other devices for training of other versions of the ANN, such as a master version, the user data can be obfuscated and/or encoded before it is shared. The obfuscation further secures the privacy of the data and the encoding allows for less network bandwidth to be used since the encoded data is usually smaller than the original data. In some embodiments, only unique local data can be transmitted to the device that hosts the master version of the ANN, so the data transfer size is a subset of the original data.

In some embodiments, one or more devices hosting the master version of the ANN can be part of a cloud computing environment. And, in such embodiments, by obfuscating and/or encoding the user data only in the device of the user, the user data can be kept private from other devices in the cloud. Also, data transmissions can be decreased because of the encoding outputting small data sets. The unmasked or pre-encoded user data can be used to locally train a local version of the ANN on the device of the user. For example, the original user data can be used to train a local version of the ANN on a user's mobile device (e.g., the user's smart phone, tablet, etc.). When it is shared in the system for training of other versions of the ANN it is always at least encoded by the system and sometimes obfuscated as well during or prior to the encoding depending on the embodiment.

In some embodiments, the master version of the ANN can be downloaded to the individual devices of users to update and improve the local versions of the ANN stored on the devices. This can improve the accuracy of the locally stored versions of the ANN in areas of the ANN that may have not been improved by training locally alone. Also, the downloaded master version of the ANN can be further trained locally for a customized version of the ANN for the user. The benefit of this is that when the ANN is locally trained on the user device, the data input does not need to be encoded or obfuscated.

In general, in the computer network that can be configured to implement using of a feature dictionary for centralized training of an ANN and reduction of data transmission, devices that host the differently trained version of the ANN can perform local training to avoid the transmission of user data (which can be large in size) unless the user data is at least encoded by a feature dictionary before transmission (which can reduce the size of the data transmitted significantly). In that in some embodiments only encoded user data can be transmitted over a network communicatively coupling the devices of the network. A central device, server, or cloud can then receive the encoded user data via the network, decode it, and use the decoded data to train the master version of the ANN with limiting use of network bandwidth and in some embodiments without compromising data privacy. Since the central device, server, or cloud does not receive the original user data (such as inputs to local versions of ANN) over the network, trafficking bulky original user data over the network to the central device, server, or cloud and the network can be avoided.

The ANN updated in the computer network can be in the form of updating neuron attributes and/or connectivity. Changes to versions of the ANN can be done through training using compatible machine learning techniques where the input is or at least includes decoded user data that was encoded for improving network communications. Also, the input can be obfuscated during or before the encoding process; however, the obfuscated user data is not obfuscated beyond a point that updating neuron attributes and/or connectivity using the obfuscated input is detrimental to the effectiveness of the ANN in making expected predictions.

FIGS.1and2illustrate the computer network100in a configuration to implement using a feature dictionary for centralized training of an ANN and reduction of data transmission, in accordance with some embodiments of the present disclosure.

InFIG.1, the computer network100is shown including a set of computing devices102. The set of computing devices102can include one or more of any type of computing device capable of hosting and executing a version of an ANN, such as any type of mobile device, personal computer, or smart device that is capable of hosting and executing a version of an ANN (e.g., see second computing device104aand Nth computing device104b). Each computing device of the set of computing devices102can host and execute a local version of an ANN (e.g., see second computing device104aand Nth computing device104bhaving respective local versions of an ANN106aand106b).

The computer network100is also shown including a first computing device108. The first computing device108can be a single computing device or multiple computing devices distributed over a network, such as distributed over a network of a cloud computing environment. The single computing device or the multiple computing devices embodied by first computing device108can include a master version of an ANN110. The first computing device108can be the central device, server, or cloud or a selected device in a peer-to-peer computing network depending on the implementation of the computer network100.

To put it another way,FIG.1illustrates computer network100including the set of computing devices102having networked computing devices (e.g., see computing devices104aand104b), wherein each networked computing devices stores and uses a separate local version of an ANN (e.g., see the local versions of the ANN106aand106b). The computer network100also includes the first computing device108(which can be part of a cloud or another type of distributed computing network). The first computing device108includes the master version of the ANN110.

The local versions of the ANN (e.g., see the local versions of the ANN106aand106b) can be used with user data (e.g., see user data120aand120b). Also, user data can be used in training the local versions of the ANN.

Encoded, transmitted, and then decoded user data can be use by the master version of the ANN110(e.g., see the path of the user data120aand120bvia the encoded features124aand124band the master version of the feature dictionary114) such as for training of the master version. An updated master version of the ANN can then be downloaded by one or more devices of the set of computing devices102(e.g., see downloading130aand130b). The downloaded master version can replace a local version of the ANN or be combined with the local version of the ANN hosted by one of the devices of the set of computing devices102.

In the computer network100, the computing devices hosting the local versions of the ANN and the local versions of the feature dictionary (e.g., see the computing devices of the set of computing devices102) can extract selected features from user data stored on the devices (e.g., see user data120aand120b) and use a data compression algorithm to encode the extracted features from the user data (e.g., see encoded features124aand124b). With the encoding, the data compression algorithm can provide pairs of extracted features and corresponding encoded features (such as codewords). The pairs can be included in the feature dictionary as definitions (e.g., see versions of the feature dictionary122a,122b, and114). The pairing of the original extracted data with its corresponding encoded data can be generated into a local feature dictionary (e.g., see local versions of the feature dictionary122aand122b). Thus, the local versions of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b) provide definitions, in the pairings, for the encoded data (e.g., see encoded features124aand124b). These pairings or definitions allow the encoded data to be decoded into its corresponding original data, such as by a master version of the feature dictionary (e.g., see the master version of the feature dictionary114). Also, the local versions of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b) can be used to update the master version of the feature dictionary (e.g., see the master version of the feature dictionary114). And, the data compression algorithm can be varied in its degree of compression to vary the reduction of data transmission.

In the computer network100, with the sending of the user data from one of the computing devices hosting the local versions of the ANN and dictionary to the computing device hosting the master versions of the ANN and dictionary (e.g., see dictionary updates/encoded features126aand126b), instead of sending a full description of a feature of the user data (such as uncompressed user data) a codeword can be sent. Each time the feature is sent to the computing device hosting the master versions of the ANN and dictionary (e.g., see first computing device108), a codeword for the feature can be defined in the local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b) and sent accordingly, such that only the corresponding codeword is transmitted over the network (e.g., dictionary updates/encoded features126aand126bcan sometimes only include encoded features). In some embodiments, a definition pairing the codeword and the feature is only transmitted to the computing device hosting the master version of the feature dictionary when the definition is newly generated or updated in the local version of the feature dictionary—such as generated or updated via the data compression algorithm. This is when both updates to the feature dictionary and the encoded features are sent together (e.g., dictionary updates/encoded features126aand126bcan sometimes include encoded features as well as corresponding definitions). Otherwise, only the encoded features are sent since the master version of the dictionary includes the same definitions as the local versions.

The computing device hosting the master version of the ANN and the master version of the feature dictionary (e.g., see first computing device108) can look up a feature from the master version of the dictionary (e.g., see the master version of the feature dictionary114) using the transmitted codeword (e.g., see dictionary updates/encoded features126aand126b). And, the computing device can then decode the codeword according to the master version of the dictionary and use the decoded feature for training the master version of the ANN via machine learning (e.g., see the master version of the ANN110and machine learning112).

The master version of the ANN (e.g., see the master version of the ANN110) can be trained over and over again by received feature dictionary updates and encoded data (e.g., see the dictionary updates/encoded features126aand126b) via machine leaning (e.g., see machine learning112) such that the master version is generic and becomes more and more accurate over time. The data can be received from the devices of different users having the different and local versions of the ANN (e.g., see local versions of the ANN106aand106b) and different stored user data (e.g., see user data120aand120b). The master version of the ANN (e.g., see the master version of the ANN110) can be downloaded to the individual user devices (e.g., see computing devices104aand104b) to update the user devices capability to benefit from advances in areas that may not have been trained locally (e.g., see downloading130aand130b). Also, the downloaded ANN can be further trained locally for a customized version of the ANN for the user.

Not shown, the computer network100includes a communications network that includes a wide area network (WAN), a local area network (LAN), an intranet, an extranet, the Internet, and/or any combination thereof. The communications network can communicatively couple the devices of the set of computing devices102with each other and with other devices of the computer network100such as with the first computing device108. The sent feature dictionary updates and encoded user data as well as the downloads of the master version of the ANN mentioned herein (e.g., see the dictionary updates/encoded features126aand126band downloading130aand130b) can be communicated or transmitted over the communications network of the computer network100. Also, updates to the master version of the feature dictionary mentioned herein (e.g., see downloading128aand128b) can be communicated or transmitted over the communications network of the computer network100.

FIG.2is somewhat similar toFIG.1in that it shows computer network100in a configuration to implement using of a feature dictionary for centralized training of an ANN and reduction of data transmission. However, different fromFIG.1,FIG.2depicts specifically the set of computing devices102including mobile devices (e.g., see mobile devices105aand105b) hosting respective local versions of the ANN (e.g., see the local versions of the ANN106aand106b). Also, inFIG.2, it is shown that a cloud computing environment (e.g., cloud computing environment109) is hosting the master version of the ANN (e.g., see the master version of the ANN110). In other words,FIG.2is a more specific implementation of the computer network shown inFIG.1, where the first device108is or is part of the cloud computing environment109and the devices of the set of computing devices102are specifically mobile devices (e.g., see mobile devices105aand105b).

In some embodiments, a system for implementing use of a feature dictionary for centralized training of an ANN and reduction of data transmission can include a plurality of computing devices (e.g., see set of computing devices102). Each one of the plurality of computing devices can be configured to host a local version of the ANN (e.g., see local versions of the ANN106aand106b) and host a local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b). The system can also include a first computing device (e.g., see first computing device108or cloud computing environment109) configured to host a master version of the ANN (e.g., see the master version of the ANN110) and host a master version of the feature dictionary (e.g., see the master version of the feature dictionary114).

For example, the plurality of computing devices can include a second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b) which can host the local version of the ANN (e.g., see local versions of the ANN106aand106b) and host the local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b). And, the first computing device (e.g., see first computing device108or cloud computing environment109) can be communicatively coupled to the second computing device over a computer network (e.g., see computer network100).

The first computing device (e.g., see first computing device108or cloud computing environment109) can include memory (e.g., see main memory608and storage system612shown inFIG.6) configured to: store the master version of the ANN (e.g., see the master version of the ANN110); and store the master version of the feature dictionary (e.g., see the master version of the feature dictionary114). The first computing device can also include a transceiver (e.g., see network interface610shown inFIG.6) configured to receive encoded features from the second computing device (e.g., see encoded features124aand124bas well as dictionary updates/encoded features126aand126b). Also, the first computing device (e.g., see first computing device108or cloud computing environment109) can include a processor (e.g., see processor606shown inFIG.6) configured to: decode the received encoded features according to the master version of the feature dictionary (e.g., see the master version of the feature dictionary114); and train the master version of the ANN (e.g., see the master version of the ANN110) based on the decoded features using machine learning (e.g., see machine learning112).

In some embodiments, the transceiver (e.g., see network interface610) of the first computing device (e.g., see first computing device108or cloud computing environment109) can be configured to transmit the trained master version of the ANN to the second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b), such as shown by downloading130aand130b. Also, the transceiver (e.g., see network interface610) of the first computing device can be configured to receive the local version of the feature dictionary from the second computing device (e.g., see encoded features124aand124bas well as dictionary updates/encoded features126aand126b). In such an example, the processor (e.g., see processor606) of the first computing device can be configured to: change the master version of the feature dictionary (e.g., see the master version of the feature dictionary114) based on the received local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122bas well as dictionary updates/encoded features126aand126b); and decode the received encoded features according to the changed master version of the feature dictionary (e.g., see the master version of the feature dictionary114). Also, the transceiver (e.g., see processor606) of the first computing device can be configured to transmit the changed master version of the feature dictionary to the second computing device (e.g., see downloading128aand128b).

The second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b) can include memory (e.g., see main memory608and storage system612) configured to: store user data (e.g., see user data120aand120b); store a local version of the ANN (e.g., see local versions of the ANN106aand106b); and store a local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b). The second computing device can also include a processor (e.g., see processor606) configured to: extract features from the stored user data (e.g., see user data120aand120b); and encode the extracted features according to the local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122band encoded features124aand124b). Also, the second computing device can include a transceiver (e.g., see network interface610shown inFIG.6) configured to transmit the encoded features (e.g., see encoded features124aand124b) to the first computing device so that the encoded features are decoded by the master version of the feature dictionary (e.g., see the master version of the feature dictionary114) and then used as input to train the master version of the ANN using the machine learning (e.g., see machine learning112).

In some embodiments, the processor (e.g., see processor606) of the second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b) can be further configured to determine whether the extracted features are included in the local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b). And, in response to the local version of the feature dictionary including the extracted features, the processor of the second computing device can be further configured to encode the extracted features according to the local version of the feature dictionary (e.g., see encoded features124aand124b). In response to the local version of the feature dictionary not including the extracted features, the processor of the second computing device can be further configured to change the local version of the feature dictionary based on the extracted features and then encode the extracted features according to the changed local version of the feature dictionary (e.g., see dictionary updates/encoded features126aand126b).

In such examples, the transceiver (e.g., see network interface610) of the second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b) can be configured to transmit the changed local version of the feature dictionary to the first computing device so that the master version of the feature dictionary is changed according to the local version of the feature dictionary (e.g., see dictionary updates/encoded features126aand126b). Also, the transceiver of the second computing device can be configured to receive the changed master version of the feature dictionary (e.g., see downloading128aand128b). In such embodiments, a processor (e.g., see processor606) of the second computing device can be configured to change the local version of the feature dictionary (e.g., see local versions of the feature dictionary122aand122b) based on the changed master version of the feature dictionary (e.g., see the master version of the feature dictionary114).

Additionally, in some embodiments, the transceiver (e.g., see network interface610) of the second computing device (e.g., see computing devices104aand104bas well as mobile devices105aand105b) can be configured to receive the trained master version of the ANN (e.g., see downloading130aand130b). In such embodiments, the processor (e.g., see processor606) of the second computing device can be configured to change the local version of the ANN (e.g., see local versions of the ANN106aand106b) based on the trained master version of the ANN (e.g., see the master version of the ANN110and downloading130aand130b).

FIGS.3and4illustrate example methods performed by example parts of computer network100that can implement using of a feature dictionary for centralized training of an ANN and reduction of data transmission, in accordance with some embodiments of the present disclosure.

FIG.3shows the method300performed by a computing device configured to host a master version of an ANN and use of a feature dictionary for centralized training of the ANN and reduction of data transmission (e.g., see first computing device108or cloud computing environment109) that is in communication with other computing devices that host other versions of the ANN (e.g., see computing devices104aand104bas well as mobile devices105aand105b). The method300begins, at step302, with hosting, by a first computing device (or the computing device), a master version of an ANN.

At step304, the first computing device hosts a master version of a feature dictionary.

At step306, the first computing device receives a local version of a feature dictionary from a second computing device which is one of the other computing devices hosting other versions of the ANN and the feature dictionary.

At step308, the first computing device changes the master version of the feature dictionary based on the received local version of the feature dictionary.

At step310, the first computing device receives encoded features from the second computing device. The received encoded features can be encoded by the second computing device according to a local version of the feature dictionary hosted by the second computing device.

At step312, the first computing device decodes the received encoded features according to the master version of the feature dictionary. The decoding can include decoding the encoded features according to the changed master version of the feature dictionary.

At step314, the first computing device trains the master version of the ANN based on the decoded features using machine learning.

At step316, the first computing device transmits the trained master version of the ANN to the second computing device.

At step318, the first computing device transmits the changed master version of the feature dictionary to the second computing device.

FIG.4shows the method400performed by a computing device configured to host another version of the ANN other than the master version of the ANN (such as a local version) and use of a local feature dictionary for assisting with centralized training of the ANN and reduction of data transmission (e.g., see computing devices104aand104bas well as mobile devices105aand105b) that is in communication with the first computing device that can perform the method300(e.g., see first computing device108or cloud computing environment109). The method400begins, at step402, with hosting, by the second computing device, a local version of the ANN.

At step404, the second computing device hosts a local version of the feature dictionary.

At step406, the second computing device extracts features from user data hosted by the second computing device.

At step408, the second computing device determines whether the extracted features are included in the local version of the feature dictionary. In response to the local version of the feature dictionary including the extracted features, the second computing device encodes the extracted features according to the local version of the feature dictionary, at step412. In response to the local version of the feature dictionary not including the extracted features, the second computing device changes the local version of the feature dictionary based on the extracted features, at step410, and then encodes the extracted features according to the changed local version of the feature dictionary, at step412.

At step414, the second computing device transmits the encoded features to the first computing device that hosts a master version of the ANN so that the encoded features are decoded by the master version of the feature dictionary hosted by the first computing device and then used as input to train the master version of the ANN using machine learning.

At step416, the second computing device transmits the changed local version of the feature dictionary to the first computing device so that the master version of the feature dictionary is changed according to the local version of the feature dictionary.

At step418, the second computing device receives the trained master version of the ANN. And, at step420, the second computing device changes the local version of the ANN based on the trained master version of the ANN.

At step422, the second computing device receives a changed master version of the feature dictionary that was changed according to the local version of the feature dictionary. And, at step424, the second computing device changes the local version of the feature dictionary based on the changed master version of the feature dictionary.

FIG.5shows the method500performed by computing devices configured to host and execute a master version of an ANN (e.g., see first computing device108and cloud computing environment109depicted inFIGS.1and2respectively) and computing devices configured to host and execute other versions of the ANN (e.g., see computing devices104aand104bor mobile devices105aand105b). Method500can include the operations of methods300and400.

The method500begins, at step502, with a computing device configured to host another version of the ANN besides the master version of the ANN (i.e., the second computing device) extracting features from user data stored in memory of the second computing device. At step504, the second computing device (e.g., see computing devices104aand104bor mobile devices105aand105b) generates a local version of a feature dictionary based on the extracted features, if no feature dictionary exists on the second computing device after an initial extraction at step502. Otherwise, at step504, the second computing device updates a local version of the feature dictionary based on the extracted features from the user data. At step506, the second computing device encodes the extracted features according to the local version of the feature dictionary.

At step508, the second computing device transmits the local version of the feature dictionary to a computing device configured to host a master version of the ANN (i.e., a first computing device). The second computing device can be configured to transmit the local version of the feature dictionary to the first computing device only when the local version has been initially generated or when the local version has been updated. Also, when the local version of the feature dictionary is updated, the second computing device can be configured to only transmit the updated portions of the feature dictionary to the first computing device. Also, at step508, the second computing device transmits the encoded features. The second computing device can be configured to only send the recently encoded features.

At step510, the first computing device receives the local version of the dictionary if it was sent from the second computing device. Also, at step510, the first computing device receives the encoded features if the encoded features were sent from the second computing device. At step512, the first computing device changes the master version of the feature dictionary based on the received local version of the feature dictionary if the local version of the dictionary is received. The receiving of the local version can trigger the changing of the master version of the dictionary.

At step514, the first computing device decodes the encoded features according to the master version of the dictionary if the encoded features are received. The receiving of the encoded features can trigger the decoding of the encoded features.

At step516, the first computing device trains the master version of the ANN based on the decoded features using machine learning. And, at step518, the first computing device transmits the trained master version of the ANN and/or the changed master version of the dictionary to the second computing device. A step520, the second computing device receives the trained master copy of the ANN and/or receives the changed master version of the dictionary according to the events that occur at step518.

Not shown inFIG.5, the local version of the ANN can be trained or re-trained by combining the updated master version with a present local version of the ANN in the user device. And, the combining in the training or re-training can include updates to neuron attributes of the local version of the ANN according to averaging corresponding attributes of the updated master version of the ANN and the “present” or previous local version of the ANN.

For the purposes of steps502to520ofFIG.5and this disclosure, it is to be understood that updates to the local version of the feature dictionary and encoded user data (e.g., see dictionary updates/encoded features126aand126b) that are transmitted, at step508, from the user devices (e.g., see computing devices104aand104bor mobile devices105aand105bshown inFIG.2) to the central device, server, or cloud (e.g., see first computing device108or cloud computing environment109) hosting the master version of the ANN can be in or derived from the form of the entire extracted features extracted in step502or parts of the extracted features extracted in step502. Also, in some embodiments, the central device, server, or cloud hosting the master version of the ANN can limit what the user devices can extract at step502and generate and encode at steps504and506respectively to adjust the network performance provided by the encoding and possibly obfuscation as well as on the other hand adjust the accuracy of the training of the version of the ANN (such as the master version) using the encoded and sometimes obfuscated user data as input for the training.

Also, it is to be understood that a peer-to-peer network can implement the method500. In such examples, a first selected device hosts the master version of the ANN for processing and the other devices of the peer-to-peer network host the other versions (e.g., local versions) of the ANN for processing. In such examples, the first selected device executes steps510to518and the other devices of the peer-to-peer network execute steps502to508and520.

With respect to the method300, method400, method500or any other method, process, or operation described herein, in some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by at least one processing device (such as processor606shown inFIG.6), cause the at least one processing device to perform the method300, method400, method500or any other method, process, or operation described herein, and/or any combination thereof.

FIG.6illustrates example parts of an example computing device600, in accordance with some embodiments of the present disclosure. The computing device600can be communicatively coupled to other computing devices via the computer network100as shown inFIG.6. In some embodiments, computing device600is the first computing device108or one or more computing devices of the cloud computing environment109. In such embodiments, another instance of the computing device600is one of the computing devices of the set of computing devices102(e.g., see computing devices104aand104band mobile devices105aand105b). The computing device600includes at least a bus604, a processor606(such as a CPU), a main memory608, a network interface610, and a data storage system612. The bus604communicatively couples the processor606, the main memory608, the network interface610, and the data storage system612. The computing device600includes a computer system that includes at least processor606, main memory608(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), static random access memory (SRAM), etc.), and data storage system612, which communicate with each other via bus604(which can include multiple buses).

To put it another way,FIG.6is a block diagram of an example computing device600having a computer system in which embodiments of the present disclosure can operate. In some embodiments, the computer system can include a set of instructions, for causing a machine to perform any one or more of the methodologies discussed herein, when executed. In such embodiments, the machine can be connected (e.g., networked via network interface610) to other machines in a LAN, an intranet, an extranet, and/or the Internet. The machine can operate in the capacity of a server or a client machine in client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment (such as the peer-to-peer networks described herein), or as a server or a client machine in a cloud computing infrastructure or environment.

Processor606represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processor606can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, a processor in memory (PIM), or the like. Processor606is configured to execute instructions for performing the operations and steps discussed herein. Processor606can further include a network interface device such as network interface610to communicate over one or more communications network.

The data storage system612can include a machine-readable storage medium (also known as a computer-readable medium) on which is stored one or more sets of instructions or software embodying any one or more of the methodologies or functions described herein. The instructions can also reside, completely or at least partially, within the main memory608and/or within the processor606during execution thereof by the computer system, the main memory608and the processor606also constituting machine-readable storage media. While the memory, processor, and data storage parts are shown in the example embodiment to each be a single part, each part should be taken to include a single part or multiple parts that can store the instructions and perform their respective operations. The term “machine-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure. The term “machine-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.

The peer-to-peer network of some embodiments can be a collection of nodes and peer-to-peer connections. For example, the first computing device108or one of computing devices of the set of computing devices102can be a node of a peer-to-peer network supported by computing devices connected through computer network100.

Some portions of the preceding detailed descriptions have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure can refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage systems.

The present disclosure also relates to an apparatus for performing the operations herein. This apparatus can be specially constructed for the intended purposes, or it can include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct a more specialized apparatus to perform the method. The structure for a variety of these systems will appear as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the disclosure as described herein.

The present disclosure can be provided as a computer program product, or software, that can include a machine-readable medium having stored thereon instructions, which can be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium such as a read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory components, etc.

In the foregoing specification, embodiments of the disclosure have been described with reference to specific example embodiments thereof. It will be evident that various modifications can be made thereto without departing from the broader spirit and scope of embodiments of the disclosure as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.