Systems and methods for online adaptation for cross-domain streaming data

Embodiments described herein provide an online domain adaptation framework based on cross-domain bootstrapping for online domain adaptation, in which the target domain streaming data is deleted immediately after adapted. At each online query, the data diversity is increased across domains by bootstrapping the source domain to form diverse combinations with the current target query. To fully take advantage of the valuable discrepancies among the diverse combinations, a set of independent learners are trained to preserve the differences. The knowledge of the learners is then integrated by exchanging their predicted pseudo-labels on the current target query to co-supervise the learning on the target domain, but without sharing the weights to maintain the learners' divergence.

TECHNICAL FIELD

The present disclosure relates generally to machine learning models and stream data processing, and more specifically, to online adaptation for cross-domain streaming data.

BACKGROUND

Internet users often tend to leave significant amounts of digital footprints, e.g., by sharing texts, photos, videos or other forms of media via social media, via purchase orders and browsing history, via share ride histories, via registration at a website, and/or the like. As a lot of user shared data may exist on the Internet for an extended period of time, online privacy of users become increasingly difficult to preserve. Some recommendation systems may actively use user data for mining user interests and developing data-driven algorithms, only rendering the right to privacy more challenging. The Right to Be Forgotten (RTBF) movement grants Internet the right to ask an organization to delete their personal data.

Therefore, there is a need to for an efficient data privacy protection mechanism.

DETAILED DESCRIPTION

As used herein, the term “network” may comprise any hardware or software-based framework that includes any artificial intelligence network or system, neural network or system and/or any training or learning models implemented thereon or therewith.

As used herein, the term “module” may comprise hardware or software-based framework that performs one or more functions. In some embodiments, the module may be implemented on one or more neural networks.

Recommendation systems often actively use user data for mining user interests and developing data-driven algorithms, creating an issue that violates the right to privacy. Some existing systems may attempt to try to preserve user privacy via Federated Leaning, which limits sensitive data to be stored only on a few specific nodes. However, private training data for Federated Learning can be leaked through the gradients sharing mechanism deployed in distributed models, thus forfeiting the purpose of privacy protection. Ideally, a data privacy protection algorithm would delete user data right after use, but existing online learning frameworks cannot meet this need without addressing the distribution shift from public data (source domain) to private user data (target domain).

Embodiments described herein provide an online domain adaptation framework based on cross-domain bootstrapping for online domain adaptation, in which the target domain streaming data is deleted immediately after adapted. At each online query, the data diversity is increased across domains by bootstrapping the source domain to form diverse combinations with the current target query. To fully take advantage of the valuable discrepancies among the diverse combinations, a set of independent machine learning models (referred to as “learners”) are trained to preserve the differences. The knowledge of the learners are then integrated by exchanging their predicted pseudo-labels on the current target query to co-supervise the learning on the target domain, but without sharing the weights to maintain the learners' divergence.

In one embodiment, at inference/testing stage, a more accurate prediction may be obtained for the current target query by an average ensemble of the diverse expertise of all the learners. In this way, the right to be forgotten of each user can be realized by deleting the target querying right after the training or testing, while knowledge contained in the target query can be transferred to the public training data (source domain) for the learners.

For example, prediction model may be trained to identify an object in an input image, e.g., a “check shirt,” a “sneaker,” a “spaghetti strap top,” and/or the like. Source domain data (e.g., public data including different types of clothing items) from Fashion-MNIST (Xiao et aL, Fashionmnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017) may be used to train the prediction model. In addition, sensitive data from a target domain, e.g., photos of people wearing different types of garments shared on social media, may also be included in the training and testing of the prediction model. Such a data query in the target domain (e.g., a photo of a person wearing a type of garment) may be incorporated into the training back to increase the cross-domain data diversity. Two copies of the prediction models may generate a prediction on the data query, e.g., a predicted type of garment that the person in the photo is wearing, which can be used as pseudo labels to supervise the training. Specifically, the two copies of prediction models exchange the generated pseudo-labels as co-supervision. Once the current query (e.g., the photo of the person wearing the type of garment) is adapted in the training phase, the query is deleted after being tested. In this way, the user photo (even if the user voluntarily shares his or her photo on the Internet) is erased from the training data.

FIG.1is a simplified diagram of a cross-domain bootstrapping framework100for online domain adaptation, according to one or more embodiments described herein. At each iteration, the training framework100adapts from a public source domain105to a streaming online target domain112. At each iteration, only the current j-th query112ais available from the target domain112. The framework100then bootstraps the source domain batches110aor110bthat combine with the current j-th query112ain order to increase the cross-domain data diversity. The learners wuand wv115a-bexchange the generated pseudo-labels ŷju118aand ŷjv118bas co-supervision to compute the training objectives. Once the current query is adapted in the training phase shown, it is tested immediately to make a prediction. Each target query is deleted after tested, e.g., at130.

Specifically, the labeled source data DS={(si, yi)}i=1NS105may include public data106(e.g., a public image dataset) drawn from source distribution ps(x, y), where NSrepresents the number of image samples in the source dataset105. For example, the public dataset105may be a public dataset such as Fashion-MNIST, which contains good quality and annotated image samples106.

The unlabeled target data DT={ti}i=1NT112may include user sensitive data (e.g., user shared photos on social media, and/or the like) drawn from the target distribution pt(x, y), where NTrepresent the number of image samples in the target dataset112. An (offline or online) adaptation approach aims at training a prediction model, e.g., a classifier115aor115b(collectively referred to as115) that makes accurate predictions on DT.

For example, offline adaptation assumes access to every data point in DSor DT, synchronous or asynchronous domain-wise. The inference on DThappens after the prediction model115is trained on both DSand DTentirely. For online adaptation, access to the entire DSis assumed, while the data from DTarrives in a random streaming fashion of mini-batches {Tj={tb}b=1B}j=1MT, where B is the batch size, MTis the total number of target batches. Each mini-batch T is first adapted, tested and then erased from DTwithout replacement. As used herein, each online batch, which contains one or more image samples, may be referred to as a target query112a.

One challenge of online adaptation is the limited access to the training data at each query, compared to offline adaptation. For example, for online adaptation, the target queries from the target dataset112may be received sequentially on a one-by-one basis while the model115is being trained. Assuming there are 103source and target batches respectively. In an offline setting, the model115is tested after training on at most 106combinations of source-target data pairs, while in an online setting, a one-stream model can see at most 103+500 combinations at the 500-th query. Thus, online adaptation faces a significantly smaller data pool and data diversity, and the training process of the online task suffers from two major drawbacks. First, the model is prone to underfitting on target domain due to the limited exposure, especially at the beginning of training. Second, due to the erasure of “seen” batches, the model lacks the diverse combinations of source-target data pairs that enable the deep network to find the optimal cross-domain classifier. In view of the challenge, the training framework100inFIG.1increases the data diversity by cross-domain bootstrapping, which are preserved in multiple independent copies of the classifier (e.g., learners115a-b). Then valuable discrepancies of these learners115a-bare explored by exchanging their expertise on the current target query to co-supervise each other.

In the online setting, the target samples cannot be reused in the training due to user privacy concern. Data diversity may be increased across domains by bootstrapping the source domain to form diverse combinations with the current target domain query. Specifically, for each target query Tj112a, a set of K mini-batches110a-bare randomly selected, e.g., {Sjk={(sb)b=1B}}k=1Kof the same size from the source domain with replacement. Correspondingly, a set of K base learners {wk}k=1K115a-bwhich are the copies of a classifier are obtained. It is worth noting that the framework100inFIG.1shows only two training branches of two learners115a-bfor illustrative purpose only. In other examples, another number K (K=3, 4, 5, . . . ) of learners may be trained from K mini-batches of data in parallel.

As shown in framework100, at each iteration, a learner wk115aor115bis first trained on the combination of source data110aand the target query112a, e.g., {Tj, Sjk}. Specifically, the independent learners115a-bmay have preserved the valuable discrepancies of cross-domain pairs, the framework100aims at integrating the learners' expertise into one better prediction on the current target query112a. The K learners115a-bmay be trained jointly by exchanging their knowledge on the target domain as a form of co-supervision.

In one embodiment, the K learners (e.g.,115a-b) are trained independently with bootstrapped source supervision, but they exchange the pseudo-labels generated for target queries112a. For example, when K=2 for simplicity, the learners are denoted as wu115aandwv115b. Given the current target query Tj112a, the loss function L for each learner115a-bconsists a supervised loss termsfrom the source domain with the bootstrapped samples {(sb)b=1B}, and a self-supervised loss termtfrom the target domain with pseudo-labels ŷb(e.g.,118aor118b) from the peer learner. In the example shown inFIG.1, if the cross-entropy between two probability distributions is denoted as H(;), the self-supervised loss termtfor each learner115aor115bis computed as:

After computing the loss terms, the learners115a-bare then updated by:

FIG.2is a simplified diagram of a testing framework200for the cross-domain bootstrapping framework100for online domain adaptation shown inFIG.1, according to one or more embodiments described herein. After the learners115a-bare trained on {Tj, Sjk}, the trained learners115a-bmay generate a predicted class for Tj112aby taking an average120of K predictions of the base learners (e.g.,115a-b). For example, the averaged prediction class may yield the testing result125, e.g., a predicted class of “Shirt.”

The testing may be performed at the end of the iteration. In this way, each iteration handles a target query112afrom the target dataset112, and comprises a training stage shown inFIG.1, and a testing stage shown inFIG.2. At the end of the iteration, the target query is erased, e.g., as shown at130inFIG.1.

Therefore, to obtain a good estimation of the current target query112a, the bootstrapping framework100for uncertainty estimation on the target domain112offsets the source dominance. With the scarcity of target samples112a-n, source-target data pairs are bootstrapped for a more balanced cross-domain simulation. At a high-level, the bootstrap simulates multiple realizations of a specific target query given the diversity of source samples. Specifically, the bootstrapped source approximate a distribution over the current query Tj112avia the bootstrap.

In this way, the bootstrapping framework200brings multi-view observations on a single target query112aby two means. First, given K sampling subsets from DS, if lettingbe the ideal estimate of Tj,be the practical estimate of the dataset, and* be the estimate from a bootstrapped source paired with the target query,

ℱ^*=K-1⁢∑k=1Kℱ^k*
will be the average of the multi-view K estimates. Second, besides the learnable parameters, the Batch-Normalization layers of K learners generate result in a set of different means and variances {μk, σk}k=1Kthat serve as K different initializations that affects the learning of*.

FIG.3is a simplified diagram of a computing device for implementing the cross-domain adaptation, according to some embodiments. As shown inFIG.3, computing device300includes a processor310coupled to memory320. Operation of computing device300is controlled by processor310. And although computing device300is shown with only one processor310, it is understood that processor310may be representative of one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs) and/or the like in computing device300. Computing device300may be implemented as a stand-alone subsystem, as a board added to a computing device, and/or as a virtual machine.

Processor310and/or memory320may be arranged in any suitable physical arrangement. In some embodiments, processor310and/or memory320may be implemented on a same board, in a same package (e.g., system-in-package), on a same chip (e.g., system-on-chip), and/or the like. In some embodiments, processor310and/or memory320may include distributed, virtualized, and/or containerized computing resources. Consistent with such embodiments, processor310and/or memory320may be located in one or more data centers and/or cloud computing facilities.

In some examples, memory320may include non-transitory, tangible, machine readable media that includes executable code that when run by one or more processors (e.g., processor310) may cause the one or more processors to perform the methods described in further detail herein. For example, as shown, memory320includes instructions for an online adaptation module330that may be used to implement and/or emulate the systems and models, and/or to implement any of the methods described further herein. In some examples, the online adaptation module330, may receive an input340, e.g., such as source data or a target query via a data interface315. The online adaptation module330may generate an output350in response to the input340. For example, the output350may be a prediction label of the input target query.

In some examples, the online adaptation module330may include a plurality of prediction models331a-331n, which may be a number of copies of a prediction model (e.g., see115a-bas an example). The cross-domain adaptation module330and its submodules prediction models331a-nmay be implemented using hardware, software, and/or a combination of hardware and software.

FIG.4is a simplified logic flow diagram illustrating a method of online cross-domain adaptation from a public domain to a target domain containing sensitive data, according to one or more embodiments described herein. One or more of the processes of method400may be implemented, at least in part, in the form of executable code stored on non-transitory, tangible, machine-readable media that when run by one or more processors may cause the one or more processors to perform one or more of the processes. In some embodiments, method400corresponds to the operation of the video pretraining module330(FIG.3) to perform contrastive pretraining with video tracking supervision.

At step402, a training dataset of data samples (e.g., public data106inFIG.1) in a source domain is received via a data interface (e.g.,315inFIG.3).

At step404, a first target query (e.g.,112ainFIG.1) from a streaming sequence of target queries in a target domain (e.g.,112inFIG.1) is received, at a first iteration. For example, the first target query comprises a set of sequentially received data samples from the streaming sequence.

In one embodiment, a first set of data samples and a second set of data samples are sampled from the training dataset in the source domain. Each has a same number of data samples as the set of sequentially received data samples. The first set of data samples or the second set of data samples are then sent as the input to the first prediction model or the second prediction model, respectively.

At step406, a first prediction model (e.g.,115ainFIG.1) generates a first prediction (e.g.,118ainFIG.1) in response to the first target query.

At step408, a second prediction model (e.g.,115binFIG.1) generates a second prediction (e.g.,118binFIG.1) in response to the first target query.

At step410, the first prediction model and the second prediction model generate a first output and a second output in response to an input from the data samples in the source domain, respectively.

At step412, a first loss objective is computed based on the first output and a corresponding label from the training dataset, e.g., the supervised loss lsu. For example, the first loss objective is computed as a supervised loss objective supervised by pre-annotated labels of the data samples in the training dataset.

At step414, a second loss objective based on the first prediction using the second prediction as a first pseudo label, e.g., the co-supervised loss ltv→u. The second loss objective is computed by: generating the first pseudo label based on the second prediction; and in response to determining that the second prediction is greater than a pre-defined threshold, computing a cross entropy between the first pseudo label and a distribution of the first prediction.

At step416, the first prediction model is updated based at least in part on the first loss objective and the second loss objective.

Similarly, the second prediction model may be updated by performing similar steps as steps412-416. For example, a third loss objective is computed based on the second output and the corresponding label from the training dataset. A fourth loss objective is computed based on the second prediction using the first prediction as a second pseudo label. The second prediction model is updated based at least in part on the third loss objective and the fourth loss objective.

In one embodiment, an entropy loss objective may be computed based on a distribution of the first prediction. A class-balancing diversity loss objective may be computed based on the distribution of the first prediction. The first prediction model may be jointly updated based on the first loss objective, the second loss objective, the entropy loss objective and the class-balancing diversity loss objective.

At step418, the first target query (e.g.,112ainFIG.1) is erased at an end of the first iteration from a memory.

Example Performance

Two metrics have been adopted for evaluating online domain adaptation methods described inFIGS.1-4: online average accuracy and one-pass accuracy. The online average is an overall estimate of the streaming effectiveness. The one-pass accuracy measures after training on the finite-sample how much the online model has deviated from the beginning. A one-pass accuracy much lower than online average indicates that the model might have overfitted to the fresh queries, but com-promised its generalization ability to the early queries. Dataset. VisDA-C (Visda: A synthetic-to-real benchmark for visual domain adaptation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018), a classic benchmark adapting from synthetic images to real. The COVID-DA (Zhang et al., Covid-da: Deep domain adaptation from typical pneumonia to covid-19. arXiv preprint arXiv:2005.01577, 2020), adapts the CT images diagnosis from common pneumonia to the novel disease. This is a typical scenario where online domain adaptation is valuable in practice. When a novel disease breaks out, without any prior knowledge, one has to exploit a different but correlated domain to assist the diagnosis of the new pandemic in a time-sensitive manner.

The online cross-domain adaptation is evaluated on a large-scale medical dataset Camelyon17 from the WILDS (Koh et aL, Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, 2021), a histopathology image dataset with patient population shifts from source to the target. Camelyon17 has 455 k samples of breast cancer patients from 5 hospitals. Another practical scenario is the online fashion where the user-generated content (UGC) might be time-sensitive and cannot be saved for training purposes. Due to the lack of cross-domain fashion prediction dataset, adaptation from Fashion-MNIST to-DeepFashion category pre-diction branch is evaluated. For example, 6 fashion categories are selected shared between the two datasets, and design the task as adapting from 36,000 grayscale samples of Fashion-MNIST to 200, 486 real-world commercial samples from DeepFashion.

In one embodiment, the cross-domain adaptation framework100is implemented using Pytorch. ResNet-101 is used on VisDA-C pretrained on ImageNet. Pretrained ResNet-18 (He et al., Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016) is used on COVID-DA. DenseNet-121 (Huang et al., Densely connected convolutional networks, In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017) is used on Camelyon17 with random initialization, and the official WILDS codebase 3 is used for data split and evaluation. The pretrained ResNet-101 is used on Fashion-MNIST-to-DeepFashion. The confidence thresh-old τ=0.95 and diversity weight λ=0.4 are fixed throughout the experiments.

Example baseline models for comparison include DAN (Long et al., Learning transferable features with deep adaptation networks, in International conference on machine learning, 2015), CORAL (Sun et aL, Return of frustratingly easy domain adaptation, in AAAI, 2016), DANN (Domain-adversarial training of neural networks, in proceedings of JMLR, 2016), ENT (Grandvalet et aL, Semi-supervised learning by entropy minimization, CAP, 2005), MDD (Zhang), CDAN (Long et al., Conditional adversarial domain adaptation, arXiv preprint arXiv:1705.10667, 2017), SHOT and ATDOC (Liang et al., Domain adaptation with auxiliary target domain-oriented classifier, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021). ATDOC has multiple variants of the auxiliary regularizer, which is compared with the Neighborhood Aggregation (ATDOC-NA) with the best performance. Among the compared approaches, SHOT and ATDOC-NA require a memory module that collects and stores information of all the target samples, thus only apply the offline setting. For the other six approaches, we compare both offline and online results. Each offline model is trained for 10 epochs, and each online model takes the same randomly-perturbed target queries to make a fair comparison.

FIG.5shows the comparison in online accuracy on VisDA-C, and performances are plotted inFIG.8. To make a fair comparison, the VisDA-C one-pass accuracy in class average is provided. Among the offline methods, SHOT and ATDOC-NA largely outperforms other approaches, showing the effectiveness of target domain clustering. Regarding the on-line setting, the Source-Only baseline loses 2.4% in the on-line average and 7.2% in the one-pass accuracy, which in-dicatates that the data diversity is also important in domain generalization. ENT, which is an entropy regularizer on the posterior probabilities of the un-labeled target samples, has a noticeable performance drop in the online setting, and illustrates obvious imbalanced results over the categories (superior at class “knife” but poor at “person” and “truck”). IT can be considered a typical example of bad objective choice for the online setting when the dataset is imbalanced. Without sufficient rounds to provide data diversity, entropy minimization might easily overfit the current target query. The 2.5% drop in one-pass from online further confirmed the model has deviated from the beginning. The cross-domain adaptation method (referred to as “CRODOBO”) outperforms other methods in the online setting by a large margin. For the sake of time efficiency, CRODOBO is superior than other approaches by achieving high accuracy in only one epoch.

The results on two medical imaging datasets COVID-DA and WILDS-Camelyon17 are respectively summarized inFIG.6andFIG.7. The online streaming ac-curacy is presented inFIG.9. COVID-DA* is the method proposed along with the dataset, which is a domain adversarial-based multi-classifier approach with focal loss regularization. CRODOBO outperforms the other approaches on COVID-DA regarding the online and one-pass metric, and achieves competitive performance against the best offline accuracy. On the large-scale benchmark WILDS-Camelyon17, CRODOBO outperforms the offline results by 1.7%, which validates the effectiveness of the approach. CRODOBO also obtains more competitive results on large-scale datasets than small ones. Compared to the COVID-DA dataset (2,029 target samples), WILDS-Camelyon17 (85,054 target samples) is significantly more extensive. The good performance on larger number of target queries indicates that CRODOBO can well exploit the underlying information from the target domain. Similar observations are made on the large-scale Fashion benchmark.

The results on the newly proposed large-scale Fashion benchmark, from Fashion-MNIST to DeepFashion category prediction branch, is summarized inFIG.11, and the online streaming accuracy is plotted inFIG.10. The offline Source-Only merely achieves 23.1% accuracy, only 6.5% gain on the basis of the probability of guessing, which indicates the benchmark is challenging. The sharp drop of performance from Source-Only online accuracy to one-pass accuracy (−6.8%) indicates the large domain gap, and how easy the model is dominated by the source domain supervision. Similar observation is made on WILDS-Camelyon17 Source-Only results (−11.6% from online to one-pass), this usually happens when the source domain is less challenging than the target domain, and the distribution of the two domains are far from each other. Faced with this challenging benchmark, CRODOBO improves the online performance to a remarkable 49.1%, outperforming the best result in the offline setting.

FIG.12shows example qualitative results of a randomly selected target query (size24). The proposed cross-domain adaptation framework100(CRODOBO) is compared with two essential baselines Source-Only and CDAN. The bootstrapped source samples (top two rows under each benchmark), target samples (third row under each benchmark), and the prediction result of each target sample are illustrated.

This description and the accompanying drawings that illustrate inventive aspects, embodiments, implementations, or applications should not be taken as limiting. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the spirit and scope of this description and the claims. In some instances, well-known circuits, structures, or techniques have not been shown or described in detail in order not to obscure the embodiments of this disclosure. Like numbers in two or more figures represent the same or similar elements.