Domain adaptation

An apparatus, method and computer program is described comprising: initialising weights of a target encoder based on a source encoder; initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; applying some of a target data set to the target encoder to generate target encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function output; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; and synchronising weights of the target discriminator and the source discriminator.

FIELD

The present specification relates to domain adaptation.

BACKGROUND

Encoders may be trained using labelled training data (for example using the principles of machine learning). If data applied to such an encoder during a training phase has similar properties to data applied during a deployment phase, then high performing encoders can be provided. This is not always the case in real-world systems. There remains a need for further developments in this field.

SUMMARY

In a first aspect, this specification provides an apparatus comprising: means for initialising weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; means for initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; means for applying some of a target data set (e.g. comprising unlabeled or partially labelled data) to the target encoder to generate target encoder outputs; means for applying the target encoder outputs to the target discriminator to generate a first local loss function output; means for training the target encoder to seek to increase the first local loss function output; means for training the target discriminator to seek to decrease the first local loss function output; and means for synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator. The training of the source discriminator may be in response to the application of source data to the source encoder to generate source encoder outputs, which are applied to the source discriminator to generate a second local loss function, which is used to train the source discriminator (e.g. to seek to decrease the second local loss function). The first algorithm may be trained based on labelled data. The target encoder may seek to implement a variant of this first algorithm.

The said source encoder may be fixed (e.g. not updated during the training process described herein).

In some example embodiments, the means for synchronising said weights of the target discriminator and the source discriminator synchronises said weights following a plurality of instances (e.g. a pre-defined number of instances) of training of said target discriminator and said corresponding training of said source discriminator.

The means for training the target encoder may comprise means for updating said weights (e.g. on each iteration/each application of target data). The means for training the target discriminator may comprise means for updating said weights of said target discriminator.

In some example embodiments, the means for synchronising weights of the target discriminator and the source discriminator comprises means for averaging the weights (e.g. using a moving average) of the target and source discriminators. Alternatively, or in addition, the means for synchronising weights of the target discriminator and the source discriminator may comprise means for averaging (e.g. using a moving average) changes to the weights of the target and source discriminators.

The means for training the target encoder comprises means for updating said weights (e.g. on each iteration/each application of target data) of said target encoder. Moreover, the means for training the target discriminator may comprises means for accumulating desired changes to said weights of said target discriminator. The means for synchronising weights of the target discriminator and the source discriminator may comprise means for averaging accumulated desired changes to the weights of the target and source discriminators (e.g. using a moving average).

In some example embodiments, the means for initialising weights of the target discriminator initialises the weights of the target discriminator randomly, pseudo-randomly or arbitrarily. The source discriminator may be similarly initialised.

The means for initialising weights of the target encoder may set the weights of the target encoder to be identical to corresponding weights of the source encoder.

In some example embodiments, the target encoder is implemented using a neural network, wherein said weights of said target encoder are weights of said neural network. Further, the means for initialising said target encoder may comprise means for receiving weights of a source encoder neural network at said target encoder.

In some example embodiments, the target discriminator is implemented using a neural network, wherein said weights of said target discriminator are weights of said neural network. The source discriminator may also be implemented using a numeral network. Initialising the target discriminator may comprise providing weights of the source discriminator to the target discriminator (or vice-versa).

Some example embodiments further comprise means for selecting an initial instance of the source encoder from a plurality of candidate encoders. The means for selecting the initial instance of the source encoder may select an optimum source encoder from the plurality of candidate encoders according to some metric. The said metric may be related to a divergence (e.g. a computing domain divergence) between domains. Thus, the source encoder may be selected based on the candidate encoder that shows the smallest divergence between domains, according to the chosen metric (i.e. the smallest difference between the domain of the candidate and the domain of the source).

The said means may comprise: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program configured, with the at least one processor, to cause the performance of the apparatus.

In a second aspect, this specification provides a system comprising: means for initialising weights of a target encoder based on a source encoder, wherein the source encoder implemented a first algorithm; means for initialising weights of a target discriminator, associated with the target encoder, and weights of a source discriminator, associated with the source encoder, such that the target and source discriminators match; means for applying some of a target data set to the target encoder to generate target encoder outputs; means for applying some of a source data set to the source encoder to generate source encoder outputs; means for applying the target encoder outputs to the target discriminator to generate a first local loss function; means for applying the source encoder outputs to the source discriminator to generate a second local loss function; means for training the target encoder to seek to increase the first local loss function output; means for training the target discriminator to seek to decrease the first local loss function output; means for training the source discriminator to seek to decrease the second local loss function output; and means for synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and said source discriminator. The said source encoder may be fixed (e.g. not updated during the training process described herein).

The system of the second aspect may further comprise one or more of the features described above with reference to the apparatus of the first aspect.

In a third aspect, this specification describes a method comprising: initialising weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder;

applying some of a target data set to the target encoder to generate target encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function output; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; and synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator.

The training of the source discriminator may be in response to the application of source data to the source encoder to generate source encoder outputs, which are applied to the source discriminator to generate a second local loss function, which is used to train the source discriminator (e.g. to seek to decrease the second local loss function). The first algorithm may be trained based on labelled data.

The target encoder may seek to implement a variant of this first algorithm.

The said source encoder may be fixed (e.g. not updated during the training process described herein).

In some example embodiments, synchronising said weights of the target discriminator and the source discriminator synchronises said weights following a plurality of instances (e.g. a pre-defined number of instances) of training of said target discriminator and said corresponding training of said source discriminator.

Training the target encoder may comprise updating said weights (e.g. on each iteration/each application of target data). Training the target discriminator may comprise updating said weights of said target discriminator.

In some example embodiments, synchronising weights of the target discriminator and the source discriminator comprises averaging the weights (e.g. using a moving average) of the target and source discriminators. Alternatively, or in addition, synchronising weights of the target discriminator and the source discriminator may comprise averaging (e.g. using a moving average) changes to the weights of the target and source discriminators.

Training the target encoder may comprise updating said weights (e.g. on each iteration/each application of target data) of said target encoder. Moreover, training the target discriminator may comprises accumulating desired changes to said weights of said target discriminator. Synchronising weights of the target discriminator and the source discriminator may comprise averaging accumulated desired changes to the weights of the target and source discriminators (e.g. using a moving average).

Some example embodiments further comprise selecting an initial instance of the source encoder from a plurality of candidate encoders. Selecting the initial instance of the source encoder may select an optimum source encoder from the plurality of candidate encoders according to some metric.

In a fourth aspect, this specification provides a method comprising: initialising weights of a target encoder based on a source encoder, wherein the source encoder implemented a first algorithm; initialising weights of a target discriminator, associated with the target encoder, and weights of a source discriminator, associated with the source encoder, such that the target and source discriminators match; applying some of a target data set to the target encoder to generate target encoder outputs; applying some of a source data set to the source encoder to generate source encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function; applying the source encoder outputs to the source discriminator to generate a second local loss function; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; training the source discriminator to seek to decrease the second local loss function output; and synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and said source discriminator. The said source encoder may be fixed (e.g. not updated during the training process described herein).

The method of the fourth aspect may further comprise one or more of the features described above with reference to the method of the third aspect.

In a fifth aspect, this specification describes an apparatus configured to perform any method as described with reference to the third or fourth aspects.

In a sixth aspect, this specification describes computer-readable instructions which, when executed by computing apparatus, cause the computing apparatus to perform any method as described with reference to the third or fourth aspects.

In a seventh aspect, this specification describes a computer program comprising instructions for causing an apparatus to perform at least the following: initialising weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; applying some of a target data set to the target encoder to generate target encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function output; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; and synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator.

In an eighth aspect, this specification describes a computer-readable medium (such as a non-transitory computer-readable medium) comprising program instructions stored thereon for performing at least the following: initialising weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; applying some of a target data set to the target encoder to generate target encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function output; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; and synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator.

In a ninth aspect, this specification describes an apparatus comprising: at least one processor; and at least one memory including computer program code which, when executed by the at least one processor, causes the apparatus to: initialise weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; initialise weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; apply some of a target data set to the target encoder to generate target encoder outputs; apply the target encoder outputs to the target discriminator to generate a first local loss function output; train the target encoder to seek to increase the first local loss function output; train the target discriminator to seek to decrease the first local loss function output; and synchronise weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator.

In an tenth aspect, this specification describes an apparatus comprising: a first initialisation module for initialising weights of a target encoder based on a source encoder, wherein the source encoder implements a first algorithm; a second initialisation module for initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; a first input for applying some of a target data set (e.g. comprising unlabeled or partially labelled data) to the target encoder to generate target encoder outputs, wherein the target discriminator generates a first local loss function output based on the generated target encoder outputs; a first training module (for example using machine learning principles) for training the target encoder to seek to increase the first local loss function output; a second training module (for example using machine learning principles) for training the target discriminator to seek to decrease the first local loss function output; and a control module for synchronising weights of the target discriminator and the source discriminator following said training of said target discriminator and corresponding training of said source discriminator.

DETAILED DESCRIPTION

In the description and drawings, like reference numerals refer to like elements throughout.

FIG.1Ais a block diagram of an example system, indicated generally by the reference numeral10A. The system10A comprises an encoder12A having an input that is receiving labelled data. The labelled data is used for training the encoder12A using machine-learning principles, as discussed further below.

FIG.1Bis a block diagram of an example system, indicated generally by the reference numeral10B. The system10B comprises an encoder12B that is a trained version of the encoder12A (e.g. trained using the labelled data ofFIG.1A). The encoder12B receives an input and generates an output based on the trained function of the encoder.

Machine Learning (ML) algorithms, as data-driven computational methods, typically attempt to fit a complicated function over a labelled dataset (e.g. a set of training data), with the expectation that comparable performance will be achieved when an unseen dataset (e.g. testing data or operational data) is applied to the trained algorithm. Such training algorithms may be referred to as supervised learning algorithms in which a labelled training set is used to learn a mapping between input data and class labels.

In both theory and practice, machine learning and supervised learning methodologies typically assume that the data distributions of training datasets and deployment (e.g. testing) datasets are the same.

Thus, in the example systems10A and10B, it may be assumed that the distribution of the input data of the system10B matches the distribution of the labelled data in the system10A.

Following this assumption, labelled training sets may be provided for each of a plurality of data distributions, even though many of these data distributions may be similar. Example sets of data for which separate labelled training data might be generated include images of the same object from different angles, paintings in different styles, human activity sensors in different body locations, processing of the same language with different accents and so on.

In real-world systems, the assumption that the data distributions of training datasets and deployment (e.g. testing) datasets are the same is not always valid. A number of real-world factors may lead to variability between training and operational (or test) data distributions. These factors could include, for example, variabilities induced by sensor processing pipelines, or by environment factors (e.g., lighting conditions), or by user-related issues (e.g., different people wear their smart devices differently, thereby causing a shift in the distribution of the generated data). This shift in data distribution between training domains and testing/deployment domains is sometimes referred to as “domain shift”.

As discussed further below, “domain adaptation” seeks to address the issue of domain shift. In general, domain adaptation provides two similar (but different) domains, referred to herein as a source domain and a target domain Data instances in the source domain are typically labelled (providing labelled training data for a source model), whereas data instances in the target domain are partially labelled (semi-supervised domain adaption) or not labelled at all (unsupervised domain adaption). The aim of domain adaption is to seek to train a target model by utilizing aspects of the source model.

Thus, instead of training each data distribution (or “domain”) from scratch, domain adaptation seeks to develop a target model by adapting a source model. This can lead to a reduction in labelling efforts and, in some circumstances, to the development of more robust models.

FIG.2is a flow chart showing an algorithm, indicated generally by the reference numeral20, in accordance with an example embodiment. The algorithm20shows an example domain adaptation algorithm.

The algorithm20starts at operation22, where a source encoder is trained. The source encoder may, for example, be trained using labelled training data (as described above with reference toFIG.1A), for example, by any artificial neural network model, such as a convolutional neural network (CNN).

At operation24, a target encoder is initialised. For example, the target encoder may be initialised to be identical to the trained source encoder, although this is not essential to all example embodiments. In some implementations, the entity managing the trained source encoder may transmit a copy of the trained source encoder to the target domain.

At operation26, the initialised target encoder is adapted to the new domain (thereby completing the domain adaptation process). For example adversarial domain adaptation techniques may be used. Example arrangements for adapting the target encoder to the new domain are described in detail below.

FIG.3is a block diagram of a system, indicated generally by the reference numeral30, in accordance with an example embodiment. The system30comprises a source encoder32, a target encoder34, a source discriminator36and a target discriminator38. The source encoder32and the source discriminator36form part of a source node or domain31a. The target encoder34and the target discriminator38form part of a target node or domain31b. A first communications link33amay be provided between the source encoder32and the target encoder34. A second communications link33bmay be provided between the source discriminator36and the target discriminator38.

Assume that the source dataset XS(or part thereof) is provided to the source encoder32and a target dataset XT(or part thereof) is provided to the target encoder34. These datasets may reside on different nodes (such as machines, apparatuses, devices, servers, and the like) and do not need to be shared with each other, thereby addressing challenges associated with user privacy. In this way, the system30can provide an adversarial domain adaptation system in which the source domain (comprising the source encoder32and the source discriminator36) may be separated, physically or logically, from the target domain (comprising the target encoder34and the target discriminator36), for example at a source node31aand a target node31brespectively. Thus, the system30enables a distributed system to be provided.

FIG.4is a flow chart showing an algorithm, indicated generally by the reference numeral40, in accordance with an example embodiment. As discussed below, the algorithm40may be used in the training of the target encoder34.

The algorithm40starts at operation41, where the elements of the system30are initialised. A source domain starts with a pre-trained source encoder32having a function ESand randomly initialized source discriminator36having a function DS; a target domain starts with the target encoder34having a function ETand target discriminator38having a function DT.

The target encoder34and the target discriminator38are initialized (at operation41) by the source node. Thus, for example, the target encoder34may be initialised to be identical to the pre-trained source encoder32and the target discriminator38may be initialised to be identical to the initialised (e.g. both randomly initialised) source discriminator36. In some implementations, the target node31bmay request the initialization and/or information of the target encoder34and the target discriminator38from the source node31a(for example using one or more of the communications links33aand33b).

At operation42of the algorithm40, data is applied to the source encoder32and the target encoder34. More specifically, the source encoder32samples a batch of the source dataset XSto obtain feature representations RSand the target encoder34samples a batch of the target dataset XTto obtain feature representations RT. The feature representations RSare provided to the source discriminator36, which source discriminator may be a neural network having weights WS(d)Similarly, the feature representations RTare provided to the target discriminator38, which target discriminator may be a neural network having weights WT(d). (WS(d)and WT(d)may represent the weights of neural networks corresponding to DSand DT).

Based on a pre-defined loss function (e.g. a cross-entropy loss), the gradients of the loss with respect to the parameters of the neural networks can be determined. Thus, the gradients GS(d), GT(d)and GT(e)of the source discriminator36, the target discriminator38and the target encoder34respectively can be determined.

At operation43, the relevant weights WS(d)and WT(d)are updated. As described further below, in accordance with adversarial training principles, the source and target discriminators36and38are trained to minimise the respective loss functions La of those discriminators and the target encoder34is trained to maximise the loss function La. The pre-trained source encoder32is not further trained. DS, DTand ETwill be updated in the adaptation process while ESis pre-trained and fixed.

At operation44, the discriminators36and38are synchronised, as discussed further below.

With the relevant weights updated and synchronised, the operations42to44are repeated. When the algorithm is complete (e.g. after a defined number of iterations or data batches, or when a defined performance level is achieved), a final value (e.g. an optimal value) of ETis generated, which can be used as the encoder model of the target domain.

FIGS.5to7are flow charts showing three algorithms that provide example implementations of the algorithm40described above with reference toFIG.4.

FIG.5is a flow chart showing an algorithm, indicated generally by the reference numeral50, in accordance with an example embodiment.

The algorithm50starts at operation51, where as described above, the elements of the system30are initialised. Thus, in common with the operation41of the algorithm40, the target encoder34is initialised to be identical to the pre-trained source encoder32and the target discriminator38is initialised to be identical to the initialised (e.g. randomly initialised) source discriminator36.

At operation52of the algorithm50, data is applied to the source encoder32and the target encoder34. In common with operation42of the algorithm40, the source encoder32samples a batch of the source dataset XSto obtain feature representations RSand the target encoder34sample a batch of the target dataset XTto obtain feature representations RT.

At operation53, the relevant weights are updated. As described above with reference to the algorithm40, the source and target discriminators36and38are trained to minimise the respective loss functions La of those discriminators and the target encoder34is trained to maximise the loss function La. The source discriminator function DSand the target discriminator function DT(which are separately updated) may be updated based on an average (e.g. a moving average) of recent weights at the respective discriminator.

At operation54, it is determined whether the current iteration of the algorithm50is a synchronisation step. If so, the algorithm50moves to operation55; otherwise, the algorithm50moves to operation57. As discussed further below, since the source and target discriminators may be provided in separate domains, data transfer costs can be reduced by enabling the discriminators to be updated locally for a number of iterations, before being synchronised.

The triggering of the synchornisation operation may be a user-defined parameter (for example, a user defined number of iterations of the operations52and53may be implemented before the algorithm moves from operation54to operation55). A number of considerations are relevant when considering such parameters, including data transfer costs, as noted above.

At operation55, the weights of the discriminator DSand DT(i.e. the weights WS(d)and WT(d)) are averaged so that an average change in those weights over the iterations since the last synchronisation step is determined. Then, at operation56, the source discriminator and the target discriminator are both updated based on the averaging of the weights in the operation55. In this way, the discriminator weights WS(d)and WT(d)(which may have diverged following a number of iterations of the algorithm50without synchronisation) are equalised.

The synchronisation of the discriminators may be symmetric such that data exchange could, in principle, happen in either direction, wherein the “receiving domain” performs the averaging operation55and sends the averaged values back to the sending domain to enable the discriminators to be updated in operation56.

Following the operation56, the algorithm50moves to operation57.

At operation57, a determination is made regarding whether the algorithm50is complete (e.g. after a defined number of batch iterations, or when a defined performance level is achieved). If so, the algorithm50terminates at operation58; otherwise, the algorithm returns to operation52and further data is applied such that the source and target discriminators and the target encoder can be further updated. As noted above, when the algorithm terminates (at operation58), a final value (e.g. an optimal value) of ETis generated, which can be used as the encoder model of the target domain.

FIG.6is a flow chart showing an algorithm, indicated generally by the reference numeral60, in accordance with an example embodiment. The algorithm60is a variant of the algorithm50described above.

The algorithm60starts at operation61, where, as described above, the elements of the system30are initialised (e.g. by initialising the target encoder34to be identical to the pre-trained source encoder32and initialising the target discriminator38to be identical to the initialised source discriminator36).

At operation62of the algorithm60, data is applied to the source encoder32and the target encoder34. In common with operations42and52described above, the source encoder32samples a batch of the source dataset XSto obtain feature representations RSand the target encoder34samples a batch of the target dataset XTto obtain feature representations RT.

At operation63, the relevant weights are updated. As described above, the source and target discriminators36and38are trained to minimise the respective loss functions La of those discriminators and the target encoder34is trained to maximise the loss function La. The source discriminator function DSand the target discriminator function DT(which are separately updated) may be updated based on an average (e.g. a moving average) of recent weights at the respective discriminator.

At operation64, it is determined whether the current iteration of the algorithm60is a synchronisation step (e.g. based on user-defined parameters as described above). If so, the algorithm60moves to operation65; otherwise, the algorithm60moves to operation67. Since the source and target discriminators may be provided in separate domains, data transfer costs can be reduced by enabling the discriminators to be updated locally for a number of iterations, before being synchronised.

At operation65, the gradients of the discriminator DSand DTare averaged so that an average change over the iterations since the last synchronisation step is determined. The operation65therefore differs from the operation55in that gradients are considered (rather than the weights themselves). Moreover, the operation65may consider a moving average of gradients. Then, at operation66, the source discriminator and the target discriminator are both updated based on the gradient moving average in the operation65. In this way, the discriminator weights WS(d)and WT(d)(which may have diverged following a number of iterations of the algorithm50without synchronisation) are equalised. Following the operation66, the algorithm60moves to operation67.

The moving average function used in the operations65and66in the synchronisation of the source and target discriminators acts as a filter, such that changes to the source and/or the target discriminator during synchronisation are generally less severe. This can, in some circumstances, improve performance. For example, the moving average functions may prevent the source and target discriminators from diverging too much from one another.

At operation67, a determination is made regarding whether the algorithm60is complete (e.g. after a defined number of batch iterations, or when a defined performance level is achieved). If so, the algorithm60terminates at operation68; otherwise, the algorithm returns to operation62and further data is applied such that the source and target discriminators and the target encoder can be further updated. As noted above, when the algorithm terminates (at operation68), a final value (e.g. an optimal value) of ETis generated, which can be used as the encoder model of the target domain.

FIG.7is a flow chart showing an algorithm, indicated generally by the reference numeral70, in accordance with an example embodiment. The algorithm70is a variant of the algorithms50and60described above.

The algorithm70starts at operation71, where, as described above, the elements of the system30are initialised (e.g. by initialising the target encoder34to be identical to the pre-trained source encoder32and initialising the target discriminator38to be identical to the initialised source discriminator36).

At operation72of the algorithm70, data is applied to the source encoder32and the target encoder34. In common with operations42,52and62described above, the source encoder32samples a batch of the source dataset XSto obtain feature representations RSand the target encoder34sample a batch of the target dataset XTto obtain feature representations RT.

At operation73, the weights of the target encoder34are updated. The weights of the source and target discriminators36and38are not updated at this stage. Rather, at operation74, the gradients generated locally for updating the discriminators are accumulated for use later in the algorithm.

At operation75, it is determined whether the current iteration of the algorithm70is a synchronisation step (e.g. based on user-defined parameters as described above). If so, the algorithm70moves to operation76; otherwise, the algorithm70moves to operation78. Since the source and target discriminators may be provided in separate domains, data transfer costs can be reduced by enabling the discriminator gradients to be accumulated locally for a number of iterations, before synchronisation occurs.

At operation76, the gradients accumulated in multiple iterations of the operation74are averaged so that an average change of the discriminator functions DSand DTover the iterations since the last synchronisation step is determined. Then, at operation77, the source discriminator and the target discriminator are both updated based on the averaging of the accumulated gradients in the operation77. The discriminator weights WS(d)and WT(d)at the last synchronisation operation are updated based on the average of the accumulated gradients and are thereby updated and equalised. Following the operation77, the algorithm70moves to operation78.

The operations76and77differ from similar operations described above in the use of accumulated gradients in the synchronisation of the source and target discriminators. This process keeps the source and target discriminators synchronised, as they are updated with the same average gradients, that are accumulated over a number of steps.

At operation78, a determination is made regarding whether the algorithm70is complete (e.g. after a defined number of batch iterations, or when a defined performance level is achieved). If so, the algorithm70terminates at operation79; otherwise, the algorithm returns to operation72and further data is applied such that the source and target discriminators and the target encoder can be further updated. As noted above, when the algorithm terminates (at operation79), a final value (e.g. an optimal value) of ETis generated, which can be used as the encoder model of the target domain.

Three possible implementations of the algorithm40have been described above (i.e. the algorithms50,60and70. Other variants of the algorithms50,60and70are possible, such that other implementations of the algorithm40are possible. For example, other empirical measures of gradient average or model average may be developed. For instance, one option could be to send the outputs of the source encoder to the target domain (or vice versa) and then update the discriminators with both the source encoder and target encoder results. This would avoid the need for discriminator synchronisation since information would be exchanged at the level of the encoders.

In general, the information exchange between the nodes can happen at discriminator-level (as in the embodiments described in detail above), or at encoder level or even at raw data level (where, for example, all raw data is sent from source to target).

As discussed in detail above, DSand DTare updated to minimize La, whereas ETis updated to maximize La. Since the gradients of ETare only related to the target data, there is no data exchange required with the source domain. However, for DSand DT, each can only get the data from one domain, hence gradients or weights are shared across domains such that the discriminators are trained properly with data from both the source and target domains. By this means, the amount of communication from the whole dataset can be reduced to only the size of discriminator, which may be only a few neural network layers.

Moreover, by adopting this approach, raw data exchange between nodes/domains may be avoided (hence, enhancing user privacy and minimizing the storage needs on each node), and the total communication cost is lowered.

The algorithms described above can be used in a wide range of scenarios, such as image recognition.

For example, in the example system ofFIG.3consider the following scenario in which developers of the source domain31ain a first country are training an artificial intelligence (AI) model which analyzes a sonogram image from a sonography machine and locates the position of a foetus in the image. Assume that the AI model is trained over hundreds of sonograms obtained from hospitals in the first country and has a 90% accuracy of locating the foetus in an image. Now the source domain comprises, a first country images of foetus (i.e. a source domain1) and/or a sonogram device A (i.e. a source domain2).

Now, consider the situation in which the developers would like to deploy this AI model in the target domain31bto hospitals in a second country. Due to differences in the precision of sonogram machines and the methodology applied by the sonographers, it is possible that the sonograms in the target domain31bin the second country may look different from those recorded in the source domain31ain the first country. Now the target domain comprises, a second country specific images of foetus (i.e. a target domain1) and/or a sonogram device B (i.e. a target domain2). This is an example of domain shift, which can lead to reduction in accuracy of the pre-trained AI model. To counter this, domain adaptation can be used to modify the first model (e.g. source encoder32) to develop a second model (e.g. target encoder34) that is adapted for use on the data obtained in the second country.

Using the system30, sonograms for the first country can reside in the source domain31a(and may be kept in the first country) and sonograms for the second country can reside in the target domain31b(and may be kept in the second country). Domain adaptation can be performed by exchanging the gradients and/or weights of the source and target discriminators36and38, thereby addressing privacy issues (since sonogram data does not need to be shared) and saving communication costs (since the data transfer required is small).

The algorithms described above can be used in many other scenarios, such as natural language processing (NLP) an/or speech recognition. Now consider a second scenario in which developers of the source domain31aare training a model, e.g. a source encoder32, that analyses human speech to infer the emotion of the speaker. Assume that this model was trained over thousands of speech samples obtained from smartphone users in a single country, e.g. a source country, and has an 85% accuracy of emotion detection.

Now when this model is shipped to other geographical regions, the model may not perform well, due, for example to variations in accents across population groups. As such, the model may need to be adapted for each population group in each target county. The principles described herein can be used to adapt the trained model for the one or more new domains.

The domain adaptation techniques described above typically work well when there are only two domains in the system. However, there remains a need for scaling in the event that the number of available domains increases.

By way of example, consider the example of the model, e.g. the source encoder32, described above for recognizing the emotion of a speaker from a given speech segment. Assume that the model is developed by a machine-learning developer in the USA. It is likely that the developer would have collected most of their training data from English speakers in the USA and obtained a good performance for the speech task. Now when this model is deployed to other geographical regions (e.g., China, Germany, Switzerland, India etc.), the variability in user accents would introduce domain shift which will degrade the model accuracy. As such, the developer may seek to adapt the model to these target user populations (e.g. using an unsupervised domain adaption technique).

One approach would be for each target domain31bto be adapted from the labelled source domain31a(i.e., US speakers). However, it may not always be optimal to adapt from the original source domain, especially if another domain is present in the system that is closer to the target domain. For example, imagine that the original model (for US speakers, e.g. the source encoder32) has already been adapted for German speaker (e.g. the target encoder34). Imagine now that we want to adapt the model for Swiss speakers (i.e. a new target encoder). It may be better to adapt the model for German speakers (rather than adapting the “original” model for US speakers), since there is likely to be a greater similarity between Swiss and German speakers than between Swiss and US speakers. In this case, although the US speakers represent the original source domain, adapting from them could be sub-optimal.

Moreover, adapting each target domain from the original source domain may put significant computational and communication loads on the source machine (especially, if, for example, we expect hundreds of new target domains to appear in the system).

FIG.8is a flow chart showing an algorithm, indicated generally by the reference numeral80, in accordance with an example embodiment. The algorithm80enables a target encoder to be initialised based on a source encoder, and then trained.

The algorithm80starts at operation82, where one or more candidate source encoders are identified. Then, at operation84, a source encoder is selected from the plurality of candidate encoders. The target encoder may then be initialised based on the selected encoder and an encoder adaptation algorithm used to adapt the selected encoder to the target domain (for example, using one of the algorithms described above).

Thus, in the context of the example described above for recognizing the emotion of a speaker from a given speech segment, the plurality of candidate source encoders may include an encoder of the original algorithm (e.g. trained based on US speakers) and other source encoders (e.g. based on Chinese, German, Swiss and Indian speakers) at least some (but not necessarily all) of which may have been modified from the original US algorithm.

FIG.9is a block diagram of a system, indicated generally by the reference numeral90, in accordance with an example embodiment.

The system90comprises one or more candidate source encoders, including a first source encoder91, a second source encoder92, a third source encoder93and an nth source encoder94. The first source encoder91may, for example, be an initial source encoder (such as the US-trained algorithm in the example described above) with one or more of the other source encoders having been modified from that initial source encoder. The source encoders91to94represent candidate encoders, as discussed above with reference to the operation82. Clearly, the system90may include more or fewer source encoders.

The system90further comprises a control module96receiving a domain input and information from each of the sources encoders91to94(such as features extracted from the relevant source encoder or the output of the relevant source encoder when input data from the source domain is passed to the said encoder). The control module determines which of the source encoders is closest to the domain as indicated by the domain input and provides that information to a target encoder98(thereby implementing operation84of the algorithm80). As discussed further below, the target encoder98may be initialised based on the selected source encoder and then adapted based on the new domain.

The control module96may select an optimum source encoder from the one or more candidate encoders according to some metric. As discussed further below, the metric may determine the smallest difference between domains (e.g. a computing domain divergence).

Thus, a collaborative multi-step domain adaption method may be provided, wherein an optimal collaborator is chosen for each new target domain. This approach of performing domain adaptation may result in higher accuracies in the target domain.

At a general level, the system90may be used as follows. Assume that we have a labelled source domain S in the system, and we have a classifier (MS) trained using supervised learning algorithms for S. Further, say we have 4 unlabeled target domains in the system (T1, T2, T3, T4), and the goal is to train a classifier for each of these target domains. As we do not have labelled data from the target domains, we will need to perform domain adaptation.

Let us say that T1 is introduced in the system. Currently, we only have S already in the system (e.g. the initial source encoder91) along with its pre-trained inference model MS. As such, we perform a domain adaptation step between S and T1, which results in a model MT1for the domain T1 (e.g. according to the one or more prosesses50,60or70inFIG.5,6or7).

Now assume that T2 is introduced in the system, which already has S, T1 and their associated model MSand MT1. Although T1 itself is an adapted domain, we can now treat it as a source domain and consider it equivalent to our original labelled source domain S As such, for T2, it now has option to adapt from either S or T1.

In an example embodiment, the control module96uses a selection technique based on computing domain divergence, i.e. how far are two domains from each other. In particular, the control module96may determine a pairwise divergence (T2, S) or (T2, T1) between the target domain (i e T2) and the candidate domains (S and T1), and identify the pair with the smallest divergence.

Computing domain divergences only from unlabeled data is not trivial. We propose using the loss of a binary classifier trained to separate the two domains as a proxy for domain divergence. More details about this technique are provided below.

Assume in the example above that T1 is found as the closest collaborator for T2. Now we perform a domain adaptation step between T1 and T2, wherein we adapt the weights of MT1to output a model for T2, namely MT2. Now if T3 is introduced in the system, the steps above can be repeated in order to select the appropriate candidate encoder for the new domain and eventually train a model MT3.

We now discuss several ways of implementing the collaborator selection technique and domain adaptation algorithm described above.

The candidate selection technique may be based on computing divergences or distances between different domains. In an example implementation, Wasserstein Distance is used as a metric to compute the distances between two distributions. Wasserstein Distance, also known as the earth-mover distance, works on the intuition that if each domain is viewed as a unit amount of “dirt” piled on a given metric space M, the Wasserstein Distance is the minimum “cost” of turning one pile into the other, which is assumed to be the amount of dirt that needs to be moved times the mean distance it has to be moved.

To compute the Wasserstein Distance, a binary classifier or a domain discriminator may be trained, wherein all the samples from source domain are labelled as 0 and all samples from target domain are labelled as 1. The loss of this binary classifier is used as a proxy for Wasserstein Distance in our implementation—the higher the loss, the higher the Wasserstein Distance, and the further the two domains are from each other. In this way, we can train pairwise discriminators between a target domain and each of a number of candidate collaborators, and find the collaborator with the least Wasserstein Distance with the target domain. This candidate is used as an optimal collaborator for adaptation.

Note that in other example embodiments, other distance metrics (such as MMD (Maximum Mean Discrepancy), KL Divergence, or Logistic Loss) may be used. The skilled person will be aware of alternative metrics, not discussed herein, that could be used.

The candidate selection algorithm described herein may be used independently of the underlying domain adaptation technique being used. In one example embodiment, Adversarial Discriminative Domain Adaptation (ADDA) is used for performing domain adaptation, but this is not essential to all embodiments.

ADDA uses a pre-trained model in the source domain (MS) and a set of unlabeled datasets XSand XTfrom the source and target domains. ADDA adopts an adversarial training approach which involves training a discriminator D and a feature encoder MTusing adversarial training. The goal of the discriminator is to distinguish the two domains while the goal of the encoder is to fool the discriminator by generating features that are indistinguishable across domains. Both the encoder and discriminator participate in this game, trying to fool each other, and in the process both get better at their respective tasks. More importantly, our encoder MTmay learn to map target domain data into the same feature space as the source encoder, thereby enhancing the inference accuracy in the target domain.

By way of example, the principles described above in the example embodiments may be used and combined for human activity recognition models (e.g. to detect walking, running etc.). Assume that a model is trained on accelerometer and gyroscope data collected from a smartphone placed in a user's thigh pocket. Now when this model is deployed, the model can be used with wearable devices placed on body parts other than the thigh. For instance, the model could be adapted and used in an application on a wrist watch, or in an ear-worn device.

The application of the thigh-trained model to these new body positions is an example of a domain shift.

In accordance with the algorithm80, when the user uses the model with a new body position, we will first find the optimal collaborator (e.g. the most similar body position for which we already have a trained model) and then perform a domain adaptation step to adapt the existing model to this new body position. Whether the optimal collaborator is the original thigh-trained model, or a different model that was itself adapted from the thigh-trained model, is immaterial. Effectively, with adaptation on just unlabeled data, the original activity recognition model can now work on new body positions, and achieve the best possible accuracy without requiring labelled data. In a similar manner, the principles described above may be used and combined for health related recognition models, such as heart rate collection and detection models between different heart rate sensors, and/or heart rate sensors placed on different parts of human or animal body.

Additionally, the principles described above in the example embodiments may be used and combined for models adapted and implemented in one or more industrial processes between different apparatuses and/or sensors. In general, the algorithms described above can be used and combined in the domain shift between any domains, such as sensors, hardware (HW) components, such as microphones, antennas, transmitters, receivers, modems and/or software (SW) applications, such as codecs, or any combination thereof.

Many of the elements described above may be implemented using neural network technology. By way of example,FIG.10is a block diagram of a neural network system, indicated generally by the reference numeral100, in accordance with an example embodiment. The example neural network system100is used, by way of example, to implement the target encoder34described above Similar neural network systems may be used to implement other modules described here (such as the source encoder32, the source discriminator36and the target discriminator38).

The neural network of the system100comprises a plurality of nodes and a plurality of connections between those nodes. The neural network is trained by modifying the nodes, including modifying connections between the nodes and the weighting applied to such connections.

For completeness,FIG.11is an example schematic diagram of components of one or more of the modules for implementing the algorithms in the target and/or the source domains described above, which hereafter are referred to generically as processing systems300. A processing system300may have a processor302, a memory304coupled to the processor and comprised of a RAM314and

ROM312, and, optionally, user inputs310and a display318. The processing system300may comprise one or more network interfaces308for connection to a network, e.g. a modem which may be wired or wireless.

The processor302is connected to each of the other components in order to control operation thereof.

The memory304may comprise a non-volatile memory, a hard disk drive (HDD) or a solid state drive (SSD). The ROM312of the memory304stores, amongst other things, an operating system315and may store software applications316. The RAM314of the memory304is used by the processor302for the temporary storage of data. The operating system315may contain code which, when executed by the processor, implements aspects of the algorithms20,40,50,60,70and80.

The processor302may take any suitable form. For instance, it may be a microcontroller, plural microcontrollers, a processor, or plural processors. Processor302may comprise processor circuitry.

The processing system300may be a standalone computer, a server, a console, an apparatus, a user device, a mobile communication device, a vehicle, an IoT device, a sensor, a software application, a communication network, or any combination thereof.

In some example embodiments, the processing system300may also be associated with external software applications. These may be applications stored on a remote server device and may run partly or exclusively on the remote server device. These applications may be termed cloud-hosted applications. The processing system300may be in communication with the remote server device in order to utilize the software application stored there.

FIG.12AandFIG.12Bshow tangible media, respectively a removable memory unit365and a compact disc (CD)368, storing computer-readable code which when run by a computer may perform methods according to example embodiments described above. The removable memory unit365may be a memory stick, e.g. a USB memory stick, having internal memory366storing the computer-readable code. The memory366may be accessed by a computer system via a connector367. The CD368may be a CD-ROM or a DVD or similar. Other forms of tangible storage media may be used.

Reference to, where relevant, “computer-readable storage medium”, “computer program product”, “tangibly embodied computer program” etc., or a “processor” or “processing circuitry” etc. should be understood to encompass not only computers having differing architectures such as single/multi-processor architectures and sequencers/parallel architectures, but also specialised circuits such as field programmable gate arrays FPGA, application specify circuits ASIC, signal processing devices and other devices. References to computer program, instructions, code etc. should be understood to express software for a programmable processor firmware such as the programmable content of a hardware device as instructions for a processor or configured or configuration settings for a fixed function device, gate array, programmable logic device, etc.

If desired, the different functions discussed herein may be performed in a different order and/or concurrently with each other. Furthermore, if desired, one or more of the above-described functions may be optional or may be combined. Similarly, it will also be appreciated that the flow diagrams ofFIGS.2and4to8are examples only and that various operations depicted therein may be omitted, reordered and/or combined.

It will be appreciated that the above described example embodiments are purely illustrative and are not limiting on the scope of the invention. Other variations and modifications will be apparent to persons skilled in the art upon reading the present specification.

Moreover, the disclosure of the present application should be understood to include any novel features or any novel combination of features either explicitly or implicitly disclosed herein or any generalization thereof and during the prosecution of the present application or of any application derived therefrom, new claims may be formulated to cover any such features and/or combination of such features.

It is also noted herein that while the above describes various examples, these descriptions should not be viewed in a limiting sense. Rather, there are several variations and modifications which may be made without departing from the scope of the present invention as defined in the appended claims.