DEVICE AND IN PARTICULAR COMPUTER-IMPLEMENTED METHOD FOR DETERMINING A SIMILARITY BETWEEN DATA SETS

A device and a computer-implemented method, for determining a similarity between data sets. A first data set that includes a plurality of first embeddings, and a second data set that includes a plurality of second embeddings, are predefined. A first model is trained on the first data set, and a second model is trained on the second data set. A set of first features of the first model is determined on the second data set, which for each second embedding includes a feature of the first model, and a set of second features of the second model is determined on the second data set, which for each second embedding includes a feature of the second model. A map that optimally maps the set of first features onto the set of second features is determined. The similarity is determined as a function of a distance of the map from a reference.

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 of German Patent Application No. DE 10 2021 202 566.8 filed on Mar. 16, 2021, which is expressly incorporated herein by reference in its entirety.

FIELD

The present invention is directed to a device and an in particular computer-implemented method for determining a similarity between data sets, in particular images.

SUMMARY

In accordance with an example embodiment of the present invention, a method, in particular a computer-implemented method, for determining a similarity of data sets provides that a first data set that includes a plurality of first embeddings is predefined, a second data set that includes a plurality of second embeddings being predefined, a first model being trained on the first data set, a second model being trained on the second data set, a set of first features of the first model being determined on the second data set, which for each second embedding includes a feature of the first model, a set of second features of the second model being determined on the second data set, which for each second embedding includes a feature of the second model, a map being determined that optimally maps the set of first features onto the set of second features, the similarity being determined as a function of a distance of the map from a reference. The method is applicable using models that provide feature representations, regardless of a particular model architecture. A similarity of the data sets may thus be detected significantly better.

The first embeddings of the plurality of first embeddings each preferably represent a digital image from a plurality of first digital images, the second embeddings of the plurality of second embeddings each representing a digital image from a plurality of second digital images. In this way, two data sets that contain digital images and whose contents are particularly similar to one another may be found.

The first embeddings of the plurality of first embeddings each preferably represent a portion of a first corpus, the second embeddings of the plurality of second embeddings each representing a portion of a second corpus. In this way, two corpora whose contents are particularly similar to one another may be found.

In accordance with an example embodiment of the present invention, it may be provided that the first model includes an artificial neural network with an input layer and an output layer, for each second embedding situated at the input layer of the first model, an output of a layer, in particular a last layer prior to the output layer, between the input layer and the output layer being determined that characterizes a feature associated with the second embedding, and/or that the second model includes an artificial neural network with an input layer and an output layer, for each second embedding situated at the input layer of the second model, an output of a layer, in particular a last layer prior to the output layer, between the input layer and the output layer being determined that characterizes a feature associated with the second embedding.

In accordance with an example embodiment of the present invention, it is preferably provided that the artificial neural networks having the same architecture, in particular an architecture of a classifier, are predefined, or that the layers whose output characterizes the features have the same dimensions.

In accordance with an example embodiment of the present invention, it may be provided that for a training, a training data set is determined that includes the first data set or a portion thereof when the similarity of the first data set to the second data set is greater than a similarity of a third data set to the second data set, and that otherwise the training data set is determined as a function of the third data set, in a training the second model being pretrained with data of the training data set and then being trained with data of the second data set. In this way, the second model is pretrained on data from a data set having a particularly great similarity to the second data set.

The in particular best possible data set for the pretraining is preferably selected by selecting the data set having a minimum distance from the second data set.

The map is preferably determined as a function of distances of each first feature from each second feature, in particular with the aid of a Procrustean method that minimizes these distances.

The similarity is preferably determined as a function of a norm of the distance of the map from the reference.

In one aspect of the present invention, it is provided that the second model is trained or becomes trained for a classification of embeddings, at least one embedding of a digital image or of a portion of a corpus being detected or received, and the embedding being classified by the second model.

In accordance with an example embodiment of the present invention, a device for determining a similarity of data sets is designed to carry out the method.

In accordance with an example embodiment of the present invention, a computer program that includes computer-readable instructions is likewise provided, the method running when the computer-readable instructions are executed by a computer.

Further advantageous specific embodiments result from the following description and the figures.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1shows a schematic illustration of portions of a device100for determining a similarity of data sets. This is described below with reference to a first data set101and a second data set102. In the example, the data sets are digital representations, in particular numeric or alphanumeric representations, of images, metadata of images, or portions of corpora. In the example, second data set102is a target data set on which a model for solving a task is to be trained. In the example, first data set101is a candidate for a training data set on which the model is to be pretrained, if the first data set proves to be suitable for this purpose.

Device100is designed to establish a similarity of data sets to second data set102. This is described by way of example for the similarity between first data set101and second data set102.

Device100includes a plurality of models.FIG. 1schematically illustrates a first model and a second model. Device100is designed to determine, using the first model and the second model, a similarity of first data set101to second data set102.

Device100may include a third model via which a similarity of a third data set to second data set102is determined. Device100may include an arbitrary number of further models for other data sets.

In the example, the first model is a first artificial neural network103that includes an input layer104and an output layer105, as well as a layer106situated between input layer104and output layer105.

In the example, the second model is a second artificial neural network107that includes an input layer108and an output layer109, as well as a layer110situated between input layer108and output layer109.

The artificial neural networks may be classifiers. In the example, the artificial neural networks have the same architecture. The architectures do not have to be identical.

Device100includes a computing device111. Computing device111is designed to train the models with the particular data sets. Computing device111is designed, for example, to train the first model with embeddings112from first data set101. Computing device111is designed, for example, to train the second model with embeddings113from second data set102.

Computing device111is designed to extract features114from layer106. Computing device111is designed to extract features115from layer110. In the example, layers106,110whose output characterizes features114,115have the same dimensions. The dimensions do not have to be identical.

Computing device111is designed to select a data set, from the plurality of data sets, that has a greater similarity to second data set102than some other data set or than all other data sets from the plurality of data sets. In the example, for this purpose computing device111is designed to carry out the method described below.

Computing device111is designed, for example, to determine a selected data set116as a function of features114,115that are extracted from layers106,110.

Computing device111is designed, for example, in a training to train the second model initially with selected data set116, and subsequently with second data set102.

In one example, the second model is to be trained for a task with second data set102. In the example, there are only few training data for second data set102. In contrast, in the example there are more training data for first data set101and other data sets from the plurality of data sets.

By use of the method described below, it is determined which of the data sets from the plurality of data sets is closest to second data set102and is suitable for pretraining the second model. The second model is pretrained with the data set thus determined, and then trained with second data set102. In this way, better performance is achieved than is to be expected from training the second model only with second data set102.

This is described using first data set101and second data set102as well as the third data set as an example. The method is correspondingly applicable to the plurality of data sets.

Instead of using one of the mentioned data sets, it is also possible to use only a portion, in particular a randomly selected portion, of the data sets.

The method may be applied for various data sets. The first embeddings112, for example, may each represent one digital image from a plurality of first digital images. The second embeddings113, for example, may each represent one digital image from a plurality of second digital images. These embeddings may each numerically represent pixels of an image, for example the red, green, and blue components of the image.

First embeddings112may each numerically represent a portion of a first corpus, for example a word, a portion of a word, or a portion of a set. Second embeddings113may each numerically represent a portion of a second corpus, for example a word, a portion of a word, or a portion of a set.

In the method, a first data set101that includes a plurality of first embeddings112is predefined in a step202.

In the method, a second data set102that includes a plurality of second embeddings113is predefined in a step204.

First artificial neural network103is trained on first data set101in a step206.

Second artificial neural network107is trained on second data set102in a step208.

In the example, the artificial neural networks are trained for classification. In the example, training is carried out with supervision. In the example, the training data include labels that associate with the individual embeddings one of the classes into which the particular artificial neural network may classify the embedding. Digital images in the training data may be classified, for example, according to an object or subject that represents them. Corpora may be classified, for example, according to names the corpora include.

These steps may be carried out in succession or essentially in parallel with one another with regard to time.

A set of first features114of first artificial neural network103on second data set102is subsequently determined in a step210. In the example, for each embedding113of second data set102a feature114of first artificial neural network103is determined and added to the set of first features114. Feature114is an output of layer106onto which first artificial neural network103maps embedding113at input layer104.

A set of second features115of second artificial neural network107on second data set102is determined in a step212. In the example, for each second embedding113of second data set102a feature115of second artificial neural network107is determined and added to the set of second features115. Steps212may be carried out in succession or essentially in parallel with one another with regard to time. Feature115is an output of layer110onto which second artificial neural network107maps embedding113at input layer108.

A map MP that optimally maps the set of first features114onto the set of second features115is determined in a step214.

In the example, a first feature114from the set of first features114is a vector F1(v) for a particular embedding v. In the example, a second feature115from the set of second features115is a vector F2(v) for particular embedding v. In the example, the embeddings are likewise vectors. In one example, map MP is conditionally defined by a matrix M having the dimensions of the features:

In the example, map MP is determined in such a way that features F1according to the map are very similar to features F2. In the example, this map is determined with the aid of the Procrustean method, in that a matrix M including the pointwise distances of the vectors is minimized by shifting, scaling, and rotating of the features:

Map MP may also be computed in some other way.

The similarity is subsequently determined in a step216as a function of a distance of map MP from a reference.

In the example, the map is compared to a unit matrix I as reference, with the aid of a matrix norm. The distance between the models is determined, for example, from the difference between MM1,M22and unit matrix I. In the example, a great deviation is interpreted as a large distance between the models, and therefore between the data sets with which these models have been trained.

Steps202through216may be carried out for the comparison of a plurality of other data sets to second data set102. In the example, these steps are carried out at least for a third data set.

It is subsequently checked in a step218whether a similarity of first data set101to second data set102is greater than a similarity of the third data set to second data set102. If the similarity of first data set101to second data set102is greater, a step220is carried out. Otherwise, a step222is carried out.

A training data set that includes first data set101or a portion thereof is determined in step220. Step224is subsequently carried out.

A training data set that includes the third data set or a portion thereof is determined in step222. Step224is subsequently carried out.

In a training with data of the training data set, second artificial neural network107is pretrained and then trained with data of second data set102in step224.

In the example, a step226is subsequently carried out.

At least one embedding is detected or predefined, and classified using second artificial neural network107thus trained, in step226.

The embedding is a function of what has been trained for, an embedding of a digital image or a portion of a corpus.