Hi, my name is Massimiliano Patakjola from the University of Edinburgh. In this video, I'm going to introduce our latest work that is set superpice-relationary, reasoning for representation learning. First of all, I would like to define the problem. The idea is to learn from unlabeled data and then transfer this knowledge to other tasks. In set superpice learning, this is done in three steps. First, we define a proxy task over the unlabeled data set. This can be classification on a solid-gate set of classes. And then we train a network called Backbone to solve this proxy task and we learn useful representation on the way. And finally, we transfer this knowledge to those two tasks, like classification, segmentation, or image editor. A previous work had done this in different ways. You can, for instance, define a text task, like solving jigsaw puzzles, predicting image rotation, generating missing batches, and so on. Other way is to define a contrastive loss. And this is done in methods like Mocco's Sinclair that achieved the state of the arts in set superpice learning. In pseudo-labeling, we are going to generate pseudo-labels and then to perform a supervised classification on top of these labels. In infomax method, instead, we are going to maximize the mutual information. And this helps in learning representations. This is done, for instance, by infomax and motivio infomax. Now, our case, we propose to use the relational reasoning. And we define a Sarrogate binary classification task, and we use a relation head to discriminate between two sets of data. We have positive pairs that are generated by randomly sampling one image, generate two augmentations by, for instance, crop the size or random Moritz-Anta flip, and aggregate the pairs. And negative pairs that are generated by randomly sampling two images perform augmentations on those images, and then generate the pair from these images. After we have got the pairs, we apply an aggregation function that is given by applying a function A on top of the pair. This can be some max or concatenation. Then we have a relation head that is used on top of the aggregated pairs. And in our case, this is a multilayer perceptron that generates prediction-wide. If, finally, we have a loss function. The loss function takes, as input, the prediction-wide, and a label T. The label T will be 0 for negative pairs and 1 for positive pairs. We impose a binary cross-center P loss that optional can have a focal factor, or we can also use regression losses like the min-square-terror loss. The idea is that by minimizing this loss, we are going to learn useful representation in the backboard. Here we see the full pipeline. On the left, we have the min-batch B, and we apply K-ommentation on this min-batch, then we are going to encode each image by using the backboard F. This produces the Latin representation in Z. Then we use the aggregation function to generate positive and negative pairs. After this, we use the relation head. We take as input the positive and negative pairs, and then we produce a prediction-wide. We then take the prediction-wide and the target T, and we plug them inside the binary cross-center P loss. This is minimized with respect to the parameter of the relation head and the neural network backbone. Here we show an overview of the experiments. First of all, we have tried the standard linear evaluation procedure. This consists of training on the a labeled version of a dataset, and then fine-tune, just the very last linear layer of the backbone on the labeled version of the same dataset. We have trained this on cifr10, cifr100, tiny matchnet, as the matchnet, using different backboards. Then we are also done a domain transfer task, where you train on the a labeled version of one dataset, and then you test with linear evaluation on the labeled version of another dataset. This could be cifr10 and cifr100 or vice versa. In grain, we have tested the method with linear evaluation on cifr120 by using the 20 superclasses of cifr100. The idea is to check if the representation are better for coarse grain or fine grain classifications. Finally, we have fine-tuning. This is done on stl10, and in particular, we train the model on the a labeled version of stl10, and then we fine-tune on the small amount of labeled data of stl10. As you can see across all this task, our method has the highest accuracy. We are also tried using different backboards, like smaller backboards, like com4, resNet8, or larger backboards, like resNet32, or resNet36. We are also provided results with ablations. We are reshore two type of ablation. In myj, we are going to remove the relation head. We are going to use a simple dot product between the representation and then impose a binary central clause. In the ablation minstead, we are going to replace the binary head with an encoder g that is going to encode the lateral representation in a Latin space, and then we are going to perform a dot product on top of this representation followed by binary central clause. In cinstead, we have our entire pipeline with our a relation module, and that takes as input the aggregated pairs and then minimizes binary central clause. As you can see from the table on the bottom, just with the full pipeline, you achieve the best performances. As you can see, there is a drastic drop in performance with both the ablations. This means that the relation head is fundamental, and without the relation head, we have a drastic drop in performances. Here we show experiments of qualitative comparison. What we do is we take the representations and we project them on our Cartesian plane by using Disney. This is done on cifr 10. With different color, we highlight the 10 different classes of cifr 10. We compare the supervised relation arisening a rotation net. As you can see, if you compare relation arisening on the center, we rotation net on the right. For relation arisening, it is a better differentiation between the different classes. You can then also perform a different task, an image retrieval task, and we take each image in the test set and we retrieve the 10 most similar images in the later representation space by using an EpliDian distance. As you can see here, we have the confusion matrix for cifr 10. And on the main diagonal, we have darker cells for our approach, meaning that we retrieve most of the time the right class. If we compare the accuracy, we see that relation arisening return the right class 67% of the time, while rotation net just 47% of the time. We can also compare in a qualitative way the images that are returned by the two methods. On the left, you see relation arisening on the right rotation net. We compare, for instance, the first row. We see that in our case, for the query that is a red car, we retrieve most of the time the red cars, while for rotation net, sometimes retrieves tracks or even boats. We can also check other images, for instance, the row four. The query is about relation arisening returns most of the time about why rotation net, sometimes retore a boat, but most of the time returns planes. So in conclusion, we have seen that relation arisening could be an effective method for such a revised learning. This method is very easy to implement flexible and robust. And as very man showed that it performs a recent set of their competitors on different benchmarks and with different backpots. The blission have shown that the relation edge is a fundamental component to achieve these top performances. If we should work, we would like to focus on better understanding the training dynamics behind relational reasoning and on evaluating this method on other settings, for instance, in the full-sphere-by-settings. Those are my contacts. If you have any questions, you can send me an email to this address. And thank you for your attention.