One-Shot Transfer of Affordance Regions? AffCorrs!
Authors: Denis Hadjivelichkov
Sicelukwanda Zwane
Lourdes Agapito
Marc Peter Deisenroth
Dimitrios Kanoulas
Venue: CoRL 2022 Poster (2022)
Keywords: One-shot
Affordance
Correspondence

Abstract: In this work, we tackle one-shot visual search of object parts.  Given a single reference image of an object with annotated affordance regions, we segment semantically corresponding parts within a target scene.  We propose AffCorrs, an unsupervised model that combines the properties of pre-trained DINO-ViT's image descriptors and cyclic correspondences.  We use AffCorrs to find corresponding affordances both for intra- and inter-class one-shot part segmentation. This task is more difficult than supervised alternatives, but enables future work such as learning affordances via imitation and assisted teleoperation.

One-Shot Transfer of Affordance Regions? A AffCorrs!
Denis Hadjivelichkko Centre for Artificial Intelligence University College London
Sicelukwanda Zwane Centre for Artificial Intelligence University College London
Marc Peter Deisenroth Centre for Artificial Intelligence University College London
Lourdes Agapito
Department of Computer Science University College London
Dimitrios Kanoulas
Department of Computer Science University College London
Abstract: In this work, we tackle one-shot visual search of object parts. Given a single reference image of an object with annotated affordance regions, we segment semantically corresponding parts within target scene. We propose AffCorrs, an unsupervised model that combines the properties of pre-trained DINO-VITY im- age descriptors and cyclic correspondences. We use AffCorrs find correspond- ing affordances both for intra- and inter-class one-shot part segmentation. This task is more difficult than supervised alternatives, but enables future work such as learning affordances via imitation and assisted teleoperation. Project page with code and dataset: https: / //sitttee...////// //itte.googllee
Keywords: One-Shot, Affordance, Correspondence
Introduction
Robot agents can significantly benefit from perceiving and understanding what the environment affords them to do. Affordances [1] are representations of how a part of the environment can be used, e.g., a spoon affords to be grasped, and to contain liquid. Being grounded on human semantics, they are intuitive and explainable. Part-based affordance representations can be efficiently used as an intermediate representation that reduces the dimensionality of many robot learning problems significantly [2]. Just people able to transfer the knowledge of object's functionality to other objects from only a few examples, e.g., the graspable handles of jugs to mugs, it would be beneficial for robots understand such correspondences, too. We motivate the topic of recognizing known affordance regions in unseen novel objects as a useful step toward more autonomous robots, assisted teleoperation, visual inspection, and scene understanding.
Evidently, semantic part correspondence can achieved with fully supervised methods [3, 4]. How- ever, they are limited objects similar to the ones present in the dataset, each requiring multiple instances for better generalization of the object category. Meanwhile, the self-supervision and un- supervised learning paradigms, present an alternative direction, which alleviates the data annotation bottleneck [5], and could enable robots to learn continually by themselves [6].
In this work, we demonstrate how the pre-trained DINO- VIT model [7], which is shown to produce good co-segmentation and point correspondences [8], can be used for part querying and finding semantically corresponding parts in one-shot a novel formulation of the One-Shot Instance Seg- mentation (OSIS) problem extended to parts, which we call One-Shot Affordance Part Segmentation (OSAPS). The method enables us to query any segmentation mask leveraging the semantic prior given by a user, or a preceding system. In particular, we demonstrate how querying on affordance
*Corresponding authors are (dennis hadjivelichkov, kanitrass.kkooulas/// kww.kkanoolaaaooo.. Guclllllssuul..... ac.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

part regions can be associated with predefined skills. As far as we are aware, this work is one of the first extend one-shot instance segmentation to part masks, and affordance parts in particular. Our proposed method presents several contributions: (i) One-Shot transfer of affordance regions in- stead of learning through supervision: (ii) Decoupled skills that allow the separation of affordance discovery and execution; (iii) Query-based transfer instead of co-part segmentation, which allows us to transfer specific semantic parts rather than parts based on visual or geometric features only; (iv) Benchmark subset of an affordance dataset curated for one-shot part transfer.
2 Related Work
Most affordance learning work focuses on fully supervised methods that learn a particular af- fordance, predefined in the training set image segments, contact points interaction re- gions [9, 10, 11]. Some works focus on learning from interaction combined with geometric and perceptual features [12, 13]. Model-based approaches are a potential path towards efficient knowl- edge transfer. However, they still rely on expensive annotations, such as full object template mod- els [14]. One-shot affordance detection was recently demonstrated [15], but this approach is limited to instance masks and bounding boxes, which are insufficient to transfer knowledge to a robot.
Models requiring little to no supervision have great potential for robot applications that benefit from adapting quickly and learning continuously. It is of particular interest to understand how previous works in computer and robot vision solve the semantic correspondence problem in the domains of point- and part-correspondence with limited supervision.
Many works enforce the semantic relationship between two inputs through cyclic matching, ei- ther through cyelic loss or a proxy representation. Recent examples include the use of uniform category-level representations of objects as template 3D objects [16] or unit spheres [17]. Dense Object Nets [18] extended previous work [19] and presented method trained with self-supervision by projecting points across multiple viewpoints of the same objects, and showing how the learned object descriptors effectively generalize over other semantically similar objects. Recently, they have been extended to learn from even less data and work with more objects through optical flow of monocular videos [20], neural radiance fields [21], and unsupervised object classification [22]. Part- based methods also often use latent representations that encode each part's appearance, shape, or pose [23, 24). Very close to our method is [25] that it is using cycle-consistency over transformer descriptors to produce one-shot instance segmentation.
Other methods rely on large image datasets, such as ImageNet, for pre-training of vision mod- els as means of encoding semantic and perceptual information. Such works in point correspon- dence [26, 27, 28, 29] and part correspondence [23, 24) show wate-of-theean performance on several computer vision benchmarks. However, these methods often struggle with highly occluded inputs with large viewpoint variance. Caron al. [7] show that DINO-VIT, a variant of ViT [30] trained via self-supervised knowledge distillation, can produce descriptors that contain explicit informa- tion about the underlying semantic content, with properties suitable for k-nearest neighbours (kNN) search. These properties were recently used in unison with cyclic correspondence for unparalleled co-part segmentation without fine-tuning [8]. However, the co-segmentation task does not solve our problem, since it often results in segmentation that is consistent but semantically meaningless.
The literature review leads several conclusions: (i) for any local descriptor to be suitable for semantic correspondence, it needs to be 'aware' of the full semantic context of the object that it is part of; (ii) in matching any two objects belonging to the same semantic category, not all points will always have a true correspondence, either due to occlusion or the lack or addition of parts (such as handle on a cup, or a switch on a lamp); (iii) the currently best performing models combine semantic priors provided by pre-trained models and cyclicity.
3 Method
The one-shot semantic instance segmentation (OSIS) problem [31] is defined as finding and seg- menting a previously unseen object in novel scene, based on single instruction example. Simi- larly, in this work, we aim to solve one-shot affordance part segmentation (OSAPS): given support reference image lg E 133xHaxx WR and query mask region Mg E 1 HHxWa, the task is to find the

---

semantically corresponding region in the target image Ir RaxHyxWr The variables HR. WR denote the height and width of the support, while Hr, Wy denote those of the target, respectively.
Element-wise Multiplication
Pre-trained
Forward matching
Softmar
Vie
Grouping
Pro trained DINC-VIT
Us
A
Centroids
Kg
Pro
CRF
Saltency
My
Fig
Mg
Backward matchung
Rec Recegroundd aaeegy
2 Nx
Step 1: Pre-processing
Step Correspondence
Step 3: Post-Processing
The support descriptors FR belonging to the query area Mo are grouped into Ke query centroids. The salient target image descriptors are grouped into Kr target descriptors. Cyclic correspondence is enforced by matching Kg to Ky (forward matching), and Ky to Fr (backward matching). A score is computed for each centroid in the target. The centroid scores are mapped to the descriptors belonging to each grouping. Finally, each pixel determined to be foreground (or not) by the CRE, which compares the scores with a baseline energy level.
Figure 1:
The overall system diagram:
3.1 Query Region Correspondence
Pre-processing:
The key takeaway from successful attempts related works that eyclicity plays an important role in correspondence-4 matching region corresponds to the query region and vice versa (i.e., the query region corresponds to the matching region). Given the kNN-like properties of DiT, propose to use its descriptors to find correspondences. Thus, get descriptor images FR and Fr corresponding to IR and Ir, respectively. Since dense descriptors retain their spatial locations, we can use the query mask Mo and target mask My (e.g.. one generated through saliency or user-provinn usov-prrvdddd) to only select the foreground pixels (Fig. 1-Step 1).
A conventional pixel correspondence method, such as [32], can then be applied to find matches between the two images. However, is dealing with several issues: (i) occluded missing parts/objects in the target (e.g., cup that does not have handle); (ii) some parts are more dis- tinctive than others (e.g., the tip of a blade versus its centre); (iii) descriptors could be noisy and Euclidean distance between them is not sufficient for best-matching (see [18, 22]). We start by grouping the descriptors into Kg query groups and Ky target groups (see Fig. 1-Step 2), we get mean descriptors Fe and Fg. respectively, which are less noisy than single descriptors.
Forward matching: Through pairwise cosine similarity, we can find how close each of the Fe centroids is to each Fr. centroid descriptors, as (m. ) E Ko For each query centroid, we determine the probability that it matches the target centroids as
Aor = softmaxr.Jg (sim(Fe, Fr.jg/ tor))
(I)
where tor is the softmax temperature and T.fg is the target centroid axis. We want to enable one-to- many matching, well matching of regions with different scales (c.g., matching small handle to large one). However, we find that the matching probability Agr in larger less distinctive areas spread more and thus much smaller. By summing over the query centroids as in Eq. (2), we get "votes" Vor which deal with this issue by summing over the query centroids as
Ver A
The votes balance well between matches to distinctive areas (which are fewer but more confident) and matches to less distinctive ones (which are more numerous but less confident).

---

Backward matching: Similarly, we could find the matches from the target to the support. We match to the full support descriptor image FR rather than the centroids of the selected area Fg with this information, any matches that correspond to another area of the support image can be excluded. Thus, our backward matching affinity is computed with the full FR as
ATR softmax (sim(Fr.se FR)/ TTR): while the probability that each target centroid corresponds to the query region of the support image is computed by summing over the mask query pixels Me:
Pre Σ ATR
(4)
Me
Post-processing: By multiplying Pro and Ver. we get a 'score' ST.fe for each target centroid, as Sr.fx Ver Pro- We map each pixel to the score value of the centroid which represents it (See Fig. 1-Step 3). Each target centroid with Pre > 0.5 is likely a correspondence. Meanwhile, Ver is not a probability, but rather sum of Ko probabilities, each of which represents match that is more likely than average to represent a true correspondence if it's larger than 1/Kr- Hence, we deem the heuristic threshold Ver > KØ/ Ky as representative of a likely match for each target centroid. Finally, the two thresholds can be multiplied to get a 'score' threshold for each centroid.
Using Conditional Random Field (CRF), smooth binary mask can be produced which loosely follows an energy boundary we set the foreground energy term to be the scores image, while the background energy constant of Kg/2K; which is the score threshold.
Design choices:
Our choice descriptor model is the DINO- ViT-S with patch size 8, pre-trained on
ImageNet, due to superior properties in the similar task of co-part segmentation [8]. The support and target descriptors filtering is done by applying the support query and saliency masks, respectively. Clustering is achieved via Fast K-Means [33] into an over-segmented image with Kg Kr 10 2
We choose for = 0.2, TyR = 0.02 empirically. Having a large Egr means the forward matching is lenient and selects many potential candidates, while the low tyR filters in only the parts of those candidates that are confidently matching to the query part rather than the rest of the reference.
3.2 Affordance Transfer
We propose one-shot affordance skill transfer by defining stack of skills S, i.c., affordance functions (((ff))) associated with manually annotated parts (and their descriptors) Oufʃ sensor inputs. Those skills would then be transferred to the corresponding parts θuʃʃ of other objects by applying (, For example, skill might be to grasp at the centroid of graspable region, or place ball containment region. In more general terms, the skill can be formulated as Sj(Oaff; Oclass, X). where Octass is the (either categorical or latent) class of the support object and X is the robot's sensory state. The skills can be either robot-specific or robot-agnostic, since the chosen affordance representation does not make assumptions. this work, we show this system with simple first-order affordances (grasping and containing) in simple scenes, and leave multi-object manipulations for the future.
4 Experiments
4.1 Quality of Part Transfer
Metrics: We use metrics standard in affordance learning literature [34, 35]: per-affordance class In- tersection over Union (also called Jaccard index), and F," -measure [36] which is a weighted version of loU that accounts for pixel location and mask interpolation.
Datasets: As our main benchmark, we use the UMD Affordance Dataset [37] due to its variety of objects, allowing us to evaluate affordance transfer in both inter- and intra-class pairs. Similarly to how works in one-shot instance transfer use modified folds of PASCAL-5' [38] and COCO-20 [39], we present UMD a one-shot correspondence variant of UMD, which is composed of a single
2 Any similar method such superpixelization could work long as the final patches are dense, loosely follow the objects' geometry and produce segments smaller equal in size the smallest distinctive part. We found that any value larger than 3 leads to comparable results for the used datasets. However, the number of segments should be larger than the number of distinct areas of the query parts.

---

instance of each object in the dataset with both RGB image and affordance ground truth annotation. The original annotations are kept to highlight the difficulties of semantic transfer, as two human annotations are the same. The classes include common objects such as bowls, ladles, and knives, with manually labelled grasp, scoop, wrap-grasp, support, contain, cut, and pound affordances.
Experimental Setup: For each object in UMD', we attempt to transfer its ground truth part masks each other object of the same class (intra-class) and to each other object of classes that possess the same affordance (inter-class) from the dataset. The quantitative metrics are averaged over each affordance type. The qualitative comparison is shown well.
Baselines: As an upper baseline, we include the reported metrics of fully supervised [34, 35] on the UMD dataset (standard split for intra- and novel split for inter-class results). These not directly comparable with our method, since they could learn the way affordances are labelled within the dataset through supervision and evaluated on the test subset of UMD, instead of UMD'. As base- lines, we have included two variants of the current SotA in one-shot instance transfer-BAM [40]. We also include the unsupervised co-segmentation method [8]. which inspired our work. Since this method produces 2 to 10 unsupervised segments (by using K-Means elbow point), we take all segments in the support that have least 50% overlap with the support ground truth, and use the aggregate of their correspondences as the 'estimated parts'. This estimate is then compared with the target ground truth.
Finally, to showease the difficulty of semantic labelling we also included a 'human level': a person shown two objects from the same class, they are given the task of one-shot transferring labels by observing the ground truth annotation of one image and annotating the second. This is done once for each UMDD object as the target, after which the metric score is scaled by the number of objects belonging to that class before computing the per-affordance means.
Support
Intra-Class Targets
Inter-Class Targets
Figure 2: The annotated affordance regions from each support image are transferred with AffCorrs to intra- and inter-class targets the same row. Colours represent grasp, cut, contain, wrap-grasp.
Grasp
Cut
Scoop
Contain
Wrap-grasp
Pound
Support
loU
IoU
Fj
IoU
F7
loU
IoU
IoU
Fj
loU
Fj'
Supervised
ResNel [34]
0.71
0.79
0.86
0.86
0.84
0.72
0.55
ADNet [35]
0.73
0.72
0.80
0.85
0.81
0.87
0.76
AffNec [3]
0.73
0.81
0.76
0.83
0.82
0.79
0.84
Unsupervised One-Shot Transfer
BAM-ResNNe [40]
0.26
0.26
0.28
0.23
0.52
0.57
0.57
0.60
0.42
0.45
0.45
0.50
0.43
0.60
BAM-VGG [40]
0.15
0.17
0.17
0.13
0.43
0.45
0.56
0.59
0.41
0.45
0.39
0.44
0.27
0.41
DINO-VIT [8]
0.45
0.51
0.57
0.64
0.61
0..4
0.42
0.48
0.53
0.66
0.66
AffCorrs (ours)
0.55
0.65
0.72
0.81
0.73
0.81
0.82
0.87
0.83
0.89
0.78
0.87
0.83
0.87
Human
0.59
0.79
0.64
0.82
0.66
0.83
0.72
0.79
0.73
0.74
0.74
0.74
Table
Comparison of per-affordance metries on intra-class pairs.
5

---

Support Set
Ground Truth
AffCorrs (ours)
Human
BAM
CoPart- Cogart-Segmentation
Figure 3: Visual comparison of the segmentation produced by various methods. Colours represent grasp, cut, contain, wrap-grasp. Note that Co-part segmentation has a separate colouring.
Grasp
Cut
Scoop
Contain
Wrap-grasp
Pound
Support
loU
B
IoU
Fj
IoU
F6
IoU
FV
IoU
F/
IoU
Fj
IoU
F
Supervised
ResNet [34]
0.33
0.51
0.69
0.52
0.85
0.09
0.51
Unsupervised One-Shot Transfer
BAM-ResNer [40]
0.22
0.25
0.22
0.25
0.20
0.21
0.51
0.34
0.17
0.18
0.13
0.16
0.12
0.13
BAM-VGG [40]
0.13
0.15
0.13
0.14
0.17
0.18
0.50
0.52
0.16
0.18
0.13
0.15
0,05
0.05
DINO-VIT [8]
0.39
0.45
0.57
0.60
0.66
0.68
AffCorrs (ours)
0.39
0.41
0.51
0.50
0.62
0.65
0.71
0.75
0.83
0.87
0.72
0.73
0.82
0.79
Table 2: Comparison per per-affordance metrics on inter-class pairs.
Results: visualization of parts transferred with AffCorrs in Figure 2 shows that the masks are rel- atively robust to viewpoint variance (see mugs), and missing correspondences (see knife-to-hand tohhammer and mug-to-cup), while being surprisingly capable of transferring affordance across dissimilar gions. In Figure 3 we show comparison with the unsupervised baselines. Both the qualitative and quantitative comparisons (in Tables 1 and 2) affirm that AffCorrs performs better on UMD The BAM baseline under-performs, likely due to being tailored for whole object instance transfer rather than parts. Meanwhile, the co-part segmentation, which uses the same saliency masks and backbone as AffCorrs, appears to often ignore the foreground when it doesn't deem it common enough across objects, and produce parts that don't align with what we would consider semantically significant.
4.2 Affordance Transfer in the Real World
To showcase the application to affordance transfer and evaluate the current limitations in realistic scenes, we present the following evaluation setup: we use Franka Emika ma- nipulator with arm-mounted RGB-D camera. As query, toy with annotated grasping area, and a mug with annotated containment. The query is one-shot transferred to the robot's unseen environment. Finally, the robot attempts to use the affordance skill.
Experimental Sctup:
single image of screwdriver
We test the method in two settings (See Figure - single object and multiple objects. Single object scenes contain one object that belongs to the same class as the query (intra-class), or an object from a different class (inter-class) that has an affordance equivalent to the queried one (e.g., graspable part). In the scenes containing multiple objects, there several objects that are 'true' correspondences (See Appx. C) ,),a along with distractor objects. In both cases, the background was varied with different surfaces (table, carton, textile, whiteboard). The objects used are relatively spaced out, with occasional occlusion. Ten examples of either setting were used for the following robot experiments.
"The appendices detail an ablation over AffCors variants (Appx. A) further outputs of AffCons (Appx. D. E) and comparison with flow-based baseline (Appx. F).

---

Target
Support
One object
Multiple Objects
Figure 4: Examples of robot test scenes either with object multiple. The red area denotes correspondences, the circle the next object selected for manipulation based on the highest point.
Two simple first-order affordance skills shown: grasping and containment. More complex multi- object interactions are left for futurre work. The grasping skill is defined as picking at the 3D centroid of a part, with grasp orientation along the largest PCA axis in XY-space (i.e., grasp). The containment skill is defined as opening the robot gripper above the XY centroid of the affordance region. Each affordance is attempted ten times for each setting. An attempt is deemed successful if the robot successfully uses the affordance of all objects that correspond to the query. e.g., grasps all objects with tool handles and doesn't it attempt grasp non-corresponding object.
assume top
Support
TLCICA
Figure 5: Example of the robot finding and using the handle-grasp grasp affordance.
Results: An example grasp in multi-object scene is shown in Figure 5. In the one-object setting, observed a 100% success rate for both grasping and containment. In the multi-object setting, the grasping success rate dropped to 70% and the containment to 80%. Refer to Appendix B for grasp- ing location examples and baseline comparison. As noted by previous OSIS approaches [31, 25], AffCorrs too shows that cluttered scenes are more challenging. which underscores the importance of extending the model in order to process such scenes robustly.
Support
Target
Support
Target
Support
Target
Figure 6: Several examples of one-shot transfer, from left right: Transfer (i) from a mug simulated in Isaac Sim to real jug: (ii) from a USB cable to an Ethernet cable; (iii) from rose to water loti. Note that the reflection of the flower was marked corresponding as well.
4.3 Transfer across Less Common Objects
Finally, we briefly motivate the practical usefulness of the one-shot property of AffCorTs over su- pervised methods: AffCorrs can produce good semantic part correspondences across less common objects, allowing it to work with very specific affordances that likely are not represented in any big dataset. Moreover, transferring a skill from simulation to reality could be made easy by using such an affordance representation; see examples in Figure 6.
7

---

Limitations
We highlight several important limitations the current method (see Figure 7): AffCorrs searches for correspondences with probability- cycle-consistency, which means that it can find corre- spondences between vastly different objects provided that they have similar descriptors. This is in the method's favour when the support image suitable e.g.. when transferring from screwdriver to a hammer's handle, but not as much when the support image is different, such as a bowl to hammer. We have seen that the descriptor similarity alone may incorrect matches from appearing. which limits the current model to simpler scenes without severe clutter. This is also limiting the method's ability perform well in clutter. Mcanwhile, the alterna- tive of having conservative matching would limit the transfer-ability across inter-class pairs.
be enough always prevent
Another issue stems from the descriptors themselves the DINO-ViT descriptors confuse between an object's texture (e.g., print of a dog on mug) and an actual object (e.g.. real dog). This would mean that same-class objects with and without textures sometimes result in different parts. The transformer's positional encoding makes the model biased toward picking a correspondence that similarly located within the image instead of the actual correspondence. This limitation could potentially be addressed by performing flipping and cropping augmentations before the clustering steps. terms affordances, this work shows some simple affordance interactions, however the method could, in theory, be applied to multi-object interactions. The limitations of that direction are yet be assessed.
Support
Ground Truth AffCorrs
Support
Ground Truth
AffCorrs
Support
Ground Truth AffCorrs
Figure 7: Six examples of model failures: some failure can likely be explained by the colour and shape of the parts shown on the top row of examples, e.g., black knife handle mismatched to black hammer. AffCorrs also fails due to the inherited positional encoding, observed in the bottom row examples.
6 Conclusion and Future Work
In this work, showed how pre-trained DINO-ViT's descriptors be adapted for part-based transfer of affordances. We have shown that AffCorrs is better suited for this task than the cur- rent best one-shot instance segmentation baseline and has impressive cross-object transfer of part segmentation. However, it's limitations in working with clutter need to be addressed before any real-world application. Potential candidates to solve this problem include using object detection model (such as DetCo [41]), more sophisticated matching, or some form of unsupervised latent clas- sification. Solving this issue would make the method suitable for more complex dataset benchmarks. such as the IIT's affordance dataset [10]. While this work is the first tackling the one-shot affordance transfer problem, opens the door for many future directions such learning both discover and use affordances from one-shot observations by looking at how people interact with objects; using affordance regions for transfer of more complex multi-object interactions; assisting teleoperation using chbordanne-guidde object manipulation rather than hand-to-robot movement transfer.
Acknowledgement
This work was supported by the UKRI Future Leaders Fellowship [MR/V025333/1] (RoboHike) and the CDT for Foundational Artificial Intelligence [EP/S021 566/1]. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

---

References
[1] J. Gibson. The theory of affordances. In R. Shaw and J. Bransford, editors, Perceiving, Acting, and Knowing: Toward Ecological Psychology, pages 67-82. Erlbaum, 1977.
[2] Borja-Diaz, O. Mees, G. Kalweit, Hermann, Boedecker, and W. Burgard. Affordance Learning from Play for on Robotics and Automation (ICRA), 2022.
Efficient Policy Learning. In IEEE International Conference
[3] T.-T. Do, A. Nguyen, and I. Reid. AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection. In IEEE International Conference on Robotics and Automation (ICRA), 2018.
[4] A. Nguyen, D. Kanoulas, D. Caldwell, and N. Tsagarakis. Detecting Object Affordances with Convolutional Neural Networks. In 2016 IEEE/RS International Conference on Intelligent Robots and Systems (IROS), 2016. doi:10. 100//ROS.201...755444
[5] L. Ericsson, H. Gouk, C. C. Loy, and T. M. Hospedales. Self-Supervised Representation Learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39(3): 42-62, 2022. 110.00010000MMMP222213333333
[6] J. M. Wong. Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics. CoRR, abs/1611.00 1.00001 2016.
[7] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerg- ing Properties in Self-Supervised Vision Transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
[8] S. Amir, Y. Gandelsman. S. Bagon, and T. Dekel. Deep ViT Features Dense Visual Descrip- tors. arXiv preprint arXiv:2112.05814 2021.
[9] T. Lüddecke and F. Wörgötter. Learning to Segment Affordances. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pages 769-776, 2017. doi:10.11 109/ ICCVW.2017.66.
[10] A. Nguyen, D. Kanoulas, D. G. Caldwell, and N. G. Tsagarakis. Object-based affordances detection with Convolutional Neural Networks and dense Conditional Random Fields. In 2017 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5908- 5915, 2017. doi: 1e211..110990 109//OOOOOOOOO 2017,8206484,
[11] A. Nguyen, D. Kanoulas, I.. Muratore, D. G. Caldwell, and N. G. Tsagarakis. Translating Videos to Commands for Robotie Manipulation with Deep Recurrent Neural Networks. In IEEE International Conference on Robotics and Automation (ICRA), pages 3782-3788, 2018. doi: 11..1..111.......... 201....0777....
8460857.
[12] T. Mar, V. Tikhanoff, G. Metta, and L. Natale. Self-supervised learning of grasp dependent tool affordances on the iCub Humanoid robot. 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 3200-3206, 2015.
[13] Mar, Tikhanoff, G. Metta, and I.. Natale. Self-supervised learning of tool affordances from 3D tool representation through parallel SOM mapping. In IEEE International Conference on Robotics and Automation (ICRA), pages 894 904-901, 2017. doi: 1119//CRA.001...99999 10.
[14) P. Abelha, F. Guerin, and M. Schoeler. A model-based approach to finding substitute tools in 3D vision data. In IEEE International Conference on Robotics and Automation (ICRA), pages 2471-2478 2016. doi:
[15] H. Luo, W. Zhai, J. Zhang, Y. Cao, and D. Tao. One-Shot Affordance Detection. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI), 2021.
[16] B. Wen, W. Lian, K. Bekris, and S. Schaal. CaTGrasp: Learning Category-Level Relevatl Grasping in Clutter from Simulation. IEFE International Conference on Robotics and Automa- tion (ICRA), 2022.

---

[17] A.-C. Cheng. X. Li, M. Sun, M.-H. Yang. and S. Liu. Learning 3D Dense Correspondence via Canonical Point Autoencoder. In Advances in Neural Information Processing Systems, 2021.
[18] Florence, L. Manuelli, and R. Tedrake. Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. Proceedings of the 1st Conference on Robot Learning (CoRL), 2018.
[19] Schmidt, R. A. Newcombe, and D. Fox. Self-Supervised Visual Descriptor Learning for Dense Correspondence. IEEE Robotics and Automation Letters, 2:420-427, 2017.
[20] U. Deekshith., N. Gajjar., M. Schwarz., and S. Behnke. Visual Descriptor I Learning from Monocular Video. Proceedings of the 15th International Joint Conference on Com- puter Vision, Imaging and Computer Graphics Theory and Applications Volume 5: VIS- APP, pages 444-451. INSTICC, SciTePress, 2020. ISBN 978-989-758-402-- doi:10.5220/
0008989304440451.
[21] L. Yen-Chen, P. Florence, J. T. Barron, T.-Y. Lin, A. Rodriguez, and P. Isola. NeRF- Supervision: Learning dense object descriptors from neural radiance fields. In IEEE Inter- national Conference on Robotics and Automation (ICRA), 2022.
[22] D. Hadjivelichkov and D. Kanoulas. Fully Self Sule-Supe Class Awareness in Dense Object Descriptors. In A. Faust, D. Hsu, and G. Neumann, editors, Proceedings of the Sth Confer- ence on Robot Learning (CoRL), volume 164 of Proceedings of Machine Learning Research, pages 1522-1531. PMIR, 08-11 Nov 2022. URL. bestps://proeeeeee html.
mlr. press//
1164/hadjivelichhoo22
[23] W.-C. Hung, V. Jampani, S. Liu, P. Molchanov, M.-H. Yang, and J. Kautz. SCOPS: Self- Supervised Co-Part Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[24] Q. Gao, B. Wang, L. Liu, and B. Chen. Unsupervised Co-part Segmentation through Assembly. In Proceedings of the 38th International Conference on Machine Learning (ICML), volume 139 of Proceedings of Machine Learning Research, pages 3576-3586. PMLR, 2021.
[25] G. Zhang, G. Kang, Y. Yang, and Y. Wei. Few-Shot Segmentation via Cycle-Consistent Trans- former. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Infor- mation Processing Systems (NeurIPS), pages 21984-21996, 2021.
[26] U. Gaur and B. S. Manjunath. Weakly Supervised Manifold Learning for Dense Semantic Object Correspondence. In International Conference on Computer Vision (ICCV), pages 1744- 1752, 2017. doi: 10. 1091111..2111...199999
[27] J. Lee, D. Kim, W. Lee, J. Ponce, and B. Ham. Learning Semantic Correspondence Exploiting an Object-level Prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[28] Lee, D. Kim, J. Ponce, and Ham. SFNet: Learning Object-aware Semantic Flow. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[29] S. Choudhury, Laina, C. Rupprecht, and A. Vedaldi. Unsupervised Part Discovery from Contrastive Reconstruction. In Advances in Neural Information Processing Systems, 2021.
[30] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. De- hghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An Image Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Conference on Learning Representations (ICLR). OpenReview.net, 2021.
[31] C. Michaelis, I. Ustyuzhaninov, M. Bethge, and A. S. Beker. One-Shot Instance Segmentation. CoRR, abs/1 81 1.1.111500 15577, 2018. doi: 185.44/AI//A..1111......
[32] T. Xiao, S. Liu, S. De Mello, Z. Yu, J. Kautz, and M.-H. Yang- Learning Contrastive Rep- resentation for Semantic Correspondence. International Journal for Computer Vision (IJCV, 13065):1293-1309 2022. ISSN 0920-5691. 10.0 0221007777111122222-----100
10

---

[33] G. Hamerly and J. Drake. Accelerating Lloyd's s Algorithm for K-Means Clustering, pages 41-78. Springer International Publishing, 2015. 1o0:100.1077997-3333333 -0119788---19-99999--9 2.
[34] Sawatzky, A. Srikantha, and J. Gall. Weakly Supervised Affordance Detection. Proceed- ings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2017.
[35] K. Chaudhary, K. Okada, M. Inaba, and X. Chen. Predicting Part Affordances of Objects Using Two-Stream Fully Convolutional Network with Multimodal Inputs. In 20/8 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 3096-3101, 2018. doi:10. 10111...//RRRS.... 2018.8593617.
[36] R. Margolin, L. Zelnik-Manor, and A. Tal. How to Evaluate Foreground Maps. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2014. doi: 10.1 10.110//VPP.2014..39
[37] A. Myers, C. L. Teo, C. Fermüller, and Y. Aloimonos. Affordance Detection of Tool Parts from Geometric Features. IEEE International Conference on Robotics and Automation (ICRA), 2015.
[38] A. Shaban, S. Bansal, Z. Liu, Essa, and B. Boots. One-Shot Learning for Semantic Segmen- tation. In British Machine Vision Conference (BMVC). BMVA Press, 2017.
[39] K. Nguyen and S. Todorovic. Feature Weighting and Boosting for Few-Shot Segmentation. In 2019 IEEF/CVF International Conference on Computer Vision, ICCV 2019, Seou,, Korea (South), October 27 November 2, 2019, pages 622-631. IEEF, 2019. 10:10.1119/CCCC 2019.00071.
[40] C. Lang, G. Cheng, B. Tu, and J. Han. Learning What Not to Segment: New Perspective on Few-Shot Segmentation. In Proceedings of the IEEF/CVF Conference Computer Vision and Puttern Recognition (CVPR), 2022. (0w.00485555//x..222..00006
[41] E. Xie, J. Ding, W. Wang, X. Zhan, H. Xu, Sun, Z. Li, and P. Luo. DetCo: Unsupervised Con- trastive Learning for Object Detection. In 202/ IEEE/CVF International Conference Com- puter Vision (ICCV), pages 8372-8381. IEEE, 2021. 10011009CCC444922220000000


Reviews:
[{"review_id": "xpgGft-ocT0H", "reviewer": "Reviewer_Mo7Z", "summary": "This work tackles one-shot visual search of object parts. Given a reference image with a query area marked using a mask, the AffCorrs model can segment the parts of the target image that correspond with the reference image's query area. The method first uses a pretrained DINO-ViT to generate image descriptors for the reference and target images. The descriptors are then masked and clustered, and corresponce between reference and target image descriptors is computed using cyclic correspondence. A conditional random field is used to determine whether pixels are foreground or background. Results are shown on a variant of the UMD dataset, and the authors show that their method outperforms the BAM and co-part segmentation baselines. They benchmark on both inter-class and intra-class image pairs. They also show that their method works in a real robot tabletop pick and place setup.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- One-shot transfer with an off-the-shelf pretrained model rather than supervised learning is very appealing, as supervised learning typically requires collecting a large custom dataset for each new setup\n- The method gives the user flexibility to specify different parts of objects for different end effectors. This is important as affordances varies with different end effectors and different tasks.\n\nWeaknesses:\n- While AffCorrs is demonstrated in a 2D pick-and-place setting, the 2D representation of affordances might not work well for robotic tasks that require 6-DOF, such as approaching a mug from the side to grasp it by its handle\n- Relatedly, the centroid of a segmented part is not always the best place to grasp an object\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper proposes an interesting way to leverage pretrained vision models for affordance prediction in pick-and-place tasks. The method is elegant, and the experiments are well-executed, showing superior performance over reasonable baselines. While the applicability of the method may be somewhat limited for tasks that are not 2D pick-and-place, I believe this paper should very much be of interest and usefulness to the community.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "- I found Sec. 3.1 somewhat hard to follow. Perhaps a figure illustrating just the query correspondence in greater detail might help?", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "o3BYwd0avOAO", "reviewer": "Reviewer_oETv", "summary": "This work proposes a way to identify object parts in a target image given an annotated query image. The goal is to pinpoint parts that are semantically similar, e.g. given a \"support\" image of a knife with a segmentation mask annotating the handle & the blade, return segmentation masks for handles and blades of knives with a different shape or orientation in the target images. Authors also demonstrate success for inter-class examples, such as obtaining a region for a handle of a hammer given the support image with a mask for a handle of a knife.\nThe work employs pre-trained transformer models from the literature for pre-processing, then applies a novel way to impose correspondence between semantically similar parts of the objects. The approach is conceptually similar to some of the prior works that imposed cycle consistency with a use of a segmentation mask [citation 25], but the proposed approach includes a clustering step and a soft way to match clusters in query and target images.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe abstract, introduction and related work sections are well written and clear.\nThe problem formulation and contributions are also stated clearly, which makes it easy to understand the scope of the paper. \nQuantitative experiments on image datasets (annotated with segmentations and affordances) show a non-negligible improvement over previous approaches.\nThe proof-of-concept experiments with grasping using a real robot show author's commitment to make the work relevant to robotics, instead of limiting evaluation to only the static image datasets.\n\nWeaknesses:\nFor this to be a strong contribution to the robotics community, the hardware experiments section should be significantly expanded. \nThis work presents results with a fixed grasping policy, which should be very quick to execute. \nHence, it should be easy to present comparisons with grasping that use at least one other baseline method (e.g. methods in tables 1 & 2). This would elucidate the impact of the improved detection of affordance regions on the target task of grasping. It could be the case that with a simple grasping policy the improved precision on static images does not translate to a significantly improved success rate for grasping (or alternatively, it could in fact have a visibly positive impact on grasp success). It would also be good to include experiments on a wider range of objects and tasks. As of now, the robot experiments are limited to just two sub-tasks: a simple pickup using a long flat handle area, and a drop into a containment area.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "From the robotics perspective: this work has the potential of being useful for task-oriented grasping. Experimental section shows visualizations with qualitative results, quantitative results that compare to several previous works on segmentation, and proof-of-concept experiments robot experiments with grasping. This shows that the work is promising, but to clearly demonstrate its potential, it would be necessary to expand the set of real robot experiments.\n\nThe \"Limitations\" section shows a number of clear failures of the method and gives an intuitive explanation of why the method fails in this particular way. The section seems to be honest and doesn't shy away from showing very visible failures, which would be useful for others in the community who might want to build on this work. Hence, this clear \"Limitations\" section strengthens the presentation of this work by providing intuition for the limitations of the proposed method.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "As of now, the robot experiments are limited to a very simple \"from the top\" grasp strategy and just two sub-tasks: a simple pickup by a long flat handle area, and a drop into a containment area. It should be possible to either attempt a wider range of grasping strategies or expand the set of tasks. For example, grasping cup handles from various angles; or completing a set of tasks after grasping. It would be particularly interesting to focus on inter-class examples, since the most relevant case for robotics would be to avoid having to obtain an appropriate support image for every object class that a robot might encounter in its environment."}}, {"review_id": "mrkfQGSnnKX", "reviewer": "Reviewer_LmPK", "summary": "This work presents a novel problem of transferring part-level affordance from one labeled image to other images in the one-shot setting and proposes a one-shot transfer method that leverages pre-trained DINO-ViT image features and computes cyclic correspondence among pixels on the labeled support image and the target image. The method is quantitatively compared against two baseline methods [40] and [8], and real-world/real-robot experiments are also presented to show the part-level affordance information can be transferred from the only one given support image to other images.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- To my best knowledge, it's the first time that researchers study the problem of part-level affordance transfer in the one-shot setting. The task is very important for robotics.\n- The proposed method of leveraging pre-trained image features and computing cyclic correspondence for converting pixel-level correspondence to part-level correspondence is valid, reasonable, and interesting.\n- Experiments do show big improvements over the two baseline methods [40, 8], both qualitatively and quantitatively.\n- Real-world/real-robot experiments are convincing.\n\nWeaknesses:\n- I'm not sure if the technical contribution of the proposed method is significant. The method directly uses and relies on the pre-trained DINO-ViT features as the per-pixel features to find the pixel-level correspondence. How about the per-pixel correspondence induced by the DINO-ViT features are not good? Then, the method mainly focuses on how to convert the pixel-level correspondence to a part-level one, by using double-way feature similarity matching (or, in another word, cyclic correspondence). The method seems quite simple to me. If the conversion from pixel-level to part-level correspondence is the only technical contribution of the proposed method, I am not sure if the technical contribution is significant enough or not.\n- The authors mentioned the one-shot affordance detection work [15] in the related work but do not provide a direct comparison to it. Can you simply extract the bounding box of the part from the support image and run [15]? How would it work?\n- Could you perform quantitative comparisons to the previous works that find per-pixel correspondence and directly uses the per-pixel correspondence for the part affordance on the target image, say using [18] and \"Choy, Christopher B., JunYoung Gwak, Silvio Savarese, and Manmohan Chandraker. \"Universal correspondence network.\" Advances in neural information processing systems 29 (2016).\"? From these papers, it seems that the part-level correspondence can be easily obtained if the pixel-level correspondence is good enough, which seems to be the cases from checking the results presented in these papers.\n- Is there any ablation study to show that the use of centroids is beneficial? The naive way is to use all the pixels.\n- As shown in Table 1, why is the method performing better than humans? Is there some explanation for this? I notice some gt error for example in Fig. 3, 2nd row & third column, the handle part is wrongly labeled in GT. Is this the cause?\n- The paper writing can be improved. For example, the method part is a bit hard to follow. And also, the setting for baseline methods. For instance, I don't understand what do you mean by \"all parts that have at least 50% overlap\" in Line 171. Why there are parts in the target image, which are overlapping with the part in the support image?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I like the novel and the important problem the paper proposes. However, the solution seems to be quite simple to me, since it's all about converting per-pixel correspondence to part-level correspondence using a simple cyclic aggregation. Also, some baseline comparisons need to be added.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Please see the weaknesses.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "YQEd2-5Jj3Ch", "reviewer": "Reviewer_Zz3M", "summary": "This work tackles the task of transferring an annotated region of interest (e.g. handle) from an image of object to an image of a different (but similar) object. In the specific use-cases considered, these regions typically correspond to possible interaction targets (although the approach is agnostic to this).\n\nThe overall framework consists of leveraging meaningful pre-trained visual features for image patches (using DINO-ViT) and using these to find correspondences between the annotated ‘query’ region in the reference image and the foreground regions in the target image. The key idea is to first cluster the features in the source and target image, and identify clusters of the target features that are both: a) mapped from query pixels when computing source -> target correspondences, and b) are mapped to query pixels when computing target -> source correspondence.\n\nThe empirical results show this leads to better correspondences than similar unsupervised baselines.\n", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- On the plus side, the overall approach is simple and intuitive. It uses strong unsupervised features to find correspondences between a query region in a source image to a target images, and several of the design choices e.g. using clusters, smoothing with CRFs, combining correspondence scores in both directions etc. make intuitive sense.\n\n- The empirical results show that the proposed method does outperform a prior work which uses similar features, but instead predicts (corresponding) unsupervised parts.\n\n- The paper also shows some results on transfer across categories and the results are encouraging.\n\nWeaknesses:\n- The approach presented here is a rather generic one, and I am not sure why the work is positioned as focusing on affordances — nothing in the method actually relies on the regions being affordance. In fact, the term ‘affordance’ may generally also subsume the mode of interaction (e.g. how the action should be performed) and this work certainly does not predict transfer of this (e.g. 6D relative orientation of hand). In fact, this paper presents a method for transferring annotated masks, and actually should have been presented as such.\n\n- While there are some reported baselines, I feel the text is missing a few rather informative ablations/baselines. In particular:\na) Baseline 1: what happens is one computes correspondence via naive nearest neighbors using DINO features?\n\nb) Baseline 2: The vision community has previous had the observation that deep features are good for correspondence. For example [A] showed that instead of naive nearest neighbors, using deep features + flow lead to better correspondences. It would be helpful to know whether the proposed approach performs better than this baseline (modified to use DINO features).\n[A] Do Convnets Learn Correspondence? Long et. al.\n\nc) Ablation: Is the combination of V_{QT} and P_{TQ} necessary? What happens if one uses just one of these terms (with an appropriately chose threshold)?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, I think the paper tackles a useful task and proposes a simple but elegant approach. However, I am not convinced that this should be framed as a ‘affordance transfer’ work as nothing in the method is specific to this fact. On a related note, I think more baselines/ablations which convince the reader that this is indeed a good way for transferring annotated regions are required.\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Adding the requested ablations/baselines would be helpful.\n\nI would also be curious to hear the authors’ opinion on why they view this work as tackling affordance transfer, rather than more generally ‘region transfer’ (and evaluate it as such)?\n"}}]

Author rebuttal:

Decision: Accept (poster)