FastJAM: a Fast Joint Alignment Model for Images
Authors: Omri Hirsch
Ron Shapira Weber
Shira Ifergane
Oren Freifeld
Venue: NeurIPS 2025 poster ()
Keywords: Computer vision
graph neural network
GNN
image congealing
joint alignment
Primary area: applications
TL;DR: Joint alignment of multiple images without a reference in 50 seconds or less

Abstract: Joint Alignment (JA) of images aims to align a collection of images into a unified coordinate frame, such that semantically-similar features appear at corresponding spatial locations. Most existing approaches often require long training times, large-capacity models,
and extensive hyperparameter tuning. We introduce FastJAM, a rapid, graph-based method that drastically reduces the computational complexity of joint alignment tasks. FastJAM leverages pairwise matches computed by an off-the-shelf image matcher, together with a rapid nonparametric clustering,  to construct a graph representing intra- and inter-image keypoint relations. 
A graph neural network propagates and aggregates these correspondences, efficiently predicting per-image homography parameters via image-level pooling. Utilizing an inverse-compositional loss, that eliminates the need
for a regularization term over the predicted transformations (and thus
also obviates the hyperparameter tuning associated with such terms), FastJAM performs image JA quickly and effectively. Experimental results on several benchmarks demonstrate that FastJAM achieves results better than existing modern JA methods in terms of alignment quality, while reducing computation time from hours or minutes to mere seconds. Our code is available at our project webpage, https://bgu-cs-vil.github.io/FastJAM/.

FastJAM: a Fast Joint Alignment Model for Images
Omri Hirsch" Ron Shapira Weber" Shira Ifergane Oren Freifeld The Faculty of Computer and Information Science, Ben Gurion University of the Negev (BGU), Israel The Data Science Research Center, BGU The School of Brain Sciences and Cognition, BGU
{omrihir, ronsha, r shiraif)@oossttttccoss bgu ac. orenfr@bgu.ac..
Abstract
Joint Alignment (JA) of images aims to align collection of images into unified coordinate frame, such that semantically-similar features appear corresponding spatial locations. Most existing approaches often require long training times, large- capacity models, and extensive hyperparameter tuning. We introduce FastJAM,, a rapid, graph-based method that drastically reduces the computational complexity of joint alignment tasks. FastJAM leverages pairwise matches computed by an off- shelf image matcher, together with rapid nonparametric clustering, to construct graph representing intra- and inter-image keypoint relations. graph neural network propagates and aggregates these correspondences, efficiently predicting per-image homography parameters via image-level pooling. Utilizing an inverse- compositional loss, that eliminates the need for regularization term over the pre- dicted transformations (and thus also obviates the hyperparameter tuning associated with such terms), FastJAM performs image JA quickly and effectively. Experimen- tal results on several benchmarks demonstrate that FastJAM achieves results better than existing modern JA methods in terms of alignment quality, while reducing computation time from hours or minutes to mere seconds. Our code available at our project webpage, https: / httpps:///buuue /bguu/uuueee cs- vasvill vil 1. github. io/FastJAM/.
1 Introduction
Joint Alignment (JA) is the task of aligning collection of images by estimating per-image spatial transformations such that, when applied, all images become geometrically consistent in a shared coordinate frame according to certain semantic or geometric criteria (see, e.g., Figure 1). Unlike pairwise alignment, which aligns each image pair independently and often leads to eтror accumulation (i.e., "drifting") ,, JA enforces a global agreement across the entire set, making JA particularly valuable for discovering shared structures between images or building a class atlas. However, achieving JA is inherently challenging: without supervision or reference image, optimization methods frequently collapse into trivial or inconsistent solutions. Moreover, existing approaches typically require extensive computational resources, taking more than an hour [1, to jointly align as few as 30 images. Recently, we proposed method called SpaceJAM [3] that, partially by virtue of new inverse-commosttional loss over dense feature maps, significantly mitigated these computational issues, thereby solved the task in only few minutes. Additionally. SpaceJAM set new state-of-the-art quantitative results. A natural question arises, however: it possible to even better terms of both speed and performance? Fortunately, the answer is positive, as show in the present paper.
Equua Contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Plane
Plane
Aligned
Bird
Bird
Aligned
Bike
Bike
Aligned
Figure 1:
Joint alignment with FastJAM. Given of images of the same object, of different objects from the same category (e.g., motorbikes), our method aligns all images in seconds, compared other methods (minutes [3] or hours [1, 2]).
Here we introduce an even more computationally-ceinn -ffficccent method that solves the JA problem in under 50 seconds, dramatically outperforming prior approaches in terms of speed while maintaining, and in fact typically improving, alignment quality.
Traditionally. JA methods relied on classical approaches such as congealing [4, 5], which iteratively align each image towards the remaining set, or centroid-basee -bssed methods that utilize reference image or latent template [3]. Classical techniques employed feature- based methods (e.g.. SIFT [6]), to establish keypoint (KP) correspondences between images. The rise deep learning, particularly through Vision Transformers (ViTs) [7] and semantic feature extraction methods like DINO [8], has significantly advanced JA by providing richer representations that alleviate some challenges with ViT features, many difficulties persist, leading
faced by traditional methods. However, recent approaches to depend heavily on high-capacity, computationally-cxx expensive models paired with extensive regularization [1, 2]. This reliance not only increases computational demands but also introduces complexity through the requirement of extensive hyperparameter (HP) tuning, ultimately resulting in methods that are slow and often brittle (as the HP tuning is usually dataset-specifiie..
Both congealing and atlas-based approaches typically rely on objective functions that fall into two main categories: geometric losses and semantic (feature-based) losses. In the geometric cases, one di- rectly minimizes spatial discrepancies between corresponding KPs across images, leveraging explicit correspondence information. In contrast, semantic losses operate over dense feature representations and provide a smoother, such dense (and typically (igh-diienssooal) representations, often derived from high-capacity models like DINO [8], are computationally expensive when used within the JA optimization (even the features themselves are kept frozen). Therefore, in this work we adopt the geometrie loss paradigm, offering a sparse, lightweight, and scalable formulation that achieves typically higher alignment accuracy while significantly reducing the computational overhead.
meally--iffereentabbee alignment without requiring explicit KPs. However,
Concretely, we introduce FastJAM, a graph-based framework that achieves fast scalable alignment. Unlike prior methods that rely on dense feature maps and/or computationally-ineen optimization, FastIAM constructs KP graph from pairwise correspondences using an off-the-shelf matcher, where nodes represent KPs and edges encode intra- and inter-image relationships. A

---

Keypeint correspondence
Add image edges build graph
Optimization
ovel Pooling Linear Projection
12
GNN
Homography
1
X
Legend:
Intra-image edge
Inter-image edge
Corresponding Keypoints images
Figure Overview of the FastJAM architecture. Given a set of images, we extract sparse keypoints (KPs) and pairwise correspondences using an off-the-shelf matcher (left; only red-dot matches are shown for clarity). A graph is built by linking KPs within each image (intra-image edges) and across matched pairs (inter-image edges). A GNN with L layers propagates alignment information through this graph (center). Image-level features are then obtained via mean pooling and used to predict per-image homography parameters (0,) for joint alignment.
Table 1: Comparison with recent JA methods on three SPair-71k categories [9]. Runtime is reported average + standard deviation in hh mm: format.
Method
# Params
Losses
#HP
Atlas-free
# Epochs
Runtime
Neural Congealing [1]
28.7N 78.7M
8
8
8000
01:18:30 + 00:06:
ASIC [2]
7.9M
4
5
X
20000
01:06:38 + 00:00:38
SpaceJAM [3]
0.016M
1
0
/
700
00:06:00 = 00:00:12
FastJAM (Ours)
0.13M
600
00:00:49 + 00:00:04
Graph Neural Network (GNN) propagates alignment cues across the graph, and a readout layer produces image-level embeddings used to predict the homography parameters (as shown in Figure 2). Combined with a robust inverse-comoxiitonal geometric loss, FastJAM aligns an entire image set in under 50 seconds. Experiments SPair-71k and CUB-200 show that FastJAM matches exceeds the accuracy of contemporary JA methods while being significantly more efficient and orders of magnitude faster (see Table for comparison). Our key contributions are follows.
1. We introduce FasJAMM, novel GNN-based framework for JA that significantly accelcrates the alignment process (compared with existing methods) from hours/minutes to seconds.
2. FastJAM graph structure allows for the information from the entire image collection to prop- agate between images during optimization, unlike previous "image- rimage-by-image approaches (including SpaceJAM), leading to improvements in JA quality.
3. The first Inverse- Compoosiioiiii JA loss that is based on sparse KP representation.
2 Related Work
Pairwise image alignment. Learning- correspondence methods have substantially improved the accuracy and robustness of image matching. Sparse approaches like SuperPoint [10] and Su- perGlue [1 detect KPs and compute context-aware matches using attention and GNNs, but might struggle in low-texture or repetitive regions due to their reliance on sparse detections. To overcome these limitations, dense methods such as LoFTR [12] compute pixel- wise matches using transformer- based architectures without requiring explicit KPs. Recent advances continue to close the gap between the sparse and dense paradigms. RoMa [13] combines DINOv2 features with hierarchical transform- ers for robust side-baseline matching. DIFT [14] introduces efficient descriptor interpolation from diffusion models. Additional types of relevant dense features appeared in the works of Mariotti et al. [15), who propose spherical viewpoint maps that encode rich geometry, and Xu al. [16], who incorporate directional priors to disambiguate symmetric cases. However, common issues with
3

---

dense matching approaches are that they can be computationally demanding and that they require post-processing because the geometry often breaks.
JA by feature matching. Classical JA methods utilize geometric transformations and handcrafted features. The idea of congealing [4, 5] is to iteratively align images by minimizing a global cost, typically entropy or
of pixel values [17, 18, 5] or descriptors like SIFT [19, 20, 21]. Other approaches simultaneously cluster and align images to their class means [22, 23] or use template matching [24, 25, 26]. In any case, such methods are limited by the quality of the extracted features. Another classical approach models image sets as low-rank linear subspaces [27, 28, 29, 30].
Deep learning has substantially advanced image JA. Huang et al. [31] adapted congealing to CNN features, while Spatial Transformer Networks (STNs) [32] introduced a differentiable module for predicting spatial transformations, enabling end-to-end alignment learning. STNs have since been widely adopted in JA tasks, including congealing [33], atlas construction [34, 35, 36, 37, 38], joint clustering [39, 40], moving-camera background modeling [41, 42], and temporal synchronization of multiple videos [43]. STNs have also been combined with GANs [44] to generate high-quality canonical atlases [45, 46], albeit data demanding.
Recent works have increasingly adopted deep features as the basis for JA. DINO features [8], in
particular, offer robust and semantically-rich representations well-suited for this task. Neural Con-
gealing [1] employs test-time optimization to build class-specific atlases (e.g.. birds) by aligning DINO features using rigid and non-rigid warps, predicted by ResNet-based STN [47]. ASIC [2] utilizes DINO features, learning dense warps from input images to a canonical space through a U-Net architecture [48]. Both these methods are computationally intensive (often exceeding an hour for 30 images on an RTX 4090), require heavy regularization to avoid degenerate solutions, and are prone to instability. ASIC also typically produces fragmented or globally-incohe incoherent align- ments due to the challenges of dense warping. More recently, a previous work from our group introduced SpaceJAM [3], more efficient solution using lightweight ConvNet (CNN) and an inverse-compositional loss over refined DINO features, reducing runtime to a few minutes. However, its reliance high-dimensional feature maps results in substantial memory overhead during the optimization process. In contrast, FastlAM combines sparse KP-based matching with a geometric loss, enabling much faster optimization and better scalability. FastJAM also differs from SpaceJAM in its architecture and the fact that the input to its neural net is based on the entire image collection, as opposed to SpaceJAM's single-image input.
JA by KP correspondence. Several methods tackle JA by explicitly leveraging KP correspondences across the image set and minimizing geometric loss. Shokrollahi er al. [21] construct similarity graph from KP matches to select optimal references for alignment. Safdarnejad et al. [49] propose a temporally-aware congealing method for video frames based on tracked KPs. Flow [50] estimates dense correspondences across images and refines them jointly through graph-based optimization. ASIC [2] also incorporates an initial KP matching step, though its primary pipeline relies on dense warping. Dense matching is often computationally demanding and memory intensive, while sparse KPs offer a more efficient alternative. Building on recent successes of GNNs in pairwise alignment tasks [11, 51, 52]. FastJAM capitalizes on this sparsity and introduces GNN to aggregate alignment information across matched KPs. Combined with a fast mnverse-compositie high-quality and regularization-free JA in seconds and with low-memory footprint.
loss, this enables
3 Method
In this section, we first formally introduce the JA problem ($ 3.1). We then detail how to construct a correspondence graph from pairwise image matches ($ 3.2), while in $ 3.3 we detail the model architecture. Finally, we explain in $ 3.4 how to perform image JA with FastJAM.
3.1 The Joint Alignment Problem and the Inverse-Composstional Framework
Given N images, I = (Ix) depicting different instances from the same semantic class (e.g., cars), the task is to facilitate JA by estimating a transformation TO: E T for each image such that the transformed images (I, T ) are spatially aligned in a shared coordinate frame C. We assume parametric family of transformations T (e.g., homographies), with 7 denoting the transformation

---

Atlas based optimization
Inverse Compositiona Optimization
liverse-Compostionll Keypeints- besed Optimization
Cononical Cocrein.tte
7
T
T
T
T
T
Source
7-9,
Source
7-
Targets
Model
Targets
Model
Model
Model
Model
Model
Model
Gruph
Shared Weights
Shared Weights
Figure 3: Comparison of joint alignment frameworks. Left: Atlas-based methods align each image independently to a canonical space C by minimizing variance. Middle: Existing inverse- pnnerse-compositional (IC) methods estimate C implicitly via relative transformations (T T 8) but process images independently. Right: FastJAM follows the IC paradigm, but differs from previous approaches in that the loss is computed between KPs and all images processed simultaneously (the model process the entire KPs graph during its forward pass), allowing shared reasoning across all images.
for image 4. parameterized by 0,. Atlas-based approaches (e.g.. [1. 2I) usually optimize for latent template, I,. jointly with the transformations. Formally, they solve
NN
arg min (. ET
4 T R(T"::))
where is a discrepancy measure (e.g., g. the Euclidean distance), R(T on the predicted transformations with HPs X, and I is the so-called canonical space or atlas. Due to its notion of centrality, I is also known as the average or centroid image.
X) is a regularization term
In contrast, FastJAM follows congealing-inspiee approach [4, 5] (particularly, Least-Squares (LS) Congealing [17, 18)) which avoids the need to maintain an explicit reference image, together with the modern Inverse Compositional (IC) approach we proposed in [3]. Concretely, the IC approach can be defined via the following loss:
Σ E jijzi D4,,,,070 o 0)).
arg min
(2)
(T ET
Of note, the historical roots of the IC approach go back to the pre-DL era [18]. SpaceJAM [3], however, rather than optimizing over the warping of single image at a time (as was done in traditional LS-congealing) simultaneously optimizes over all transformations.
The IC formulation indicates that the image collection is mapped to a shared space, since
1, ~ 1 o T + I o T = I, T
(3)
As explained in [3], the IC approach obviates the need for using regularization terms. An important distinction between our work and [3] is that in [3] the optimization is based on the discrepancy between dense feature maps (e.g., DINO features), while we adapt it to KPs and rely on geometric measure, as detailed below. Figure 3 illustrates the different JA approaches.
By design, and due to the invrrsecoomoosiiional nature, IC losses are invariant to a single global homography. That is, for any (T )r and any additional transformation T the (T o T ) transformations would give rise to the same value of the loss. The same phenomena happens in not only [3] (and, for Hlihtly-differee over groups (see, e.g., [53, 54]). This is a feature, not a bug, as it simplifies the optimization considerably. Importantly: 1) While this means there are infinitely-many solutions, the implied N-fold joint correspondence is unique. 2) After the fact (i.e., after the optimization done), and for visualization purposes, for example, one can pick the value of T without affecting the quality of the
reasons, [38, 43]) but also many works on synchronization
5

---

Aligned Image Colored
Original Image Colored
Original Image
Aligned Image
Figure 4: Canonical Space Visualization. We visualize the canonical space (C) via a predefined RGB colormap. The first row shows an example of color projection from the canonical space onto a reference triangle. From the second row, we color each image 4, by applying its inverse transformation on C (i.e., C FastAAM maps semantically similar regions to the same areas of C, as shown by the consistent color mapping.
solution. Plausible choices include the inverse of the average homography (which can be computed in various ways, including the so-called Karcher mean [55, 56]) or the inverse of, say, T Note that the latter choice does not imply that the first image had any special importance in the optimization.
3.2 Graph Construction
The first stage of FastJAM involves constructing a sparse graph over KPs extracted from the image collection. This graph encodes both intra-image structure and inter-image correspondences and serves as the input to the GNN. The construction process consists of three steps, detailed below.
Object-centric region extraction. We follow 21 3] and extract object-centric masks for the image collection. We use Grounded-SANN [57], combination the Grounding DINO object detector and the Segment Anything Model (SAM) [8, 58]. Given text prompt corresponding the object category, Grounded-SAM produces a segmentation mask focusing on the object of interest. This mask is used to restrict the KP extraction and matching the relevant region, improving robustness to background clutter and occlusions. Mask extraction takes ~ 0.4 seconds per image and can be parallelized over the GPU.
KP detection and matching. For each image 1, we extract set of sparse KPs X; (ur")) ,...,,
_ (M.)) (where Mj is the number KPs in It) using off-the-shelf image matcher in- side the objects' mask. We use RoMa [13]. but any matcher producing KP correspondences can be used. We chose RoMA due to robustness and fast inference time (~ 0.3 seconds for an image pair on an RTX4090, which can be done in parallel across pairs). For each image pair Ly), RoMa returns a set of KPs, XX), and correspondences Mij C. X; x X, along with confidence score for each match. To improve spatial coverage and reduce redundancy, we apply non-max mn-mmxximmm suppres- sion (NMS) over the matcher confidence scores using a 30 30 window, and retain the top-scoring points. We found that selecting as few as 10 KPs per image is sufficient for our framework.
Intra-image KP clustering. Consider three images, (J,, J2, Jz), and recall that RoMa a pairwise KP extractor and matcher. Running RoMa on (11, 12) produces 10 KPs on I and 10 12; running

---

it on Is) produces another 10 KPs on I1 and 10 on I3; and so on. Consequently, each image (e.g,, I1) accumulates multiple sets of KPs obtained from different pairwise runs. Importantly, these sets are not guaranteed to be identical, even though they all refer to the same image, leading to redundancy. For instance, in bird images, most of the N-1 KP sets extracted from T1 are likely to include KP near the beak tip. though at slightly different locations. To reduce this redundancy and merge semantically similar KPs within an image, we apply a fast, nonparametric clustering step. Specifically, we use Dinari and Freifeld's parallel DP-Means algorithm [59] (with init_ and 1), a highly efficient variant of DP-Means [60], which itself generalizes K-Means to an unknown number of clusters. It takes ~ 0.13 seconds cluster ~ 6000 points. After clustering, we discard the original KPs and retain only the cluster means as the representative intra-image KPs.
(m) Graph definition. We define a single graph g = (V, E) over the entire image set. Each KP, z; is represented node e E V. We add two types of edges: (1) Intra-image edges: In each image I, we fully connect all KPs in X; to model local spatial structure. (2) Inter-image edges: In each matched KP pair (mm) J
(n)) E MMij we add an edge between the corresponding nodes. Each
node E V is initialized with vector h. consisting of the KP's 2D coordinates such that H)) is the initial nodes coordinates matrix. In addition, each node is tagged with a categorical identifier indicating its source image, which is later used to perform image-level pooling. Unlike traditional graph-level readout layers that summarize the entire graph, FastJAM performs structured readout by pooling node embeddings per image. The edge structure is encoded as a binary adjacency matrix A E {0, 1) where Aur = 1 if there is an edge (intra- or inter-image) between nodes and The resulting graph G EV,E, H A) encodes both local geometric structure and cross-image semantic correspondence, and serves as input to a message- mnessag--pssing GNN that propagates alignment information throughout the image collection.
3.3 Model Architecture
Given the graph 9 H)) A), our objective is predict transformation parameter vector 0, E T for each image Jú representing its homography. We treat this as a structured regression task over node features: each node corresponds to KP, and the GNN must propagate alignment-relevant information across the graph to produce per-image outputs. Formally, let f(G) = (0:) be a GNN that predicts the warping parameters from the graph. In our setting, where the goal is to regress image-specific transformations from KPs structured within a shared graph, it is essential to preserve the distinction between a node and its neighbors. GraphSAGE [61] achieves this by applying separate transformations to self and neighbor features, enabling more effective modeling of local asymmetries and node-specific roles which is relevant when propagating alignment cues across inter- and intra-image connections. While intention-based models such as GAT [62] offer expressive edge-aware aggregation, we found them to be empirically slower. In contrast, GraphSAGE provided a favorable balance of speed, stability, and alignment accuracy. making it well-suited for FastJAM's test-time optimization regime, The message protocol for GraphSAGE is
h = a ( (1) h (11) + ( mc)) h(
(4)
where h is the embedding of node layer 1, (v) denotes the neighbors of W! is learnable weight matrix, and a is a non-linear activation function. After L = pressage-passingg layers, we perform per-image readout via global average pooling over all nodes belonging each image:
Z z= = M.
E. VEV.
(1)))
where V; C V is the set of nodes from image 4t and 2; Rd is the resulting image-level embedding. Finally, we project the embedding of each image to the estimated homography parameters:
0 = 2! Wont E R
3.4 FastJAM Joint Alignment
Lie-algebraic parameterization. To ensure matrix invertibility, which is essential for our IC formulation and for stable optimization [63, 3], we represent homographies using the Special Linear group (T) via a Lie-algebraic parameterization as was done in, e.g., [64, 54, 3]. For details, see our supplemental material (SupMat).
7

---

Robust anverse-composition KP loss. Our geometric loss builds upon the IC formulation intro- duced in [3], adapted to sparse KP correspondences. For each image pair (I, I5), the forward warp from I is composed with the inverse warp from I (i.e., T 0)) We penalize the discrepancy between each matched KP pair (17 [m) (n))
E Mij after applying the IC transformation
CXP-IC ΣΣ
Σ
Po (la))m) x1(") o T o T 0:(12)
jzhi (x) (m)) JEM
where P=(2) =2402 is the
robust loss function [65] with parameter a.
This formulation allows alignment to computed at the original KP locations without any reg- ularization term the warps the need to render warped images via expensive interpolation. Compared to dense alignment over nigh-dimensional DINO feature maps, our sparse formulation is both significantly more efficient and more robust missing KPs, wrong matches, and outliers.
Handling reflections. We follow [3] and explicitly check for flips every K epochs during optimiza- tion (where K = 100) and compute the gradient and update the model's weight only for the best configuration. We have found that only checking for horizontal flips is sufficient. This ensures that flipped images still participate in alignment without requiring reflection-awan parameterization.
Implementation details. All experiments were conducted on a single NVIDIA RTX 4090 GPU with 24GB of memory. We optimize FastJAM for 600 epochs using Adam [66] with Geman- McClure robustness parameter a 0.25. We use pretrained Grounding-SAM [57] and RoMa [13] with the default HP once, before starting the optimization. For more details, please see our SupMat.
Limitations. Our main limitation is the reliance on an external image matcher to generate initial KP correspondences. While modern matchers like RoMa provide high-quality matches in many scenarios, the overall alignment quality depends on the accuracy of these correspondences. In addition, FastJAM models geometric transformations using homographies, which may be insufficient in cases involving strong non-planar deformations. Extending the model to support more expressive transformation families remains direction for future work.
4 Results
Datasets, evaluation metrics, and baselines. We evaluate FastJAM under a test-time optimization setting, where the model is optimized independently on each image collection. We use two benchmark datasets: SPair-71k [9] and CUB-200 [67] (classes and subsets). SPair-7Ik' ' test set comprises 18 object categories, each with ~0 images, with annotated KPs and large intra-class variation. We report both per-category performance and average results across all categories. Following prior works, evaluate on the first 3 categories of CUB-200 test set, each containing ~0 images well. We use the Percentage of Correct Keypoints (PCK) as the evaluation metric. predicted KP is deemed correct if it falls within normalized distance threshold a of the ground-truth location. We report mean PCK across all KPs and categories, with a = 0.1. We average the results over 3 runs.
As in prior work [2, 3), we compare FastJAM to several baselines. Neural Best Buddies (NBB) [68] aligns image pairs using mutual nearest neighbors with Moving Least Squares warping [69], using VGG (VGG-MLS) or DINO (DINO-MLS) features. DINO-NN performs dense neighor matching. GANgealing [46] uses GANs but is limited to seen categories. Neural Congealing [1] builds explicit atlas but requires hyperparameter tuning and reported results only three SPair-7Ik classes. ASIC [2] predicts dense warps to a canonical space, and SpaceJAM [3] applies an IC loss over DINO features. FastJAM instead uses sparse KPs and graph-based model, enabling faster and more scalable alignment. We cite the results reported in [2, 3].
4.1 Qualitative Results
We illustrate the qualitative performance of FastJAM in Figure 1, Figure 4, and Figure 5. Given collection of category-level images (e.g., birds), FastJAM aligns all images within seconds, producing visually coherent and semantically consistent outputs across instances. To interpret the alignment,
8

---

Original
SpMceJAM
FastJAM (Ours)
Original
SpMceJAM
FastJAM (Ours)
Figure 5: JA Visual Comparison. We compare FastJAM with SpaceJAM [3] on both rigid (TV) and non-rigid (Sheep) classes. In both cases, FastJAM alignment is visually better, where the improvement particularly noticeable for close-up images, such as the middle or rightmost sheep.
Table 2: SPair-7Ik results: PCK @0.10 10 on the test set. Among test-time optimization (TTO) methods, the best is in bold, second- best is underlined. (x) Denotes use of reference image. Indicates missing results. (+) Marks non-TT TTO methods.
Method
Azto
Bise
Bind
Boa
Boule
Bus
Car
Cut
Chair
Cow
Dog
Horse
Moioe
Person
Plazz
Sheep
Trais
TV
All
GANgmalng (46)
37.5
67.0
23.1
$7.9
VOC-MIL [68]
70.5
22.7
61.9
26.
X16
25.
14.1
21.7
14.2
216
3000
20.1
34.7
20.4
19.1
19.3
24.4
23.6
3744
DINDMMLS (68, 70]
49.7
20.9
63.9
19.1
32.5
27.6
22.4
48.9
14.0
36.
36.0
30.1
21.7
41.1
17.1
18.1
35.9
21.4
31.1
DINDANN [7]]
57.2
24.1
67.4
24.5
26.8
29.3
27.1
52.1
15.7
424
43.3
30.1
23.2
40.7
16.6
24.1
31.0
24.9
35.0
Nestongeal
20.1
33.3
352
ASIC [2]
$7.9
25.2
68.1
24.7
35.4
28.4
30.9
54.8
21.6
45.0
47.3
39.9
26.2
48.8
14.5
24.5
49.0
24.6
37.0
SpaceAAM (VIT [3]
33.6
53.5
45.4
47.5
71.0
54.3
16.0
66.0
25.8
45.6
28.5
47.6
58.0
50.7
34.0
09.0
718
15.4
45.7
FastAMM (Durs)
64.4
43.3
60.0
29.5
25.5
66.8
56.5
63.7
33.0
49.2
80.8
53.7
62.8
89.1
42.9
33.4
76.2
71.2
53.0
visualize the canonical space C using fixed RGB colormap, Each image 1, is colored by applying the inverse of its predicted transformation to C. i.e., - As shown in Figure 4, semantically similar regions (e.g., heads, wings, tails) align to consistent areas in C, indicating robust JA across pose and appearance. As shown in Figure in comparison to [3], FastJAM alignment is visually better. Additional examples and comparisons are available in the SupMat.
4.2 Quantitative Results
Table 2 reports alignment accuracy on the SPair-71k test set, measured by mean PCK @0.10 across 18 object categories. FastJAM achieves the best overall performance with average PCK of 53.0, outperforming all competing methods. ranks first in 11 categories and second in 6 others, with comparable performance across wide range of object classes and viewpoints. Compared to [1, 2], FastJAM offers on- on-par or better accuracy while being significantly faster and more memory-effice efffii,en,,,,, validating the benefits of its sparse, graph-based formulation. While FastJAM performs competitively across most categories, we observe reduced performance on These cases pose inherent challenges due to visual ambiguity, where the initial matcher struggles to disambiguate symmetric parts (e.g., right versus left). In such scenarios, FastJAM can propagate incorrect correspondences. Addressing this remains an interesting direction for future work. Results for the CUB-200 dataset are reported in Table 3 where FastJAM outperforms SpaceJAM and achieves comparable results to ASIC [2] (in fraction of computation time) on the categories benchmark and the best results across subsets (results were not reported for [2]).
objects such as bicyeles.

---

Table 3: comparison on CUB-200
Table 5: Ablation Study.
CUB-200
Method
CUB-300 (Subsets)
CUR-2200
SPai--1K
(fiss 3 catc.)
(frst 3 conc.))
VOG+MLS [68)
25,8
LoFTR R [12] (No RoMa)
33.8
17.0
DINO+ME S 68)
67.0
DINOANN
68.3
GANgealing
56.8
Lincar projection
33.2
14.0
ASICTE]
NeuCougeal [3]
MLP (per image, no graph)
73.5
47.8
SpineJAM [3]
69.6
SpocelAM [3]
69.9
Homograghy optimizttion deep
74.1
FisZAM (Qurs)
75.
FastAA (Chrs))
73.6
L2 loss Geman: McClure function)
61.0
33,9
No noe-maxinum suppression
74.3
47.2
Table 4: Dense vs. KP warping runtime analysis [sec].
No Lie Group
75.0
49.3
No image edges
74.7
50.0
Loss
Bra
Nysiss
D Gaid warging Interpolation
Pad. 4Book
Total
GNN Backhomes
Dense
10
7U736
1.28
0.13
2.82
8.46
Dense
10
70736
25
1.29
0.77
4.12
12.36
GCN [72] (53 sees)
73.2
47.4
16
0.36
GKT [62] (66
75.0
25
0.34
GlspSAAGE [61] (49 secs)
75.3
4.3 Runtime Analysis
Runtime comparison. We evaluate the computational efficiency of FasUAM by comparing its runtime against: NeuCongeal [1]: ASIC [2]: SpaceJAM [3]. As shown in Table FastJAM achieves over order-of-magnituue speedup (measured over three SPair-71k categories), aligning image collections in under ~0 seconds, compared 5-6 minutes for SpaceJAM and over an hour for ASIC and NeuCongeal. The reported runtime includes preprocessing (i.e., pairwise matches). We also compare FastJAM and SpaceJAM on an increasing number of images (N 10 to 100). The full experimental setup is available in our SupMat.
of 30 images (see Table 4, all reported runtime in this table are in seconds) for one epoch. We evaluate how the number of points (N pointts in the coordinate grid and the feature dimension (D) affect the overall warping time. For dense matching, we set N points = 266 x 266 70,756 (the image resolution used SpaceJAM) and D 25, corresponding is computed. For FastJAM, we set Npoiits = 16 and D = 2 (i.e., 8 KPs in 2D). For completeness, we also evaluate D E {2, 25) for both methods. We measure the time required for (i) grid warping. (ii) interpolation (used only in dense warping). (iii) forward and backward warping for the IC loss (Fwd Back), and the total runtime over all batches in a single epoch. For dense warping (e.g., SpaceJAM), the maximum batch size is Bmax = 10, resulting in a total runtime of 3 x (Fwd+Back). The key observations are: (1) reducing Npoints crucial for achieving fast warping, and (2) avoiding interpolation further accelerates computation, making the runtime largely independent of D.
Dense KP warping. We analyze the individual warping components of both methods
the feature dimension on which the loss
4.4 Ablation Study
Table 5 summarizes our ablation study on CUB-200 categories) and SPair-71k. Replacing RoMa with LoFTR [12] significantly reduces performance, underscoring LoFTR's limitations in cross- instance correspondence. Substituting the Geman-MeClure loss [65] with an &, loss or removing the object mask also causes notable accuracy drops, confirming the importance of robust error modeling and spatial masking. NMS and intra-image edges provide additional gains. Although removing the Lie-algebraic parameterization has little effect accuracy, it ensures warp invertibility, essential for the IC loss, as without about 2% of runs fail due to non-invertible matrices, whereas using it eliminates such failures entirely. Replacing the GNN with a linear projection caused substantial performance drop, while MLP-based and direct homography optimization models also reduced accuracy, though less severely. Among GNN backbones, GraphSAGE [61] outperforms both GCN [72] and GAT [62] in accuracy and runtime.
Conclusion
We introduced FastJAM, graph-based framework for fast and scalable image JA. By leveraging sparse KP correspondences and a lightweight GNN architecture, FastJAM propagates alignment cues across image collections and regresses per-image transformations, Our method achieves state-of-the- art alignment quality while significantly reducing runtime and memory usage.
10

---

Acknowledgments
This work was supported by the Lynn and William Frankel Center BGU CS, by the Israeli Council for Higher Education via the BGU Data Science Research Center, RSW's work was supported by the Kreitman School of Advanced Graduate Studies. Both OH and SI were also supported by the VATAT National excellence scholarship for MSc students in Al and Data Science.
References
[1] Dolev Ofri-Amar, Michal Geyer, Yoni Kasten, and Tali Dekel. Neural congealing: Aligning images to a joint semantic atlas. In CVPR, 2023. 2, 3, 4, 5, 6, 8, 9, 10
[2] Kamal Gupta, Varun Jampani. Carlos Esteves, Abhinav Shrivastava, Ameesh Makadia, Noah Snavely, and Abhishek Kar. ASIC: Aligning sparse in-the-wild image collections. In ICCV, 2023. 1, 2, 3, 4, 5, 6, 8, 10
[3] Nir Barel, Ron Shapira Weber, Nir Mualem, Shahaf E Finder, and Oren Freifeld. SpaceJAM: a lightweight and regularization-free method for fast joint alignment of images. In European Conference on Computer Vision, pages 180-197. Springer, 2024. 3, 4, 5, 6, 7, 8, 9, 10
[4] Erik G Miller, Nicholas E Matsakis, and Paul A Viola. Learning from one example through shared densities on transforms. In CVPR. IEEE, 2000. 2, 5
[5] Erik G Leamed-Miller. Data driven image models through continuous joint alignment. IEEE TPAMI, 2006. 2, 4,
[6] David G Lowe. Object recognition from local scale-invariant features. In ICCV. IEEE, 1999. 2
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 0rXiv:2010.11922 2020.
[8] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
4, 6
[9] Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Spair 71k: A large-scale benchmark for semantic correspondence. arXiv preprint arXiv: 10543, 2019. 3, 8
[10] Daniel DeTone, Tomasz Malisiewiez, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference computer vision and pattern recognition workshops, pages 224-236, 2018.
[11] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewiez, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 4938 4947, 2020. 3, 4
[12] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8922-8931 2021. 3, 10
[13] Johan Edstedt, Qiyu Sun, Georg Bökman, Märten Wadenbäck, and Michael Felsberg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19790-19800, 2024. 3, 6,
[14] Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emer- gent correspondence from image diffusion. Advances in Neural Information Processing Systems, 36:1363-1389 2023. 3
[15] Octave Mariotti, Oisin Mac Aodha, and Hakan Bilen. Improving semantic correspondence with newpointtguidee spherical maps. In Proceedings of the IFFF/CVF Conference on Computer Vision and Pattern Recognition, pages 19521-19530, 2024. 3

---

[16] Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming- Hsuan Yang. Telling left from right: Identifying geometry- semantic correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3076-3085, 2024.
[17] Mark Cox, Sridha Sridharan, Simon Lucey, and Jeffrey Cohn. Least squares congealing for unsupervised alignment of images. In CVPR, 2008. 4,
[18] Mark Cox, Sridha Sridharan, Simon Lucey, and Jeffrey Cohn. Least-squares congealing for large numbers of images. In ICCV. IEEE, 2009. 4, 5
[19] Gary B Huang, Vidit Jain, and Erik Learned-Ne Milleer. Unsupervised joint alignment of complex images. In ICCV. IEEE, 2007.
[20] Wen- Yan Lin, Linlin Liu, Yasuyuki Matsushita, Kok-Lim Low, and Siying Liu. Aligning images in the wild. In CVPR. IEEE, 2012. 4
[21] Fatemeh Shokrollahi Yancheshmeh, Ke Chen, and Joni-Kristios Kamarainen. Unsupervised visual alignment with similarity graphs. In CVPR, 2015. 4
[22] Xiaoming Liu, Yan Tong, and Frederick W Wheeler. Simultaneous alignment and clustering for
image ensemble. In ICCV. IEEE, 2009. 4
[23] Marwan A Mattar, Allen R Hanson, and Erik G Leamed--Milee Unsupervised joint alignment and clustering using bayesian nonparametrics. arXiv preprint arXtv:/210.4892,, 2012. 4
[24] Anil K. Jain, Yu Zhong, and Sridhar Lakshmanan. Object matching using deformable templates. IEEE TPAMI, 1996. 4
[25] Dariu M Gavrila. Multi-feature hierarchical template matching using distance transforms. In
ICPR. IEEE, 1998. 4
[26] Pedro Felzenszwalb and Joshua D Schwartz. Hierarchical matching of deformable shapes. In CVPR, pages 1-8. IEEE, 2007. 4
[27] Ira Seseelmachhe-SShizeemeee and Steven Seitz. Collection flow. In CVPR. IEEE, 2012.
[28] Jun He, Dejiao Zhang, Laura Balzano, and Tao Tao. Iterative grassmannian optimization for robust image alignment. Image and Vision Computing. 2014. 4
[29] Xiaoqin Zhang, Di Wang, Zhengyuan Zhou, and Yi Ma. Robust low-rank tensor recovery with rectification and alignment. IEEE TPAMI, 2019.
[30] Yigang Peng, Arvind Ganesh, John Wright, Wenli Xu, and Yi Ma. Rasl: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE TPAMI, 2012.
[31] Gary Huang, Marwan Mattar, Honglak Lee, and Erik Learned-Miller. Learning to align from scratch. In NeurIPS, 2012. 4
[32] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In NeurlPS, 2015. 4
[33] Roberto Annunziata, Christos Sagonas, and Jacques Cali. Jointly aligning millions of images with deep penalised reconstruction congealing. In ICCV, 2019.
[34] Chen-Hsuan Lin and Simon Lucey. Inverse compositional spatial transformer networks. In
CVPR, 2017. 4
[35] Adrian Dalca, Marianne Rakic, John Guttag, and Mert Sabuncu. Learning conditional de- formable templates with convolutional networks. NeurIPS, 2019.
[36] Ron Shapira Weber, Matan Fyal, Nicki Skafte Detlefsen, Oren Shriki, and Oren Freifeld. Diffeomorphic temporal alignment nets. In NeurIPS, 2019. 4
12

---

[37] Matthew Sinclair, Andreas Schuh, Karl Hahn, Kersten Petersen, Ying Bai, James Batten, Michiel Schaap, and Ben Glocker. Atlas-istn: joint segmentation, registration and atlas construction with image-and-spatial transformer networks. Medical Image Analysis, 2022. 4
[38] Ron Shapira Weber and Oren Freifeld. Regularization-free diffeomorphic temporal alignment
nets. In ICML. PMLR, 2023. 5,5
[39] Tom Monnier, Thibault Groueix, and Mathieu Aubry. Deep cuansformationninvanne clustering. NeurIPS, 2020. 4
[40] Romain Loiseau, Tom Monnier, Mathieu Aubry, and Loic Landrieu. Representing shape collections with alignment-aware linear models. In 3DV. IEEE, 2021. 4
[41] Irit Chelly, Vlad Winter, Dor Litvak, David Rosen, and Oren Freifeld. JA-POLS: a moving- camera background model via joint alignment and partially-overlappinng local subspaces. In CVPR, 2020. 4
[42] Guy Erez, Ron Shapira Weber, and Oren Freifeld. A deep moving-cames background model. In ECCV. Springer, 2022.
[43] Avihai Naaman, Ron Shapira Weber, and Oren Freifeld. Synchronization of multiple videos. In ICCV, 2025. 4,
[44] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 2014.
[45] Jiteng Mu, Shalini De Mello, Zhiding Yu, Nuno Vasconcelos, Xiaolong Wang, Jan Kautz, and Sifei Liu, Coordgan: Self-supervised dense correspondences emerge from gans. In CVPR, 2022.
4
[46] William Peebles, Jun- Yan Zhu, Richard Zhang, Antonio Torralba, Alexei A Efros, and Eli Shechtman. Gan-supervised dense visual alignment. CVPR, 2022. 4, 8, 9, 10
[47] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ECCV. Springer, 2016.
[48] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. MICCAL Springer, 2015.
[49] Morteza Safdameejd, Yousef Atoum, and Xiaoming Liu. Temporally robust global motion
compensation by keypoint- based congealing. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14, pages 101-119. Springer, 2016. 4
[50] Tinghui Zhou, Yong Jae Lee, Stella Yu, and Alyosha Efros. Flowweb: Joint image set alignment by weaving consistent, pixel-wise correspondences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1191-1200, 2015.
[51] Philipp Lindenberger, Paul-Edouard Sarlin, and Mare Pollefeys. Lightglue: Local feature matching at light speed. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 17627-17638, 2023.
[52] Christiano Gava, Vishal Mukunda, Tewodros Habtegebrial, Federico Raue, Sebastian Palacio, and Andreas Dengel. Sphereglue: Learning keypoint matching on high resolution spherical im- ages. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 61 34-44444, 2023. 4
[53] David M Rosen, Luca Carlone, Afonso Bandeira, and John J Leonard. SE-Syne: A certifiably correct algorithm for synchronization over the special Euclidean group. The International Journal of Robotics Research, 2019. 5
[54] Federica Arrigoni and Andrea Fusiello. Synchronization problems in computer vision with closed-form solutions. International Journal of Computer Vision, 2020. 5, 7
13

---

[55] X. Pennec. Probabilities and statistics on Riemannian manifolds: Basic tools for geometric measurements. In NSIP. pages 194-198, 1999. 6
[56] Hermann Karcher. Riemannian center of mass and so called karcher mean. arXiv preprint arXiv: 2014. 6
[57] Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng. Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024. 6, 8
[58] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 6
[59] Or Dinari and Oren Freifeld. Revisiting DP-means: fast scalable algorithms via parallelism and delayed cluster creation. In Uncertainty in Artificial Intelligence. PMLR, 2022.
[60] Brian Kulis and Michael Jordan. Revisiting k-means: New algorithms via bayesian nonpara- metrics. arXiv preprint arXiv:1111.0333 2011.
[61] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017. 7, 10
[62] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, al. Graph attention networks. stat, 1050(20):10-4533 485500 2017. 7, 10
[63] Nicki Skafte Detlefsen, Oren Freifeld, and Søren Hauberg. Deep diffeomorphic transformer networks. In CVPR, 2018.7 7
[64] Christopher Mei, Selim Benhimane, Ezio Malis, and Patrick Rives. Homography based tracking for central catadioptric cameras. In 2006 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 669-674. IEEE, 2006. 7
[65] Stuart Geman and Donald E McClure. Statistical methods for tomographic image reconstruction. In BISI, 1987. 8, 10
[66] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, 2014.
8
[67] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd 2irds-200-2011 dataset, 2011. 8
[68] Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen, and Daniel Cohen-Or. Neural best-buddics: Sparse cross-domain correspondence. ACM TOG, 2018. 8, 9, 10
[69]
Scott Schaefer, Travis McPhail, and Joe Warren. Image deformation using moving least squares. In ACM SIGGRAPH 2006 Papers, pages 533-540. ACM, 2006.
[70] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. NeurIPS, 2020. 9, 10
[71] Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. In ECCV Workshops, 2022. 9, 10
[72] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2017. 10
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
All the claims made in the abstract are backed in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In section 3.4, we elaborate about the limitations FastJAM.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
there are no theoretical results in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Most if not all needed information about reproducibility given either in the paper itself, in the Supplementary in the future to be published code.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: As described the abstract, the code would be released upon acceptance. The datasets we used are publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https: ctsps.cc//uullic/uu gwwgg//ies////// /Cooeeubeiiiieeee wwwiiisssoooooiiiioo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We describe all the details either in the paper itself or in the supplementary.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification; Due to the substantial computational cost associated with running all competing methods, it was not feasible to perform multiple trials necessary for estimating variance or reporting error bars. On our results, we averaged our experiments on 3 different seed runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
show in tables
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The experiments where maybe on internal cluster, GPU RTX4090, mention it where needed. Memory size is also indicated.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: cttprrpp....ppppi//////ccccaaaaa
Answer: [Yes]
Justification:
All the data we used is publicly available.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification:
The method presented a generic tool for aligning images and is not directly tied to specific application. While it may enable both beneficial and potentially harmful uses (e.g., in surveillance or misinformation), the paper does not explore these societal implications.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We did not found high risk misuse for our method.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Any credit or citation needed was provided in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi twwode..ooo..... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
All needed documentation would be provided with the code upon acceptance.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: the paper did not involve crowd-sourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowd-sourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The usage of LLM, as declared, was mainly for writing and editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: http:///eeuu.peeeeeee cc/ caarrrppsscc/cooffeeeneeee/////s for what should or should not be described.
21

---

FastJAM: a Fast Joint Alignment Model for Images
Supplemental Material
Contents
This document contains the following:
A: Visual Comparison
Additional qualitative comparisons against existing joint alignment methods (Figure 6).
B: Additional Visualizations
Additional joint alignment results on the CUB dataset (Figure 7).
Full pairwise alignment grids for representative categories (Figure and Figure 9)
C:
Explaining the Colormap Visualization in More Detail
Illustration FastJAM's canonical alignment using fixed RGB colormap (Figure 10).
D: Runtime Analysis
Scalability runtime assessment for increasing number of images (N = 10 to 100)
(Figure 11).
$ E: Model Configuration and Training Setup
Full architecture summary (Table 5).
Optimization settings, matcher configuration, and training procedure.
$ F: External Tools and Frameworks
Summary of third-party tools and libraries used, including RoMa, LoFTR, Grounded- SAM, and torch-geometric..
G: Lie Group Parameterization
Full explanation of the Lie group parameterization.
Additionally, key notational conventions used throughout the main paper and this document are summarized in Table
A Additional Visual Comparisons
To qualitatively assess alignment quality, we compare FastJAM against existing joint alignment methods, including SpaceJAM and ASIC. As illustrated in Figure 6, FastJAM produces more coherent and natural-looking alignments, particularly in challenging cases involving pose variation. Unlike ASIC, which applies dense warping and often introduces distortions. FastJAM preserves global structure by relying on sparse keypoints and homographic transformations.
Additional Visualizations
This section provides supplementary qualitative results that further demonstrate the alignment capa- bilities of FastJAM across various categories and settings.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Source
Target
ASIC
SpaceJAM
FastJAM (Ours)
Figure 6: Qualitative Comparison of Pairwise Alignment Methods. Visual comparison of joint alignment results across ASIC [1]. SpaceJAM [2], and FastJAM (ours) several image pairs. ASIC, which relies on dense warping fields, often introduces spatial distortions and unrealistic deformations in low-texture structured regions. In contrast, SpaceJAM and FastJAM apply global homographies, resulting in more coherent transformations. Notably, in the top row (bird), FastJAM produces the most geometrically consistent alignment, evidenced by the parallel lines and precise correspondence of semantically meaningful points such as the beak and tail.
B.1 Additional Joint Alignment CUB Dataset
Figure 7 presents further joint alignment results on additional classes from the CUB-200 dataset. For each class, the original input images are shown in the top row, while the bottom row displays their aligned counterparts. The outputs demonstrate FastJAM's ability to handle fine-grained categories and produce visually coherent canonical views across varying poses and appearances.
B.2 Pairwise Alignment
We visualize full pairwise alignment grids for two representative categories: "aeroplane" and "horse." As shown in Figure 8 and Figure 9, each grid displays how the source images (rows) are aligned to the target images (columns) using the estimated inverse-compositional warps. The diagonal entries, which correspond to self-alignment (i.e., identity transformations), are highlighted with purple dashed-ling frame. These grids illustrate FastJAM's ability to produce consistent, symmetric mappings across image pairs and maintain semantic structure throughout the alignment process.

---

CUB class
CUB class
Aligned
CUB class 2
CUB class 2 Aligned
CUB class
CUB class 3 Aligned
Figure 7: Qualitative alignment results on CUB-200 classes. Each pair of rows corresponds to distinct semantic class from the CUB-200 [3] dataset. The top row in each pair shows the original, unaligned images; the bottom row shows the corresponding aligned images produced by FastJAM. The alignment process successfully maps semantically consistent parts (e.g., heads, wings, tails) to similar spatial locations across different instances within each class.
3

---

Image
Image
Image
Image
Image
Input images
RYAAN
Image! I
Target
Image? 2
Painwise Aligned
Image 3
Image 4
Image 5
Figure 8: Pairwise Alignment Grid "Aeroplane" Class. Top row: five input images from the "aeroplane" category. Below: a 5 x 5 alignment matrix, where each entry in column j displays the corresponding source image from row i, warped align with target image j using the inverse- compositional transformation 7 7-8, Each row thus visualizes the same source image aligned to five different targets. Diagonal entries show the self-warped images (i.c., identity transformation), but with the canonical background rather than the original, and are marked with red dashed-line frame. This layout highlights FastJAM's ability to achieve coherent, semantically meaningful alignments across all image pairs.

---

Image
Image 2
Image 3
Image
Image
Input images
Image 1
Torget
Image? 2
Painwise Aligned
Image 3
Image 4
Image 5
Figure 9: Pairwise Alignment Grid "Horse" Class. Top row: five input images from the "horse" category. Below: 5 grid showing the full pairwise alignment structure. Each image in row column j corresponds to the source image i aligned to the target image using 7h o T= The diagonal entries depict self- welf-warping (identity), rendered with the canonical background rather than the original, and are marked with red dashed-line frame. This visualization reveals consistent alignment behavior across the set, illustrating how FastJAM handles pose and appearance variation within a semantic class.
5

---

C Explaining the Colormap Visualization in More Detail
To provide an intuitive understanding of how FastJMM aligns images shared canonical space, we visualize the warped outputs using a predefined RGB colormap. As shown in Figure 10, each input image is first aligned the canonical frame and blended with the colormap. The result is then warped back to the original image space, allowing us to visualize how semantic regions are mapped consistently across instances. This process highlights FastJAM's ability establish meaningful correspondences without relying on dense features or explicit templates.
Aligned Image Colored
Original Image Colored
Original Image
Aligned Image
Figure 10: Illustrated Canonical Space Visualization. This figure provides an intuitive explanation of how FastJAM visualizes the canonical space C using a predefined RGB colormap coooormp-- For each input image I; (left column),
first apply the estimated homography to obtain its aligned version canonical space: I, o T We then blend this aligned image with the colormap via averaging: (Joolormap + (1; o ))))). Finally, apply the inverse warp visualize the blended canonical signal in the original image frame: [ (Taolommup (T, T8))) T- This provides visual explanation of how semantically similar regions across instances are mapped to consistent spatial locations.

---

D Runtime Analysis
Pa
SpacejA01
200
Number of Images
Figure 11: Runtime analysis between SpaceJAM and FastJAM over increasing number of images. SpaceJAM PCA processing step runs out RAM after 40 images.
As illustrated in Figure 11, FasJAAM remains efficient across all tested sizes, completing alignment in under 450 seconds even for 100 images. In contrast, the current implementation of SpaceJAM runs out of RAM after N images due the PCA preprocessing step.
E
Model Configuration and Training Setup
We train the model for 600 epochs using the Adam optimizer with an initial learning rate of 5 x 10 multiplied by 0.5 after 200 epochs without improvement. The loss function is based the Geman- McClure formulation with robustness parameter a = 0.25, and no weight decay is applied. The feature extractor within the GNN uses 5 layers of hidden size 128, followed by linear projection to an 8-dimensional homography parameter vector. full summary of the model architecture and its 133,256 trainable parameters provided in Table 5. To encourage geometric stability, the final projection layer is initialized to approximate the identity transformation, and transformations are parameterized using Lie algebra to ensure invertibility. Alignment is applied iteratively using single pass of the inverse compositional (IC) spatial transformer network. During optimization, horizontal flips are checked every 100 epochs.
Table 5: GraphSAGE GNN Model Summary.
Layer (type:name)
Output Shape
Param
GraphSAGE GNN
convs.0.lin tteigh
[128,
256
Convs.O.li__l.bite
[128]
128
convs.0.lin
wwwssO.lin_rweighh
[128, 21
256
convs.1.lin I.eigh
[128. 128]
16,384
convs.1.lin_Iee
[128]
128
convs.l.lin_r.weet
[128, 128]
16,384
convs.2.lin lonls.
[128, 128]
16,384
convs.2.lin_I.a.
[128]
128
convs.2.lin rweight
[128, 128]
16,384
convs.3.lin Lweight
[128, 128]
16,384
convs.3.lin 1biis
[128]
128
convs.3.lin_rwwee
[128, 128]
16,384
convs.4.lin_l.weer
[128, 128]
16,384
convs.4.lin_I.bb
[128]
128
convs. convs...ii__.weight
[128, 128)
16,384
fc. feigeight
[8, 128]
1,024
fc.bias
[8]
Total
133,256
For correspondence estimation, we employ the RoMa matcher at fixed image resolution of 560 x 560 for both coarse and upsampled stages. A maximum of 10 keypoints is retained per image, filtered by
7

---

ma--max suppression (NMS) using radius of 0.054 in normalized coordinates wordinates-corresooooeee corresponding 30 30 pixel window the original image space. This ensures spatial coverage while avoiding redundant detections.
Table 6: Summary of Notation
Symbol
Description
I (T
Set of input images
M.
Number of keypoints in image 1;
X, (2 ...., (Ma))
Keypoints image 4,, s; (m)) [-1.
Mij
Set of matched keypoints between 1, and 1, Family of parametric transformations (e.g.. homographies) Transformation for image 4,, parameterized by 0:
T
1 ET
9= (V,8)
Keypoint graph with intra- and inter-image edges
((9)
GNN- based function that predicts {0. {0.).-, from graph
1010
Image I, warped by transformation 7
External Tools and Frameworks
We gratefully acknowledge the use of several open-source libraries and resources in this project. Our GNN implementation based on forch.geometricc primarily using the GraphSAGE [4] architecture, along with other variants for comparison. We used Weights & Biases for experiment tracking and visualization. For keypoint matching. we built upon the official implementation of RoMa [5], and also incorporated components from the LoFTR framework [6]. Object-centric masks were obtained using Grounded-SAM, which combines Grounding DINO [7, 8] and the Segment Anything Model (SAM) [9]. We thank the authors of all these works for making their code and models publicly available.
G Lie Group Parameterization
homography has 8 degrees of freedom and corresponds an equivalence class of invertible matrices, where representative with unit determinant can be used. Now consider
s((3) e
=
04 06
01
OG +8))
and SL(3) = H 33x3 det H 1
(I)
#7
The space s3) is the Lie algebra of trace-zero matrices. The matrix exponential maps s((3) into (3),), yielding smooth eight-parameter representation of homographies. Our network predicts (0, (0:), where each 0 E R is mapped to θ E s((3) as shown above, and the homography is obtained via 7 H; exp(O)). This construction guarantees det T 1. In particular, 0, 08x1 yields the identity matrix, and T-8; is the inverse of T

---

References
[1] Kamal Gupta, Varun Jampani, Carlos Esteves, Abhinav Shrivastava, Ameesh Makadia, Noah Snavely, and Abhishek Kar. ASIC: Aligning sparse in-the-wild image collections. In ICCV, 2023.
[2] Nir Barel, Ron Shapira Weber, Nir Mualem, Shahaf E. Finder, and Oren Freifeld. Spacejam: lightweight and regularization-free method for fast joint alignment of images. In European Conference on Computer Vision, pages 180-197. Springer, 2024. 2
[3] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset, 2011.
[4] Will Hamilton, Zhitao Ying, and Jure Leskovee. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017. 8
[5] Johan Edstedt, Qiyu Sun, Georg Bökman, Märten Wadenbäck, and Michael Felsberg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19790-19800, 2024.
[6] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector- free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8922-8931, 2021.
[7] Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang. Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024. 8
[8] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li. Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.08 05499, 2023.
[9] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo. Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.022 026443, 2023.


Reviews:
[{"review_id": "yZv7wcGFsu", "reviewer": "Reviewer_xNrY", "summary": "The paper - FastJam - introduces a rapid, graph-based technique for joint image alignment that relies on pairwise matches and a graph neural network to swiftly predict homography parameters. It surpasses existing methods in alignment quality while drastically cutting down computation time from hours or minutes to seconds. It builds upon extensive prior work in the field, drawing inspiration from early techniques like congealing and least-squares congealing, especially integrating ideas from inverse-composition. The approach achieves state-of-the-art results in joint alignment, with a tenfold speed increase in computation time compared to current leading methods.", "questions": "Looking at the results in Table 2, there definitely seems to be categories where SpaceJam significantly outperforms FastJam. Some insights into why some categories do well, and others do not would be useful to better understand the utility of the approach? The authors offer up bicycles as a problem category. Is there a way to know ahead of time which categories will struggle with FastJam?\nAnother question for the authors is what practical applications are now possible thanks to the considerable speedup of FastJam over SpaceJam? Discussion or event some results on this question would be great.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The technical novelty of the approach seems low, as it heavily relies upon ideas from SpaceJam [3] with the exception that they rely on key points (KPs) rather than dense features (e.g. DINO). Further, the approach is dependent upon an external image matcher for the initial correspondences. The big advantage over SpaceJam seems to be the computational speed, which is considerable. This empirical result seems to be the most impressive part of the paper.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough comments.\n\n---\n\n### **Technical novelty and relation to SpaceJAM (Barel et al., ECCV 2024)**\n \nWhile FastJAM relates to SpaceJAM (in particular in the use of inverse compositional framework), it introduces **key architectural and conceptual differences** that make it a novel and distinct method.\n\n- **Joint Forward Pass over All Images:** In FastJAM, the model receives **all keypoints from all images simultaneously** in a graph structure. This allows the GNN to perform **joint reasoning process during the forward pass**, enabling global context to directly influence alignment.  In other words, FastJAM leverages all images together in not only its (keypoint-based) loss function but also in its input. \n  In contrast, **SpaceJAM's input is a single image**. As such, it must rely entirely on its (dense-feature based) loss function to capture multi-image relationships. \nThis architectural difference is illustrated in **Figure 3**.\n\n- **Sparse vs. Dense Features:** FastJAM takes **sparse keypoints as input**, whereas SpaceJAM relies on dense DINO features. This transition changes the data representation from image-like tensors processed by CNNs to a graph of keypoints processed by a GNN. This difference enables a **significant reduction in runtime and RAM usage**.\n\n---\n\n### **Dependency on external keypoint matchers**\n\nThat is correct. Our pipeline uses a **keypoint matcher as a preprocessing step**. However, we view this as a **strength** rather than a limitation. The modular design allows FastJAM to **benefit from future advances** in keypoint detection and matching (e.g., replacing RoMa (Edstedt et al., CVPR 2024) with a newer and better matcher) without modifying the FastJAM's alignment framework.\n\n---\n\n### **Performance variation across categories**\n\nWe agree that performance varies across categories and appreciate the opportunity to discuss this behavior.\n\n**Compared with SpaceJAM**:  there are some categories that challenge the keypoint matcher, occasionally leading to subtle mismatches between similar parts (e.g., front and rear wheels), which may affect overall quality. Thus, when the matcher fails, minimizing the loss of the entire image (e.g., SpaceJAM) yields better results. \n\n**In  general**: Our method (as does SpaceJAM) models the transformation between images using **homographies**. Consequently, categories where the image variation can be **well captured by a global transformation**. For example, objects like **cars or buses**, which have relatively rigid, consistent shapes across instances, often conform well to homographic warping.\n\nOn the other hand, categories like **sheep or bicycles**, which exhibit high intra-class variation and frequent occlusions, are less suitable for accurate alignment using a homography. In such cases, the mismatch between the real deformation and the homography model leads to lower performance.\n\nThis behavior is a natural limitation of using homographies to represent inter-image transformations, and we believe the observed variation is consistent with this modeling assumption.\n\n---\n\n### **Practical applications enabled by FastJAM's speed**\n\nFastJAM’s remarkable speed opens the door to real-time and scalable applications:\n\n- **Video and image editing**: As in Neural Congealing (Ofri-Amar et al., CVPR 2023) and ASIC (Gupta et al., ICCV 2023 Oral), solving joint alignment once enables editing in canonical space and transferring the result to all input images. With FastJAM, this becomes **interactive** or feasible at larger **video scale**.\n- **Large-scale pipelines**: The ability to process larger sets of images in seconds allows FastJAM to be embedded in **auto-labeling**.\n\n| Application                 | Feasibility with previous methods | Feasibility with FastJAM |\n|----------------------------|-----------------------------------|---------------------------|\n| Interactive editing        | ✗                                 | ✓                         |\n| Video alignment            | ✗                                 | ✓                         |\n| Large-scale pipelines      | ✗                                 | ✓                         |\n\nImportantly, FastJAM is **not only faster** but also delivers **better alignment accuracy**. As shown in Table 2, it outperforms all existing methods on the **SPair-71k** dataset with a **~5% improvement** in average PCK (50.4 vs. 45.7) over SpaceJAM (Barel et al., ECCV 2024). On **CUB-200**, it achieves results comparable to ASIC (Gupta et al., ICCV 2023 Oral) while being significantly more efficient.  \nIn addition, we would wish to refer to the expansion of the ablation study as suggested by *SQ4g*."}, {"review_id": "EgHnhkuPf5", "reviewer": "Reviewer_aDpz", "summary": "The paper considers the joint alignment problem, which is given a collection of images find consistent warps between them. These images might contain objects of the same class, but different instances (e.g. images of different cats), and in different poses. The authors propose to model the warps using homographies and in particular each image is associated with a homography going from the image to a shared canonical space. Following the Inverse Compositional (IC) approach, the relative transformations can then be found by composing a homography for an image with the inverse from another image. Between each pair of images a dense feature matcher is used to find pairwise correspondences which are used to constrain the relative transformations. The homographies themselves are predicted by a GNN that operates on the keypoints in each image, and is trained/fit independently to each image collection. The method is compared to a couple of other alignment approaches and performs well, in particular it is significantly faster compared to previous work.", "questions": "1. Did you try optimizing the homographies directly, i.e. without the GNN? If so, how did it work?\n\n2. For the homographies there is a gauge freedom where you can change \"coordinate system\" in the canonical space. Did you try regularizing or fixing this? Does it ever happen that the canonical space is very skewed/distorted?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Overall\nOverall I like the general setup of splitting the problem into predicting pairwise correspondences, followed by a more global optimization that figures out the warps jointly. I am not an expert in the area, but the experiments seem reasonable and show that the proposed approach achieves similar or better accuracy at much lower runtime.\n\n# Presentation and clarity\nI found the paper easy to read and believe I have a good understanding of how it works. Though some details might be missing, the paper could probably mostly be reproduced.\n\n# Using RoMa matches\nThe paper use RoMa (CVPR'24) for finding pairwise matches between the images. I was a bit surprised that this works, considering RoMa was not specifically trained for this kind of cross-instance matching (e.g. matching different birds etc.), but rather match the same scene under different conditions (viewpoint changes etc.). It would be nice to have some more discussion about this. Though I guess the DINOv2 back-end in RoMa might allow for some more semantic correspondences. \n\n# Missing ablation motivating the GNN\nAs I understood the paper, there is a test-time trained GNN which predicts the homographies. This is trained by minimizing the transfer error between images. It seems that natural baseline here would be to directly optimize for the homographies, i.e. only having $\\theta_1,\\dots,\\theta_n \\in \\mathbb{R}^8$ as the optimizable parameters. Showing that this does not work would motivate the complexity of the GNN.", "was_revised": true, "final_justification": "The authors provided good answers for the concerns raised by the reviewers. The additional ablation study showed that the method only improves slightly over directly optimizing the homographies. This makes the method a little bit less exciting as this small improvement comes at a large compute and complexity cost. Still, I am positive towards the method and the overall approach using sparse matches + IC seems to be working well for the problem.", "rebuttal": "We thank the reviewer for their thorough comments.\n\n---\n\n### **Use of RoMa for cross-instance matching**\n\nRegarding RoMa's cross-instance matching: \nYou raise an excellent point,  and we appreciate the opportunity to elaborate on it. While RoMa was originally designed for wide-baseline matching within the same scene, its use of DINOv2 as a feature backbone making it aware of semantic matching. This makes it fairly robust for cross-instance matching, such as aligning different birds, cars, or other object categories.\n\nWe found empirically that RoMa (Edstedt et al., CVPR 2024) consistently produced higher-quality correspondences across instances compared to several other off-the-shelf matchers. These observations led us to choose RoMa as the default matcher.\n\nTo better illustrate this, we hoped to include here pairs of images of different instances with keypoints matching by RoMa but according to this year's NeurIPS rules we cannot share those.\n\n---\n\n### **Motivation for using a GNN vs. directly optimizing homographies**\n \nWe agree, this is a great point. \nIn short, even though in theory optimizing the thetas directly could achieve similar results, in practice this doesn't happen as the optimization problem becomes (much) easier when using a GNN. Concretely, we ran an experiment where we directly optimized the homography parameters $\\{\\theta_i\\}$ (using Adam) with no GNN (or any other deep net) involved. That approach led to poorer results (in comparison to using a GNN) and does not allow for generalization to new images. Following reviewer *SQ4g*'s suggestion, we have added additional conditions (including the direct optimization of the thetas) to the ablation study, presented below. \n\nThe GNN, on the other hand, provides an inductive bias by enabling information propagation across keypoints and images, helping the optimization to escape poor local solutions. That is also one of the reasons FastJAM quantitative results are more effective: as shown in Table 2, FastJAM achieves ~5% higher accuracy than the previous best method (SpaceJAM, Barel et al., ECCV 2024).\n\nWe will include a quantitative comparison of this baseline in the updated version.\n\nAdditional ablations: \n- **Linear projection baseline**: Instead of a GNN, we use a single linear projection, then per-image pooling.\n- **MLP per-image baseline**: We use an MLP that receives keypoints from one image at a time, without graph connectivity.\n- **Direct optimization of homographies**: We optimize $\\{\\theta_i\\}$ directly without any learnable model.\n\nWe also added a **Vertical Flip** study.  \n  \n| Variant                               | PCK (CUB-200 first 3 cate.) | PCK (SPair-71K)|\n|---------------------------------------|------------------|-----------------------|\n| Linear projection                     | 33.2              | 14.0                   |\n| FastJAM with Vertical Flips                  | 56.9             | 44.1                   |\n| MLP (per-image, no graph)             | 73.5              | 47.8                   |\n| Direct homography optimization        | 74.1              | 48.3                   |\n| FastJAM (ours, full)                  | 75.3             | 50.4        |\n\n---\n\n### **Gauge freedom in homographies and canonical space distortion**\n\nThis is an excellent observation. Indeed, there is gauge freedom in the canonical space -- any set of transformations $\\{T_{\\theta_i}\\}$ that map to a consistent shared space is valid up to a global change of coordinates.\n\nWe do not impose an explicit regularization or constraint on the canonical frame. As a result, some of the learned canonical spaces can appear distorted or skewed. However, since the alignment quality is determined by **relative transformations**, this has no negative impact on downstream use cases.\n\nFor visualization purposes, we subtract the **the average transformation** from all images, which helps normalize the canonical space and improves interpretability. This strategy is sufficient for display."}, {"review_id": "aYhcVd3sBO", "reviewer": "Reviewer_SQ4g", "summary": "This paper proposed FastJAM, a novel GNN-based framework for fast and scalable image JA tasks. It combines the off-the-shelf sparse key point detector/matcher and a lightweight GNN, to propagate alignment cues across image collections and regress per-image transformations, and utilizing an inverse-compositional warping strategy. Extensive experiments show that FastJAM can significantly reduce computation time from hours or minutes to mere seconds, while maintaining similar accuracy compared to other SOTA methods.", "questions": "1. In Line 278, the authors said they replaced the RoMA with LoFTR in the ablation study, but as far as I know, LoFTR is a keypoint-free matching method, how to replace RoMA with it to extract the spare KP sets (as described in Line 147)? Also, both RoMA and LoFTR are not designed for cross-instance matching, why LoFTR show more limitations than RoMA in the JA tasks?\n2. Theoretically, the semantic matching method seems to be more suitable for JA. Have the authors tried to replace RoMA with other cross-instance semantic matching methods?\n3. In Line 210, the author said \"We have found that only checking for horizontal flips is sufficient\". Do you have ablation study results to support it?\n4. In Line 259, can you provide some figures to show the \"ambiguity\" of symmetric object \"bicycles\"?\n5. In Figure 5, it seems that SpaceJAM only uses ~85% of total GPU memory when number of images is 40. Why the authors said \"the PCA preprocessing in SpaceJAM fails to run beyond N = 40 due to memory constraints\" in Line 275.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Stengths:\n1. The method proposed by the author is simple and the experiment proves its huge speed improvement compared to previous works.\n2. Extensive experiments support the author's claim about the speed very well.\n\nWeakness:\n1. The ablation study section is too simple, only showing limited results and lacking a more in-depth analysis.\n2. The authors did not provide enough explanation about why FastJAM is so fast.\n3. Please refer to the \"Questions\" for other weaknesses.", "was_revised": true, "final_justification": "This paper has notable strengths and value of acceptance, especially regarding its acceleration effect, and the method is simple but effective. The experiments are slightly insufficient (about where the acceleration comes from), though this is not a fatal flaw.", "rebuttal": "We thank the reviewer for their thorough comments.\n\n---\n\n### **Ablation study and extended analysis**\n\nAs requested, we expanded the ablation study and will include it in the revised version. \nConcretely, we expanded the study with the following additional conditions. \n\n- **Linear projection baseline**: Instead of a GNN, we use a single linear projection then per-image pooling.\n- **MLP per-image baseline**: We use an MLP that receives keypoints from one image at a time, without graph connectivity.\n- **Direct optimization of homographies**: We optimize $\\{\\theta_i\\}$ directly without any learnable model.\n\nWe also added a **Vertical Flip** study.  \n  \n| Variant                               | PCK (CUB-200 first 3 cate.) | PCK (SPair-71K)|\n|---------------------------------------|------------------|-----------------------|\n| Linear projection                     | 33.2              | 14.0                   |\n| FastJAM with Vertical Flips                  | 56.9             | 44.1                   |\n| MLP (per-image, no graph)             | 73.5              | 47.8                   |\n| Direct homography optimization        | 74.1              | 48.3                   |\n| FastJAM (ours, full)                  | 75.3             | 50.4        |\n\n\nRegarding the Vertical flips, the results are significantly worse than for a model only trained with horizontal flips. We speculate that the reason for these results is that checking $2\\times 2$ flip configurations (vertical X horizontal) leads to slower convergences and worse overall performance.\n\n---\n\n### **Why is FastJAM so fast?**\n\nFastJAM's efficiency stems from three key design decisions:\n\n1. **Sparse keypoint-based formulation**: Unlike previous methods that use dense DINO features (e.g., ASIC, Neural Congealing, SpaceJAM), we rely on sparse correspondences. This dramatically reduces memory and compute requirements.\n2. **Global joint graph processing**: FastJAM processes all images in a single GNN forward pass over a unified graph. In contrast, prior approaches process one image at a time and rely on many forward passes, especially during test-time optimization.\n3. **Regularization-free loss**. Thanks to the IC formilation, and as in SpaceJAM, not regularization on the predicted transformations is needed. As was shown in SpaceJAM, obivating the need of regularization does not only remove the need for dataset-specific regularization-term hyperparameter tuning  but also enables a much smaller network and faster convergence. \n\nTogether, these enable FastJwith several new experiments that isolate key components in FastJAM:AM to align images much faster than previous methods.\nNote that we share with SpaceJAM only the third point, but not the first two. Thus, while prevous methods like ASIC or Neurocongealing take more than a hour, SpaceJAM takes minutes and our method takes seconds. \n\n---\n\n### **Use of LoFTR**\n\nIndeed, LoFTR is a detector-free matcher. However, it outputs sparse matches in the form of matched keypoints for both images along with confidence scores. Specifically, it returns:  \n- `mkpts0_f`: matched keypoints in image A  \n- `mkpts1_f`: matched keypoints in image B  \n- `mconf`: match confidence  \n\nThese outputs are used similarly to those of RoMa. We sample the top-N confident matches (after NMS) and use them as our sparse input keypoints. This allows us to treat LoFTR and RoMa comparably in our framework.\n\nDisplayed here is a similar code to what we used taken from LoFTR repository on Github (demo/demo_loftr.py):\n```python\nframe_tensor = frame2tensor(frame, device)\ndata = {'image0': frame_tensor}\nmatcher = LoFTR(config=default_cfg)\nmatcher.load_state_dict(torch.load(opt.weight)['state_dict'])\nmatcher = matcher.eval().to(device=device)\nmatcher(data)\nmkpts0 = data['mkpts0_f'].cpu().numpy()[vis_range[0]:vis_range[1]]\nmkpts1 = data['mkpts1_f'].cpu().numpy()[vis_range[0]:vis_range[1]]\nmconf = data['mconf'].cpu().numpy()[vis_range[0]:vis_range[1]]\n```\n\n---\n\n### **LoFTR vs RoMa - performance**\n\n\n\nYou're correct that both matchers were not explicitly designed for cross-instance matching. However, we believe that RoMa performs better in our setting due to its **DINOv2 feature backbone**, which captures higher-level semantic similarity between object parts.\n\nLoFTR uses a different architecture, which may not generalize well to semantic variation across instances. In our experiments, RoMa yielded more consistent and spatially meaningful matches on SPair-71k and CUB-200.\n\n---\n\n### **Semantic matchers for JA**\n \nWe explored the use of several state-of-the-art semantic matchers, but most rely on computing cosine similarity over dense DINO features in order to compute the Nearest Neighbor for each pixel to output a matching, which is computationally expensive. These approaches scale poorly: for N images, the pairwise cosine similarity matrix scales as O(N²HW), which is impractical for our joint alignment setup.\n\nRoMa offers a good tradeoff between semantic awareness and runtime, which aligns with FastJAM’s goal of practical and scalable joint alignment.\n\n---\n\n### **Ambiguity in symmetric objects**\n\n**Response:**  \nTo better illustrate this, we wished to include here such images of different instances but according to this year's NeurIPS rules we cannot share those. However, we can include such figures in the revised version.\n\nBy \"ambiguity,\" we refer to the difficulty in distinguishing front and rear regions in symmetric objects. For example, the keypoint matcher may confuse the front and back wheels of a bicycle. While the alignment remains geometrically valid, the direction of the object in canonical space may be flipped. These are rare but observable failure cases.\n\n---\n\n### **Memory usage in Figure 5**\n \nThank you for this question. Let us clarify why there was no mistake here. \n\n- SpaceJAM’s failure beyond N=40 is not only due to GPU memory, but also due to **host (CPU) memory and tensor size limitations** during PCA and DINO pre-processing.\n- We attempted to run SpaceJAM on 50–90 images using the same RTX 4090 setup used for FastJAM. It consistently failed with out-of-memory errors (in CPU, outside GPU) and could not complete the PCA.\n\nWe will update the wording in the revised paper to clarify this distinction more accurately."}, {"review_id": "D0HZ0oBwQe", "reviewer": "Reviewer_dWUX", "summary": "FastJAM is a method for aligning a small collection of images (typically 10 to 100) into a unified coordinate frame by estimating and applying a homography for each image. The pipeline consists of the following steps:\n1. Object Mask Extraction: For each image, extract object-centric masks using Grounded-SAM.\n2. Keypoint Matching: For each image pair, extract correspondences within the object masks using RoMA and NMS, retaining only 10 keypoints per image.\n3. Graph Construction: Build a graph where:  \n    - Nodes represent keypoints (2D coordinates).  \n    - Each image contributes edges between its keypoints.  \n    - Each image pair contributes edges between matched keypoints.\n4. Graph Neural Network: Apply a 5-layer GraphSAGE model for message passing over the graph. Use average pooling over keypoints per image to obtain an embedding, followed by a linear layer that predicts 8 homography parameters per image.\n5. Optimization: Train the network using Adam to minimize a robust inverse-compositional keypoint alignment loss.\n6. Homography Application: Apply the estimated homographies to align each image.", "questions": "1. In Figure 2, it appears that 2D tracks are constructed—e.g., the red 2D point in image I1 is matched across images I2, I3, and I4. Is this correct? If so, applying RoMA + NMS to I1–I2 and I1–I3 would yield two different sets of keypoints for I1. Do you merge these sets? If so, the paper does not describe this process.\n\n2. RoMA is trained for general image matching, not semantic keypoint matching—correct? For example, it is surprising that it can match different instances of the same object class (e.g., different dogs). Could the authors clarify or provide evidence for this behavior?\n\n3. The global parameterization of homographies via the matrix exponential on SL(3) seems inelegant. I assume this choice is numerically stable due to the “small” nature of the estimated homographies. Was the reflection check introduced for numerical stability? Did you consider alternative parameterizations, such as using four corner points?", "limitations": "The method appears to have quadratic complexity in the number of images, which could be a significant limitation when scaling to larger datasets.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n1. The paper is well written and easy to follow.\n2. FastJAM builds on well-established methods such as RoMA and Grounded-SAM.\n3. The method is simple and relatively elegant.\n4. FastJAM is an order of magnitude faster than the state-of-the-art SpaceJAM, and achieves slightly better performance in terms of PCK@0.10.\n\n## Weaknesses\n1. JAM (we are talking about aligning 10 to 100 images) seems to be a niche topic. It is unclear whether this problem attracts a wide audience at NeurIPS. It would be helpful for the authors to clarify why JAM is of broad significance to the AI community.\n2. The inverse compositional approach was proposed in SpaceJAM, so the main contribution of FastJAM is to consider a loss over 2D coordinates instead of DINO features.  While this change contributes to the improved efficiency and simplicity of FastJAM, it reduces the overall originality and significance of the contribution.\n3. All the evaluations are performed using PCK@0.10. I understand that a choice had to be made to obtain a quantitative evaluation, but I am wondering if this actually reflects the qualitative results. The only comparative qualitative results that I found are in the supp. mat. Fig.6 and the difference between SpaceJAM and FastJAM is not clear/impressive. \n\nIn summary, the paper's clarity and technical quality are strong points. However, in my opinion, the contributions lack the level of novelty and broader impact typically expected at a conference like NeurIPS.", "was_revised": true, "final_justification": "The authors have addressed all my questions. Based on their clarifications, my understanding is that:\n- JAM addresses a relatively niche problem within computer vision.\n- The key architectural and conceptual differences from SpaceJAM appear to be limited to changes in the loss function and the use of sparse inputs instead of dense.\n- FastJAM is still limited to small-scale scenarios (10 to 100 images).\n\nWhile I appreciate the detailed responses, these points continue to limit my assessment of the paper’s potential impact, and I therefore maintain my original rating.", "rebuttal": "We thank the reviewer for their thorough comments.\n\n---\n\n### **Collection size (10–100 images)**\n\nWe agree that 100 images may appear small in absolute terms, but this is already **way more** than what existing joint alignment methods support. For example,  ASIC (a recent work from Google), and Neural Congealing take over an hour to operate on ~30-40 images.  Compared with SpaceJAM, our method is more efficient and usually more accurate. In fact, as seen in Figure 5, applying FastJAM to 90 images is still faster than the time it takes SpaceJAM to process 30. Moreover, (CPU) memory limitations prevent the SpaceJAM's implementation to process more than 40 images.  \n \n\n\n---\n\n### **Broader significance of Joint Alignment (JA)**\n\n\nJoint Alignment and Image congealing have a long and well-established history in computer vision. Key works, out of many others, include:\n\n- **Learning from One Example Through Shared Densities on Transforms** (Miller et al., CVPR 2000)  \n- **Data Driven Image Models through Continuous Joint Alignment** (Learned-Miller et al., TPAMI 2006)  \n- **Unsupervised joint alignment of complex images.** (Huang et al., ICCV 2007)  \n- **Least Squares Congealing for Unsupervised Alignment of Images** (Cox et al., CVPR 2008)  \n- **Least-Squares Congealing for Large Numbers of Images** (Cox et al., ICCV 2009)  \n- **Aligning images in the wild** (Lin et al., CVPR 2012)\n\nIn recent years, interest in JA has resurged with the rise of unsupervised learning and structured representation discovery:\n\n- **ASIC** (Gupta et al., ICCV 2023 *Oral*)  \n- **Neural Congealing** (Ofri-Amar et al., CVPR 2023)  \n- **SpaceJAM** (Barel et al., ECCV 2024)\n\nIn particular, we believe this field remained mostly underexplored due to computational limitations associated with JA. Since the problem can now be solved in less than a minute, FastJAM could be applied to problems such as video frames alignment, panoramas, and more (please also see our response to *Reviewer xNrY* under `Practical applications enabled by FastJAM's speed'). \n\n---\n### **Technical novelty and relation to SpaceJAM (Barel et al., ECCV 2024)**\n \nWhile FastJAM relates to SpaceJAM (in particular in the use of inverse compositional framework), it introduces **key architectural and conceptual differences** that make it a novel and distinct method.\n\n- **Joint Forward Pass over All Images:** In FastJAM, the model receives **all keypoints from all images simultaneously** in a graph structure. This allows the GNN to perform **joint reasoning process during the forward pass**, enabling global context to directly influence alignment.  In other words, FastJAM leverages all images together in not only its (keypoint-based) loss function but also in its input. \n  In contrast, **SpaceJAM's input is a single image**. As such, it must rely entirely on its (dense-feature based) loss function to capture multi-image relationships. \nThis architectural difference is illustrated in **Figure 3**.\n\n- **Sparse vs. Dense Features:** FastJAM takes **sparse keypoints as input**, whereas SpaceJAM relies on dense DINO features. This transition changes the data representation from image-like tensors processed by CNNs to a graph of keypoints processed by a GNN. This difference enables a **significant reduction in runtime and RAM usage**.\n\n\n\n---\n\n### **Qualitative vs. Quantitative Evaluation**\n\n\nWe follow the `PCK@0.10` metric, which is standard in recent JA literature, including ASIC, SpaceJAM, and Neural Congealing, to compare quantitative results to those methods. While the visual difference may be subtle due to the nature of alignment tasks, FastJAM shows a **~5% PCK gain over SpaceJAM** on SPair-71K. \nWe have also included runtime analysis and, unlike previous works, RAM consumption. \n\nWith that being said, we will expand qualitative visualizations in the revised version to clarify differences. Due to NeurIPS' new policy, we cannot inlcude such visualizations here.  \nIn addition, we would wish to refer to the expansion of the ablation study as suggested by *SQ4g*.\n\n---\n\n### **Use of RoMa for cross-instance matching**\n\n\nRegarding RoMa's cross-instance matching: \nYou raise an excellent point,  and we appreciate the opportunity to elaborate on it. While RoMa was originally designed for wide-baseline matching within the same scene, its use of DINOv2 as a feature backbone making it aware of semantic matching. This makes it fairly robust for cross-instance matching, such as aligning different birds, cars, or other object categories.\n\nWe found empirically that RoMa (Edstedt et al., CVPR 2024) consistently produced higher-quality correspondences across instances compared to several other off-the-shelf matchers. These observations led us to choose RoMa as the default matcher.\n\nTo better illustrate this, we hoped to include here pairs of images of different instances with keypoints matching by RoMa but according to this year's NeurIPS rules we cannot share those.\n\n---\n\n### **Keypoint merging**\n\nCurrently, we do not merge keypoints across pairwise matchings. We experimented with merging them into a consistent set per image, but found it to increase the runtime and without any noticeable improvement in results. \n\n---\n### **Parameterization via SL(3) and matrix exponential**\n\nIn our opinion, using the Lie group is an elegant way to ensure invertible matrices and parametrize homographies, but to each his own. \nIn any case, the SL(3) and matrix exponential  parameterization has been widely used for decades. For example, in addition to SpaceJAM that we mentioned in this context in the paper, see also:\n\n- Mei, Christopher, et al. \"Homography-based tracking for central catadioptric cameras.\" 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2006.\n\n- Arrigoni, Federica, and Andrea Fusiello. \"Synchronization problems in computer vision with closed-form solutions.\" International Journal of Computer Vision 128.1 (2020): 26-52.\n\n\nWe use it for the following reasons. \n\n- It ensures **invertible** matrices, which are critical for computing the IC loss. Without this, ~2% of runs crashed due to singular matrices (we repored this in Lines 282–285 in the paper).\n- It introduces a **smooth, differentiable, and globally valid** parameter space, which is standard in homography regression and pose estimation tasks.\n\n\n\nWhile the **reflection check** also adds numerical stability (by computing the gradients only for the best configuration) the purpose is to handle symmetric ambiguities, such as flipping in the canonical frame. \n\nWe did not consider corner-point parameterization. We believe it should be applicable as long as one can find a way to ensure an *invertible* transformation matrix is recoverable for each image to compare the keypoints between image pairs."}]

Author rebuttal:

Meta-review: Following the rebuttal and discussion phases, this submission received two Borderline Accept and two Borderline Reject ratings. Reviewers generally agreed that the submission is well written, the proposed method has novelty, and the results outperform baselines in both accuracy and speed. However, some concerns were raised regarding the level of novelty and the completeness of the experiments (e.g., missing runtime analysis). The rebuttal addressed several of these concerns by clarifying technical details and adding new ablation studies. 

Among the negative reviews, R-xNrY's assessment lacked substance, included unsubstantiated claims about the method's novelty, and the reviewer did not meaningfully engage in the discussion phase(s). The other Borderline Reject, from R-dWUX, focused primarily on whether NeurIPS was an appropriate venue for the work. However, this reviewer acknowledged the paper’s strengths during the discussion. 

After weighing the reviews and discussion, the Area Chair found the arguments of R-aDpz to be the most compelling. The method demonstrates novelty, the results are strong, and concerns about venue suitability were not shared by the other reviewers or the AC. Accordingly, the Area Chairs have decided to accept the paper. Please incorporate the reviewers’ feedback when preparing the camera-ready version.
Decision: Accept (poster)