JAFAR: Jack up Any Feature at Any Resolution
Authors: Paul Couairon
Loick Chambon
Louis Serrano
Jean-Emmanuel HAUGEARD
Matthieu Cord
Nicolas THOME
Venue: NeurIPS 2025 poster ()
Keywords: Feature Upsampling
Dense Vision Tasks
Primary area: deep_learning
TL;DR: We propose a novel architecture and training objective specifically designed to upsample features from foundation vision encoders at any resolution.

Abstract: Foundation Vision Encoders have become indispensable across a wide range of dense vision tasks. However, their operation at low spatial feature resolutions necessitates subsequent feature decompression to enable full-resolution processing. To address this limitation, we introduce JAFAR, a lightweight and flexible feature upsampler designed to enhance the spatial resolution of visual features from any Foundation Vision Encoder to any target resolution. JAFAR features an attention-based upsampling module that aligns the spatial representations of high-resolution queries with semantically enriched low-resolution keys via Spatial Feature Transform modulation. Despite the absence of high-resolution feature ground truth; we find that learning at low upsampling ratios and resolutions generalizes surprisingly well to much higher scales. Extensive experiments demonstrate that JAFAR recovers intricate pixel-level details and consistently outperforms existing feature upsampling techniques across a diverse set of dense downstream applications.

JAFAR: Jack up Any Feature at Any Resolution
Paul Couairon 1.2 "Loick Chambon 1.3
Louis Serrano'
Nicolas Thome 1,4 Sorbonne Université, CNRS, ISIR, F-75005 Paris, France Thales, TSGF, cortAlx Labs, France Valeo.ai Institut Universitaire de France (IUF)
Jean-Emmanuel Haugeard Matthieu Cord 33
CLIP
JAFAR
Semantic Segmentation
DINOVZ
JAFAR
Depth
JAFAR
Class Activation Maps
Input Image
Upsampled Features
Outputs
Figure 1: JAFAR upsamples features from any foundation vision encoder any image resolution, using the input image as high-resolution guidance. It generates sharp, boundary-alignn feature maps and serves versatile drop-in module for variety of downstream tasks, including semantic segmentation, open- sopen-vocabulary segmentation, depth estimation, CAM evaluation, and bird's s-eye-view segmenttaioonncoosistentyy enhancing performance.
Abstract
Foundation Vision Encoders have become essential for a wide range of dense vision tasks. However, their low-resolution spatial feature outputs necessitate feature up- sampling to produce the righrresolution modalities required for downstream tasks. In this work, we introduce JAFAR, a lightweight and flexible feature upsampler that enhances the spatial resolution of visual features from any Foundation Vision En- coder to an arbitrary target resolution. JAFAR employs attention-based module designed promote semantic alignment between high-resolution queries, derived from low-level image features, and semantically enriched low-resolution keys, using Spatial Feature Transform (SFT) modulation. Notably, despite the absence of high-resolution supervision, we demonstrate that learning at low upsampling ratios and resolutions generalizes remarkably well to significantly higher output scales. Extensive experiments show that JAFAR effectively recovers fine graiigd spatial details and consistently outperforms existing feature upsampling methods across diverse set of downstream tasks. Project page: https:/jafaa-upsaaplergggiiiiiii pissampler.iithu.iio
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Foundation vision whether trained with language supervision [1, 2, 3, 4, 5] or purely on visual data [6, 7, have become core components of modern computer vision pipelines. Vision-language models excel at tasks requiring generalization, such as zero-shot classification and open-vocabulary segmentation [9, 10J. In contrast, image-only models, which focus on visual structure, often outperform in dense prediction tasks that demand fine-grained spatial reasoning, including semantic segmentation, depth estimation, object discovery, and point tracking [11, 12, 13].
To handle nigh-resolution inputs and large-scale training, foundation vision encoders typically downsample spatial information aggressively-by a factor of 14x to 16x-yielding semantically rich but spatially coarse feature maps. This compression introduces bottleneck for downstream tasks that require pixel-level accuracy. As result, downstream pipelines [14, 15, 11, 16, 17] often rely on interpolation or dedicated modules [18, 19] designed to produce high-resolution outputs.
Several strategies have been explored to overcome this bottleneck, but each comes with trade-offs in efficiency and output quality. A straightforward solution is apply training-free interpolation meth- ods, such as bilinear upsampling. While computationally efficient, these direct interpolations- solely on low-resolution feature maps-fail leverage information from the original high-resolution image, often resulting in blurry outputs. Alternatively, one can upsample the input image prior to encoding to increase feature resolution. However, this approach significantly increases computational cost due to the quadratic complexity of selfaatiention--ommon common in foundation models mod may introduce artifacts in the feature maps, ultimately degrading performance [20, 21].
Focusing specifically on a target downstream task, [22, 23, 24, 25, 26] learn feature upsamplers using high-resolution supervision from task-specific labels. While generally lightweight, these upsamplers depend on labeled data tied to the end application, which limits their generalization and may bias the learned features toward optimizing task-specific losses. To address this, recent methods such as LiFT [27] and FeatUp [28] adopt task-agnostic training objectives. LiFT is trained to perform 2x upsampling by regressing feature maps extracted from images twice the input resolution. However, its convolution- based architecture is limited to fixed 2 scaling, restricting its ficxibility for arbitrary output resolutions. FeatUp, in contrast, uses augmented views and self-reconstruction to support higher upsampling ratios. Yet, its Joint Bilateral Upsampling (JBU) variant suffers from over-smoothed outputs, while its implicit variant requires training the upsampler for each image, making it impractical in real-world scenarios.
In this paper, introduce feature upsampler designed satisfy the following criteria: (i) task-agnostic training objective, (ūi) support for arbitrary output resolutions, (iii) compatibility with any vision encoder, and (iv) minimal computational overhead inference time.
To enable upsampling to arbitrary target resolutions, we formulate our approach as a global in- terpolation mechanism using cross-attention block. The success of this attention-based method depends critically on achieving strong semantic alignment between the queries and keys. In JAFAR, construct these representations asymmetrically (see Fig. 2): the queries retain high-resolution, low-level details such as color and texture, while the keys are hybrid features that combine high-level semantics with spatial cues. We find that enriching the keys with low-level information significantly improves query-key alignment and enhances generalization to unseen output resolutions.
Additionally, we propose a simple training objective similar to [27], but without being constrained to a fixed upsampling factor. Notably, we find that training on low upsampling factors at low resolutions (c.g., 8 x 8 + 32 x 32) is sufficient to generalize effectively to much larger scales (e.g.. 32 x 32 + 448 x 448) while keeping memory requirements low during training, unlike training directly at higher resolutions and factors. Our contributions can be summarized as follows:
We introduce JAFAR, novel lightweight attention-based feature upsampler that naturally supports upsampling to arbitrary resolutions. It explicitly promotes spatial alignment between high-resolution queries extracted from low-level image features and semantically enriched low-resolution keys.
We enforce this alignment by computing both queries and keys from the same input features, and injecting semantic information from the encoder's deep features via spatial feature modulation. This design enables precise fusion of spatial detail and semantic context without reliance on external supervision.

---

We propose a highly efficient, task-agnostic training objective that requires no high- resolution supervision signal. Remarkably. we show that training at low resolutions and low upsampling ratios generalizes robustly to significantly higher output scales.
We demonstrate that the combination of our architecture and training objective yields substantial performance gains across a variety of downstream tasks. When used as a drop-in module, JAFAR consistently outperforms existing upsampling methods by wide margin.
2
Related Work
Feature Upsampling Feature upsampling aims to increase the spatial resolution of intermediate feature maps within deep networks -otoukks-nnalgouu image upsampling, but performed in a latent space. This process is essential for dense prediction tasks such as segmentation and depth estimation, where fine spatial detail is critical. Traditional interpolation techniques, such as bilinear, spline, Lanczos [29, 30, 31, 32], provide simple and efficient baselines but do not adapt to the underlying content. Recent neural methods improve on static approaches by learning to reconstruct high-resolution features from data. These methods fall into two categories: task-dependent, trained with downstream labels supervision, and task-agnostic, trained independently of the end task. For example, CARAFE [22] and DySample [24] predict content-aware kernels or dynamic sampling positions. SAPA [23] and ReSFU [25] exploit similarity based approach to refine spatial semantics. However, task specific reliance on labels limits generalization. Recent task-agnostic methods like LiFT [27] and FeatUp [28] remove this dependeney. LiFT introduces a CNN module trained with a simple fixed scale training objective, while FeatUp relies on complex multi-loss objective which makes training difficult to tune in practice. Moreover, it requires training both an upsampler and a downsampler, adding unnecessary computational overhead. Notably, its best performance is achieved through per-image optimization, further limiting its practicality. In contrast, JAFAR provides a scalable task-agnostic framework that generalizes across resolutions without complex pipelines or per-image optimization, showing strong performance even when trained on small upsampling factors at low resolution.
Architectural Design for Upsampling Modules Upsampling modules architectures vary from fixed-scale decoders continuous resolution predictors. LiFT [27] relies lightweight CNN module trained to upsample by fixed factor, making further scaling dependent on iterative use which leads to performance degradation or additional interpolation steps. FeatUp [28] introduces two architectural variants: a fast Joint Bilateral Upsampler (JBU) and a more accurate implicit network allowing continuous querying. While the implicit model yields superior results, it suffers from significant inference latency due to per-image optimization. JBU, on the other hand, trades expressivity for scalability, stacking multiple x2 stages to achieve higher upsampling ratios. Attention- based designs, as in SAPA [23] and ReSFU [25], offer increased flexibility by modeling affinities between features across scales. These methods exploit spatial similarities to reconstruct high- resolution maps. JAFAR innovates by unifying low- and high-resolution streams: it aligns high- resolution queries and low-resolution keys using shared low-level features while enriching the representation with additional semantic cues. This design maintains spatial alignment and expressivity even at large upsampling ratios, offering a robust and scalable architecture for feature reconstruction.
Semantic Guidance and Feature Modulation Feature modulation techniques modulate features using conditioning information, thereby enabling spatially semantically guided transformations. Early forms such as Conditional BatchNorm [33], AdaIN [34], and FiLM [35] apply learned scale (7) and shift (3) parameters per channel, derived from global conditioning signals. These methods are effective for tasks involving global transformations like style transfer or classification. However, their spatial invariance limits expressiveness in tasks requiring spatial sensitivity. SPADE [36] and SFT [37] address this limitation by computing y and |3 as full-resolution maps conditioned on dense inputs like segmentation masks. This spatial modulation enhances expressivity by enabling unique adjustments at each feature location. It can be interpreted as parameterized, learned recombination of feature channels, analogous to a 1 x convolution but extended with spatially varying weights. In JAFAR, feature modulation is used not only to shift feature distributions but also to inject semantics directly into the upsampling pipeline. This provides richer linear combinations of features, improving generalization and spatial fidelity without the need for per-image optimization [28].
3

---

3 JAFAR
JAFAR is feature upsampler that uses the input image as high-resolution guidance to reconstruct dense feature maps. To support upsampling to arbitrary target resolutions, we formulate the method as a global interpolation mechanism based on cross-attention. The effectiveness of this attention- based approach hinges on achieving strong semantic alignment between the queries Q and the keys K. In JAFAR, we construct the query and key representations asymmetrically. The queries retain high-resolution, low-level details such as color and texture, while the keys are designed as hybrid representations that combine high-level semantics with low-level spatial cues. We find that enriching the keys with low-level information significantly improves query-key alignment and enhances generalization to unseen output resolutions.
3.1 Architecture
The overall flow of our architecture is illustrated in Fig. 2. JAFAR takes as input a high-resolution image I E 3 Hx and low-resolutin feature map Fir f(] E Cxx x extracted from frozen vision encoder f. The image I is first projected into a dighe--di space and processed by lightweight encoder Fg to obtain an intermediate representation L = (T E RdxxHW further enriched with RoPE positional embeddings [38].
Quary Projector
Adaptive Pooling
Image Projector
S-hmax
Q.8" vā
Projector
Poaling
SFT
+
Pux
Vision Encoder
Figure 2: Overview of JAFAR. To construct the upsampling kernel, queries and keys are derived from a shared image representation. Queries are downsampled to match the target output resolution, while keys downsampled align with the spatial resolution of the vision encoder's 's features. Keys are then semantically enriched via SFT modulation to promote semantic alignment between queries and keys. The resulting kernel then used interpolate features from the foundation vision encoder.
Query features Q E gdxh, XM, are derived by passing the image representation I through small query encoder, producing lg. followed by adaptive average pooling to reach the target resolution (h wy). Key features K e gdxhhxxu X are similarly obtained by encoding Ig to Iς and downsampling it to match the spatial resolution of the semantic features Fi.. These semantic features provide modulation parameters that inject high-level information into the keys. A cross-attention mechanism then enables the queries Q to attend to the keys K by computing attention map:
(Q K Vd
A Softmax
which is then used to interpolate the low-resolution feature map Fir and produce the upsampled output features F A Fir E @C The resulting representation preserves fine-grained spatial details while remaining semantically consistent with the input image. We provide detailed description of each of the main components of the architecture below.
Query Branch Directly aligning high-resolution, low-level queries with high-level semantic keys often results in weak or noisy attention, as the disparity in abstraction levels limits meaningful interac- tions. To overcome this challenge, we apply adaptive average pooling downsample the intermediate representation 1ę and generate the query features Q. This operation, performed exclusively during

---

training, reduces the spatial resolution of the queries while aggregating local context into region-level descriptors. As result, the downsampled queries are more semantically aligned with the keys, less susceptible to pixel-level noise, and computationally more efficient due to the reduced number of tokens. These effects collectively make query downsampling an effective strategy for bridging the gap between fine-gres lieeggainedd visual details and abstract semantic representations, promoting more stable and scalable cross-scale attention. Importantly because downsampling is only applied during training, the model maintains its capacity to generate high-resolution outputs during inference.
Key Branch Relying exclusively on low-resolution features from the vision encoder construct keys leads to poor generalization and noticeable artifacts, primarily due to an abstraction gap between these coarse features and the fine-grained queries. As demonstrated in Sec. 4, this mismatch results in inconsistent alignment across resolutions. To address this issue, we construct hybrid key representations that retain structural alignment with the queries while incorporating the semantic richness of the vision encoder. This is achieved by encoding the intermediate representation Ig to produce Ix. which then downsampled to match the spatial resolution of the encoder's s feature map to produce preliminary keys K. These are further modulated using the vision encoder feature map Fir E R C'Ch,, Xwn, through a spatial semantic feature modulation inspired by [36, 37]:
(2) where YF. Br E Rdxxhxx x are spatially varying parameters obtained via linear projections from Fir- This adaptive, feature- wise modulation enriches the keys with localized semantic context, enhancing both spatial and semantic alignment and supporting more faithful and generalizable upsampling across resolutions.
K = NP @ K + 3p:
Similarity Based Upsampling To perform upsampling, we use simplified attention mechanism where attention weights are computed via a scaled dot product between queries and semantically modulated keys. Crucially, both queries and keys have been enriched with relative positional embeddings using RoPE [38], which introduces an inductive bias that captures spatial relationships between queries and keys. This positional encoding allows us to entirely bypass the arbitrary selection of neighboring keys for each query, a common heuristic in prior similarity-based methods such [23, 25]. Without this positional grounding, the attention mechanism lacks spatial awareness and generalizes poorly to unseen resolutions. In practice, we use multiple attention heads to increase expressivity and average the resulting attention weights across heads after applying softmax. The resulting attention map A is then used to interpolate the fow-resolution encoder features Fir via simple matrix product: FHR A Fir. By avoiding learned value projection, we preserve the original feature content and enable a resolution-agnon design that generalizes reliably across scales.
3.2 Training Pipeline
Learning to upsample high-resolution features without access to ground-trus truth supervision poses a natural challenge: how can model learn to produce sharp high-resolution features (e.g., 448 x 448) when only low-resolution features are available (e.g., 32 x 32)? Thanks to JAFAR's 's architectural design, the model can be trained with a simple objective low target resolution without requiring supervision at the original image size, yet it still generalizes effectively to much higher upsampling ratios during inference.
Training with Multi-Resolution Views scheme that relies only on multi-resolution views of the same image, easily obtained through standard downsampling. Given a high-resolution image TH E R Rx3xx* HxW we generate downsampled
To enable this, we introduce a fully annotation- training
version IL E 3 ]xLW] using a randomly sampled factor 5 E [2, 4]. Both images are passed through the frozen vision encoder f. producing two feature maps: For = f([nR) E Cxxxx and
Fir = f((RR) E Cx | respectively. JAFAR then takes HR and Fir as input to predict an upsampled feature map For The predicted output is aligned with the target Für using a simple alignment loss, which combines cosine similarity and I.2 distance [20]:
[(Frr: Fnr) = cos( Fnrr Fir)) + || -
Notably, during training, JAFAR is only exposed to moderate upsampling factors (up to 4x), yet it generalizes remarkably well to much higher resolutions at test time time-withou access to any ground- truth high-resolution features.
5

---

How is it different from LiFT? While our training objective is similar to that of LiFT, our approach demonstrates significantly greater capability, as shown Tabs. 1 and 2. LiFT relies on a CNN- architecture and is trained for fixed 2x upsampling at two predefined resolutions. As result, it struggles to extrapolate beyond that setting without additional heuristics such as iterative upsampling or bilinear fallback. In contrast, JAFAR maintains a resolution-agnoste design which generalizes to much higher upsampling factors using this similar simple training setup.
Experiments
4.1 Experimental Setup
In our experiments, train JAFAR on single NVIDIA A100 on ImageNet training set for 100K steps using AdamW optimizer [39], with a learning rate of 2e-4 and a batch size of 4. The input images fed into the foundation vision encoder are resized to 448 x 448, producing high-resolution target feature maps Für of size 32 32 or 28 x 28, depending on the encoder's patch size (14 or 16). For improved training efficiency, the guidance image input JAFAR is downsampled 224 x 224.
4.2 Qualitative Comparisons
Image
Silneer
Large-Image
Strided
CARAFE
SAIAA
DySample
Re8FU
FestUp
JAFAR
Training Free
Task Dependent
Task Agnestic
Figure 3: PCA Feature Visualization. DINOv2 ViT-S/14 features at 32 x 32 resolution from the ImageNet validation set are upsampled 448 x 448. Baseline methods- wethhods-whether training-free, task-dependent, or task-agnostic -introduce varying levels of blurriness and artifacts. Besides being task-agnostie, JAFAR produces sharp, content-awan feature maps with fewer artifacts.
-ggootic-iitrrouuce
To qualitatively evaluate the upsampled feature maps produced by various baselines, we project all features onto a shared 3-dimensional PCA basis, mapping them into a common RGB space. As shown Figs. and 5, the low-resolution features-due to the spatial compression imposed by the vision encoder's 's patch size-rrever large, blocky regions that capture semantic content but fail preserve fine image geometry, object boundaries, or shape details. Bilinear upsampling, which interpolates features without considering image content, yields blurry output feature maps that preserve positional embeddings artifacts without adding meaningful detail. While methods like Large- Image and Strided preserve sharpness, their outputs are noisier and less coherent than JAFAR's s. Furthermore, they more computationally demanding, as they require the vision encoder to process a larger number of patches (see Tab. 12). JAFAR shows clear qualitative advantage over all baselines, consistently producing sharp features that accurately capture image structure. is also the only task-agnostic method that effectively suppresses artifacts from positional embeddings in the low-resolution features.
4.3 Transfer on Downstream Tasks
Since upsampled features are expected to provide richer signal for downstream tasks, we evaluate their effectiveness on two benchmarks: amear-probing semantic segmentation and depth estimation, using DINOv2 ViT-S/14 as the foundation vision encoder. For the Large-Image and Strided baselines, upsampling is performed during the encoder's forward pass and followed by bilinear interpolation to reach the target output resolution. For task-agnostic upsamplers such as LiFT, FeatUp, and JAFAR, we pre-train the upsampling module on the corresponding backbone, then freeze it and apply it after feature extraction. The linear probe is trained independently of the upsampler. For task-dependent

---

methods, including CARAFE, SAPA, ReSFu, and DySample, we jointly train both the upsampler and the linear probe on each dataset and task. All experiments (except Large-Image) use input images of resolution 448 x 448, with target labels at the same resolution.
4.3.1 Semantic Segmentation
For semantic segmentation, we train linear projection head to predict coarse class labels using cross-entropy loss across several benchmark datasets: COCO-Stuff [40] (27 classes), ADE20K [41] (150 classes), Pascal VOC [42] (21 classes including background), and Cityscapes [43] (27 classes). The linear layer is trained for 5 epochs on COCO-Stuff and 20 epochs on the remaining datasets, using a batch size of 4. Performance is evaluated on the respective validation sets using mean Dntersection-vvr--niteee
(mloU) and pixel-wise accuracy.
Table 1:
Linear Probing on Downstream Tasks. JAFAR consistently outperforms other baselines across all segmentation benchmarks while reaching competitive depth metrics without being opti- mized on a specific downstream task.
Semantic Segmentation
Depth Estimation
2INSS1-VIT-S/14
COCO
VOC
ADE20K
Cityscapes
COCO
mloU (?)
Acc
mloU (?)
Acc (f)
mloU (1)
Acc
mloU (1)
Acc (1)
RMSF (
Training-free
Nearest
56.17
76.97
76.41
37.27
71.91
54.05
90.36
58.08
Bilinear
59.03
79.07
80.70
95.17
39.23
73.69
59.37
92.47
59.92
0.66
Large Image (x8)
56.94
88.60
26.42
66.39
47.72
92.49
Strided
55.93
77.40
75.88
93.94
36.15
72.08
59.26
92.57
56.98
0.70
Task- Deppeeent
CARAFE (22)
59.73
79.65
80.26
95.
38.30
73.42
56.05
91.83
61.42
0.64
SAPA [23]
57.77
78.28
77.02
94.07
35.87
71.85
50.12
90.02
60.34
0.67
DySample [24]
59.50
79.42
81.62
95.48
38.99
73.62
59.71
92.69
61.25
0.64
ReSFU
60.08
79.84
80.30
95,05
38.91
73.93
55.53
91.62
66.14
0.56
Ausk-Agnostic
FeatUp [28)
60.10
79.95
81.08
95.32
38.82
73.74
56.06
91.86
61.69
0.64
LIFT
58.18
78.95
78.06
94.62
38.73
73.69
58.75
92.60
57.04
0.70
JAFAR
60.78
80.47
84.44
96.28
40.49
74.92
61.47
93.42
62.18
0.62
As shown in Tab. 1, JAFAR consistently achieves the highest performance across all four semantic segmentation benchmarks, in both mloU and accuracy. On average, JAFAR delivers +1.63 mloU improvement over the next-best method across all datasets. Compared to FeatUp, JAFAR achieves an average gain of +2.78 mloU corresponding a +4.8% gain, with a peak improvement of +5.41 mloU (+9.7%) on Cityscapes. Fig. shows linear probe segmentation result.
4.3.2 Depth Estimation
For depth estimation, follow the approach in [28] and train on pseudo-labels generated by the state-of-the-art Depth Anything V2 network [16]. We report two standard metrics from the monocular depth estimation literature: root mean square error (RMSE) and 61 1.25. The ôn metric measures the percentage of pixels where the predicted depth y is within 25% of the ground-trutt y", formally
1.25. We train the linear probe for epochs on the COCO training set, using batch size of 4. Although JAFAR not trained on this specific task, observe that it reaches competitive scores, ranking second among the baselines. Notably, JAFAR outperforms both FeatUp and LiFT while also surpassing all task-dependent methods but ReSFU. Fig. 4 shows linear probe depth estimation result.
defined as 81 = max
g. v gn
4.3.3 Class Activation Maps Faithfulness
Following the approach in [28], our method can be seamlessly integrated into explainability tools such as Class Activation Maps (CAMs). Despite recent advances, CAMs are still fundamentally limited by the low-resolution feature maps produced by standard vision encoders, which hinders their ability to localize fine-grained details. By upsampling the features, our method yields sharper and more informative explanations. To assess the quality of the resulting CAMs, we adopt standard evaluation metrics from the literature: Average Drop (A.D), Average Increase (A.I), Average Gain (A.G), Coherency (Coh.), and Complexity (Cplx.).
7

---

San Sec
Fridas
Dust
Traksing Tree
Tark Depundor
Tank Agpnstic
Figure 4: Visual Comparison of Upsampler Outputs in Downstream Tasks. JAFAR-upsan -ppapldp features produce sharper outputs that align more accurately with object boundaries across various downstream tasks respectively class activations maps, semantic segmentation and depth estimation.
In particular, A.D, A.I, and A.G mea- sure how sensitive the classifier's 's out- put is the most salient regions of the input an effective CAM should highlight areas that, when masked, lead to notable change in classifi- cation confidence. Since each of these metrics captures only a single aspect of CAM quality, we also report the
Table 2: Grad-CAM Evaluation. Integrating JAFAR into Grad-CAM analysis yields significantly more faithful expla- nations compared to baseline methods. Top three methods are highlighted as first, second, third according to ADCC.
A.D
Cpls.
ADCC
Training-free
Billnear
19.0
18.5
3.4
88.8
59.9
61.7
48.8
67.9
38.0
ADCC score-an aggregate metric proposed in [44] that provides more holistic evaluation. Additional details are provided in Supp. A.1. As illus- trated qualitatively in Fig. 4, JAFAR generates sharper and more semanti- cally accurate CAMs compared to all baselines. While training-fre free methods don't help to recover important regions, task-dependent approaches typically produce blurrier and less precise maps. Quantitative results in Tab. 2 further support this, with JAFAR achieving the highest score on the aggregate ADCC metric--otperforming the second-best method by points, relative improvement of 12.5%.
Task-Dependens
CARAFE [22]
49.9
66.9
34,9
59.7
[25]
Dysample [24]
17.8
20.0
90.2
60.7
61.6.11.3
ReSFU [25]
14.5
24.0
3.7
92.1
64.2
59.4 (43.9)
Task-Apuostic
FeaUUp [28]
15.3
24.0
91.6
58.2
64.3 co.
LIFT [27]
66.9
8.7
2,3
65.2
9.4
$3.0 c20.5
JAFAR
17.4
30.9
6.5
91.4
44.1
73.3
4.3.4 Zero-Shot Open- Vocabuaaay Segmentation
We further evaluate our method on a zero-shot ppen-vocabulary segmentation task, following the setup from [9], where class labels from the dataset serve as textual inputs and predictions are made by selecting the class with the highest similarity score (argmax). Using a CLIP-ViT-BL B/6- backbone, this approach entirely training-free
does not require a learned probing head. Results show that JAFAR significantly outperforms all baselines, with particularly strong improvements Pascal VOC. Despite the increased difficulty of ADE20K, which includes 150 classes, our method still achieves the highest performance in both mloU and accuracy. We report only FeatUp among the task-agnostic baselines, as is the second-best performing method.
Table 3: Zero-Shot Open- Vocabulary Evaluation. Using MaskCLIP [9] for zero- shot mpen-vocabulary segmentation, JAFAR consistently improves performance, indicating strong alignment with the original features.
VOC
ADE20K
Cityscapes
Upsampling
mloU (1)
Acc (t)
mloU (f)
Acc (1)
mloU (t)
Acc (1)
Nearest
24.13
30.80
9.33
24.65
19.66
50,27
Bilinear
27.87
35.27
11.03
27.78
21.56
53.21
Large Image (x2)
23.24
32.16
8.08
24.94
21.91
52.22
FeatUp [28]
32.27
39.78
13.03
33.28
24.76
60.11
JAFAR
35.70
44.93
13.61
33.28
25.26
61.73

---

4.3.5 S-ye View Segmentation
Finally, we studied the impact of our upsampler in a complex training pipeline. The task, eval- uated on nuScenes [48], takes several images taken from cameras as input and consists on out- putting the bird's s-eye view (BeV) segmentation map. In our setup, we used frozen DINOv2 [7] backbone and trained the rest of the architecture, namely, the upsampler, the BeV encoder, and the segmentation head. This task is particularly challenging. as the model must learn to map features from the image plane to the BeV plane. To ensure fair comparison, we also trained the architecture without an upsampler, using lower-resolution input images (224 x 476). We adopted the optimization hyperparameters from PointBeV [46]. adjusting the batch size to 1 and training for 100 epochs. Our results show that using an upsampler consistently improves predictions, regardless of the architecture Sopploydd-Simpleee [45]. V [46], BevFormer [47]. Notably, performance improves significantly when using JAFAR as the upsampler, with mloU gains up to +5 points.
Table 4: BeV Vehicle Segmentation. JAFAR con- sistently improves vehicle-IoU in complex BeV architectures, outperforming all other baselines.
Upsampling
SimpleBeV [45]
Pointlle [40]
BeVFormer [47]
Low. Low-Res
31.75
34,89
33.72
Bilinear
36.01
34.18
PemUUp
33,95
35.
34.01
JAFAR
36.59
37.20
36.54
4.4 Ablations
To evaluate the benefit of deriving both queries and keys from shared image encoding, we compare in Tab. several strategies to obtain keys. In the Linear Projection baseline, keys are obtained by applying a linear layer to the vision encoder's 's low-resolution features Fir, to match JAFAR's embedding dimension. In the Concatenation baseline, semantics is injected via direct concatenation of Fir and preliminary keys K.
Table 5: Attention mechanism ablations with respect to key strategy and number of attention heads. Best scores per dataset are in bold and selected choices are highlighted in blue.
Semantic Segmentation
Ablation Type/ / Setting
VOC
ADE20K
Cityscapes
mloU (1)
Acc (f)
mloU (1)
Acc (1)
mloU (f)
Acc (?)
Keys Strategy
Linear Projection
80.02 (-4.2)
94.87 (.1.4)
37.87 (262)
73.22 (.1.7)
52.45 (9.02)
90.80 (:262)
Concatenation
83. (-1.7)
95.94 (0.34)
40.06 (4.43)
74.56 (-0.36
58.70 (2:77)
92.65 (4.77)
w/o SFT
83.25 (-1.19)
95.93 (1035)
39.62 (407)
74.32 (-0.00)
56.53 (4.94)
92.19 (-1.25)
w/ SFT
84.44
96.28
40.49
74.92
61.47
93,42
Attention Heads
π
84.13 (-0.31)
96.21 (.0am)
40.15 (-034)
74.79 (-0.13)
60.94 (41.53)
93.32 (-0.10)
π
84,27 (0.17)
96.27 ( (.000)
40.42 (0007)
74.95 (+0.0)
61.19 ( 0.28)
93.42 (0.00)
84.44
96.28
40.49
74.92
61.47
93,42
83.82 (-1.22)
96.13 (40.15)
40.07 (-042)
74.20 (40.72)
60.56 (41.91)
93.33 (-0.99)
In comparison, the Linear projection baseline shows a significant performance drop, and SFT consistently outperforms the concatenation approach. Increasing the number of attention heads up to further enhances performance by producing more robust upsampling kernels through averaged post-softmax scores. Beyond this point, however, the benefits reverse: the per-head dimensionality becomes too low to support effective alignment, while the computational cost increases, ultimately degrading output quality.
5 Conclusion
We introduce JAFAR, a lightweight, attention- feature upsampler designed with simple training objective. It can upscale features from any foundation vision encoder arbitrary output resolutions, without requiring supervision at the original image size or annotations from downstream tasks. Although task-agnostic, JAFAR outperforms prior state-of-the-art upsamplers across a variety of downstream tasks, despite not being trained specifically for them. This work lays the groundwork for a unified feature upsampler that could enable significantly more efficient architectures for dense vision tasks. Currently, the method requires training separate upsampler for each backbone. Future

---

work will focus on making JAFAR backbone- backbone-independ independennt at inference time and on further reducing feature-level artifacts to produce sharper outputs.
6
Acknowledgment
This work has been supported by chair VISA DEEP (ANR-20-CHIA--022) Cluster PostGienAld Paris (ANR-23-1ACL-0007, FRANCE 2030). This work has been supported by PEPR Sharp (ANR-23- PEIA-0008, FRANCE 2030). This work was granted access to the HPC resources IDRIS under the allocation 2025-AD01 101463RR made by GENCI.
References
[1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning,
pages 8748-8763. PmLR, 2021.
[2] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. Proceedings of the IEEE/CVF international conference on computer vision, pages 11975-11986, 2023.
[3] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alab- dulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip Multilingual
encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786, 2025.
2:
[4] Zineng Tang, Long Lian, Seun Eisape, XuDong Wang, Roei Herzig, Adam Yala, Alane Suhr, Trevor Darrell, and David M Chan. Tulip: Towards unified language-image pretraining. arXiv preprint arXiv:2503. 1 15485, 2025.
[5] Greg Heinrich, Mike Ranzinger, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, Pavlo Molchanov, et al. Radio amplified: Improved baselines for agglomerative vision foundation models. arXiv preprint arXiv:2412.07777 2024.
[6] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self supervueee vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-96600 2021.
[7] Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojeiech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision, 2023.
[8] Timothée Darcet, Federico Baldassarre, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Cluster and predict latents patches for improved masked image modeling. arXiv preprint arXiv:2502.0877 2025.
[9] Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In European Conference on Computer Vision, pages 696-712. Springer, 2022.
[10] Feng Wang, Jieru Mei, and Alan Yuille. Sclip: Rethinking self-attention for dense vision- language inference. In European Conference on Computer Vision, pages 315-332. Springer, 2024.
[11] Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10371-10333 2024.
10

---

[12] Cijo Jose, Théo Moutakanni, Dahyun Kang, Federico Baldassarre, Timothée Darcet. Hu Xu, Shang- Wen Li, Mare Szafraniec, Michael Ramamonjisoa. Maxime Oquab, Oriane Sim' coni, Huy V. Vo, Patrick Labatut, and Piotr Bojanowski. Dinov2 meets text: A unified framework for image- and pixel-level vision-language alignment. ArXiv, abs/2412. 16334, 2024.
[13] Narek Tumanyan, Assaf Singer, Shai Bagon, and Tali Dekel. Dino-tracker: Taming dino for self-supervised point tracking in single video. In European Conference on Computer Vision, pages 367-385. Springer, 2024.
[14] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer, 2020.
[15] Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. Open-vocabulary panoptic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2955-2966, 2023.
[16] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Processing Systems, 37:21875- 29911,2024
[17] Paul Couairon, Mustafa Shukor, -mmaneema HAUGEARD, Matthieu Cord, and Nicolas THOME. Diffcut: Catalyzing zero-shot semantic segmentation with diffusion features and recursive normalized cut. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[18] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 22159-12168 2021.
[19] Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-atteniine mask transformer for universal image segmentation. 2022 IEEE/CVF Confer- ence on Computer Vision and Puttern Recognition (CVPR), pages 1280-1289, 2021.
[20] Jiawei Yang, Katie Z Luo, Jiefeng Li, Congyue Deng, Leonidas Guibas, Dilip Krishnan, Kilian Q Weinberger, Yonglong Tian, and Yue Wang. Denoising vision transformers. In European Conference on Computer Vision, pages 453-469. Springer, 2024.
[21] Qihang Fan, Quanzeng You, Xiaotian Han, Yongfei Liu, Yunzhe Tao, Huaibo Huang, Ran He, and Hongxia Yang. Vitar: Vision transformer with any resolution, 2024.
[22] Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, and Dahua Lin. Carafe: Content- aware reassembly of features. In Proceedings of the IEEE/CVF international conference computer vision, pages 3007-3016, 2019.
[23] Hao Lu, Wenze Liu, Zixuan Ye, Hongtao Fu, Yuliang Liu, and Zhiguo Cao. Sapa: Similarity- aware point affiliation for feature upsampling. Advances in Neural Information Processing Systems, 35:20889-20901, 2022.
[24] Wenze Liu, Hao Lu, Hongtao Fu, and Zhiguo Cao. Learning to upsample by learning sample. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6004-6014, 2023.
[25] Minghao Zhou, Hong Wang, Yefeng Zheng, and Deyu Meng. A refreshed similarity-based upsampler for direct high-ratio feature upsampling. CORR, 2024.
[26] Hao Lu, Wenze Liu, Hongtao Fu, and Zhiguo Cao. Fade: Fusing the assets of decoder and encoder for task-agnostic upsampling. In Proc. European Conference Computer Vision (ECCV), 2022.
[27] Saksham Suri, Matthew Walmer, Kamal Gupta, and Abhinav Shrivastava. Lift: A surprisingly simple lightweight feature transform for dense vit descriptors. In European Conference on Computer Vision, pages 110-128. Springer, 2024.

---

[28] Stephanie Fu, Mark Hamilton, Laura E. Brandt, Axel Feldmann, Zhoutong Zhang, and William T. Freeman. Featup: A model-agnostic framework for features at any resolution. In The Twelfth International Conference on Learning Representations, 2024.
[29] Einar Maeland. On the comparison of interpolation methods. JEEE transactions medical imaging, 1988.
[30] Claude E Duchon. Lanczos filtering in one and two dimensions. Journal of Applied Meteorology. 1979.
[31] Isaac Schoenberg. Cardinal spline interpolation. SIAM, 1973.
[32] Sky McKinley and Megan Levine. Cubic spline interpolation. College of the Redwoods, 1998.
[33] Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. A learned representation for artistic style. In International Conference Learning Representations, 2017.
[34] Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. Proceedings of the IEEE international conference on computer vision, pages 1501-1510, 2017.
[35] Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In AAAI Conference on Artificial Intelligence, 2018.
[36] Taesung Park, Ming- Yu Liu, Ting-Chun Wang, and Jun- Yan Zhu. Semantic image synthesis with spatially-adaptic normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2337-2344 2019.
[37] Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings the IEEE conference on computer vision and pattern recognition, pages 606-615, 2018.
[38] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 128:12 2024.
[39] Uya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 171.551711.05 2017.
[40] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[41] Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision, 127:302-32 2019.
[42] M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 11 111(1):98-135 136, January 2015.
[43] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3213-3223, 2016.
[44] Samuele Poppi, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Revisiting the evaluation of class activation mapping for explainability: A novel metric and experimental analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2299 22042 2021.
[45] Adam W. Harley, Zhaoyuan Fang, Jie Li, Rares Ambrus, and Katerina Fragkiadaki. Simple- BEV: What really matters for multi-sensor bev perception? In IEEE International Conference on Robotics and Automation (ICRA), 2023.
12

---

[46] Loick Chambon, Éloi Zablocki, Mickaël Chen, Florent Bartoccioni, Patrick Pérez, and Matthieu Cord. Pointbev: A sparse approach bev predictions. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2024, 2024.
[47] Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's s-eee-view representation from multi-camera images via spatiotemporal transformers. In Computer Vision ECCV 2022, 2022.
[48] Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, 2020.
[49] Holger Caesar, Jasper R. Uijlings, and Vittorio Ferrari. Coco-stuff: Thing and stuff classes in context. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1209-1214 1218, 2016.
[50] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficiint exact attention with io-awareness. Advances in neural information processing systems, 35:16344-16359, 2022.
[51] Hanwei Zhang, Felipe Torres, Ronan Sicre, Yannis Avrithis, and Stephane Ayache, Opti-cam: Optimizing saliency maps for interpretability. Computer Vision and Image Understanding. 048:104101, 2024.
[52] MMCV Contributors. MMCV: OpenMML.ab computer vision foundation. https:///ithub. com/ cettammmmlmm/mmmee 2018.
[53] Ross Wightman. Pytorch image models.
attps://ggithub.coom/m//
pytorch-iman sethh-iimmmm-mmeeeee models, 2019.
[54] Haiwen Huang, Anpei Chen, Volodymyr Havrylov, Andreas Geiger, and Dan Zhang. Loftup: Learning a coordinate-baset feature upsampler for vision foundation models, 2025.
[55] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg. Wan-Yer Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015 4026, 2023.
[56] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
[57] Ali Hassani, Fengzhe Zhou, Aditya Kane, Jiannan Huang, Chieh- Yun Chen, Min Shi, Steven Walton, Markus Hoehnerbach, Vijay Thakkar, Michael Isaev, et al. Generalized neighbor- hood attention: Multi-dimensional sparse attention at the speed of light. arXiv preprint a504 16922. 2025.
[58] Xuran Pan, Tianzhu Ye, Zhuofan Xia, Shiji Song, and Gao Huang. Slide-transformer: Hierar- chical vision transformer with local self-attention. Proceedings of the IEEE/CV conference on computer vision and pattern recognition, pages 2082-2091 2023.
13

---

JAFAR:
Jack up Any Feature at Any Resolution
Supplementary Material
A Additional Details
A.1 Evaluation
To evaluate Class Activation Maps (CAMs), employ frozen pre- pre-trained VIT-B/16 model the backbone and extract Grad-CAMs. We randomly sample 2,000 images from the ImageNet validation set for which the model produces correct predictions. For each image, compute the gradients with respect to the predicted class, average them, and use the result to weight the corresponding activation maps. The weighted activations are then summed to produce the final CAM. These activation maps are upsampled from 14 x 14 to 224 x 224, resulting in high-resolution CAMs. For each CAM, generate masked version of the input image by applying binary mask that highlights regions positively associated with the model's 's prediction. Formally, a masked image is obtained as Tmasked (x)20- These masked images then used compute the evaluation metries.
Average Drop Average Drop (A.D) quantifies how much the model's confidence the predicted class decreases when it is presented with the masked image instead of the full image. For single image, the metric is defined as:
max Y 0Ω) 100, Y,
A.D
(4)
N
where Y denotes the model's output score for class c when using the full image, and O denotes the score when using the masked version derived from the explanation map. The final A.D value is computed by averaging over set of images.
Average Increase Average Increase (A.I) measures how often the model's confidence in the predicted class is higher when using the masked image than when using the full image. is defined
N A.I = 100 Σ
Iypso
(5)
N
where Y, is the model's output score for class when using the full image, and Oş is the score when using the masked image based on the explanation map. The metric reflects the percentage of images where the explanation-basse input yields higher confidence score than the original image.
Average Gain [51] Average Gain (A.G) quantifies the improvement in predictive confidence for the target class when using the masked image instead of the full image. It is defined as:
m0 Of -Y;) Y
A.G
100,
N
where Y is the model's output score for class on the full image, and Of is the score when using the masked version derived from the explanation map. This metric captures how much the explanation enhances the model's confidence, normalized by the room for improvement (1 Y,").
14

---

Coherency [44] A Class Activation Map should highlight all the relevant features that contribute to a model's prediction, while suppressing irrelevant ones in a coherent and consistent manner. Consequently, for given input image and a target class c, the CAM should remain unchanged when the image conditioned on the CAM itself. This self-consistency can be expressed as:
C= O CAM₂(*)) = CAM,(x)
where 2 denotes element-wise multiplication. This condition implies that the CAM produced from the masked image should be identical to the original CAM, ensuring that the explanation stable. Following the approach in [44], we use the Pearson Correlation Coefficient between the original CAM and the CAM obtained after masking:
Coherency( (z) = Cov(CAM, @ CAM,(x)), CAM,())
(8)
OCAM,
where Cov denotes the covariance and a the standard deviation each CAM. Since the Pearson Correlation Coefficient ranges from - to 1, we normalize it to the range [0, 1] and express it as percentage for interpretability. A coherency score of 100% indicates that the attribution method is fully invariant to input perturbations guided by its own explanations.
Complexity In addition ensuring that a CAM discarding irrelevant ones-it is also desirable for the CAM to be as simple as possible. That is, it should highlight the minimal subset of pixels necessary to explain the model's prediction. To quantify this notion of simplicity, we use the &o norm proxy for the Complexity of a CAM:
ppeeservi- predictive features while
Complexity (z) = |=)AM,
where ||- counts the number non-zero (i.e., activated) pixels. A lower Complexity score indicates that the attribution method focuses on fewer, more relevant regions, thereby producing more concise and interpretable explanations.
ADCC [44] Since each individual metric captures distinct aspect of CAM quality, we compute an aggregated evaluation metric- Average DCC (ADCC)-whid combines Coherency, Complexity, and Average Drop into single score using the harmonic mean:
ADCC(=)= = (Coherencyx)) + Complexity(z)
1
1
(10)
A.D(2))
ADCC offers unified, single-valued measure that enables direct and consistent comparison. By balancing coherency, sparsity (via low complexity), and confidence preservation (via low Average Drop), provides a more comprehensive assessment of attribution quality.
A.2 Baselines
Large Image: For the Large Image baseline, we upsampled the original image via bilinear upsampling and use it as input the foundation vision encoder. During evaluation downstream tasks (see Tabs. and 2), we upsample the input to the maximum ratio that fits in memory (i.e., x8), and subsequently apply bilinear upsampling to the resulting feature map to match the target output resolution. Due to the high computational cost and training time, omit results the COCO dataset. For efficiency, on Seen-Vocabulary segmentation we limit upsampling to a x 2 ratio in Tab. 3.
Strided: To obtain higher-resolution feature maps, we modify the stride of the ViT backbone to produce more patches. While the stride typically equals the patch size (e.g., 14 in DINOv2), we reduce the former to 6 our experiments corresponding to a x2.3 upsampling. We then apply bilinear upsampling to the resulting feature map to reach the desired output resolution.
CARAFE [22]: For CARAFE, we use the CARAFEPack module from MMCV [52], stacking four upsampling stages with a x2 ratio each, resulting in a final feature map upsampled by factor of x16
SAPA [23]: We adopt the default implementation from the official SAPA repository, stacking four SAPA upsampling modules, each with an upsampling factor of This results final feature map with total upsampling factor of x 16.
15

---

DySample [24]: We adopt the default implementation from the official Dysample repository, stacking four Dysample upsampling modules, each with an upsampling factor of 2. This results in final feature map with total upsampling factor of x 16.
ReSFU [25]: We use the default implementation from the official ReSFU repository, performing a direct upsampling to the target output resolution.
FeatUp [28]: For FeatUp, we use the scalable JBU variant from the official FeatUp resposi- tory stacking 4 upsampling modules to achieve total upsampling factor of x x16. We re-train FeatUp using the provided training scripts. While the original paper trains FeatUp on the COCO dataset for 2,000 steps with batch size of 4 and evaluates on the same dataset, we train on the ImageNet training set for 50,000 steps (25x more) using the same batch size, ensuring fair comparison across methods.
LiFT [27]: For LiFT, slightly adapt the official implementation by resizing the in- termediate representations after the downsampling module to ensure compatibility with backbones using a patch size of 14 (i.e., a downsampling factor of 14). In the official code, LiFT performs x2 upsampling and then relies on a bilinear interpolation to upsample the features to the target output resolution.
We summarize in Tab. 6 the differences between upsamplers.
Table 6: Comparison of feature upsampling methods.
Method
Task-Agnostic
Direct Upsampling
Lightweight Inference
CARAFE -RAPAE-SAPAA-A SAPA -DSSmple
ReSFU
LIFT
X
Fea (BU))
FeatUp (Implicit)
/
B Additional Visualizations
We provide in the following subsections additional comparison visualizations for upsampled feature maps Supp. B.I, class activation maps predictions Supp. B.2, depth estimation Supp. B.3 and semantic segmentation Supp. B.4.
B.1 Feature Visualization
Fig. 5 shows additional PCA visualizations of upsampled feature maps. Starting from 32 x 32 feature maps extracted using a DINOv2-S/14 backbone, each is upsampled to 448 x 448 using different baseline methods. These baselines wwether training-fre free, task-dependent, or task-agnostic-Bee tend to introduce varying degrees of blurriness and visual artifacts. In contrast, JAFAR, while remaining agnossgn, produces sharp, content-aware features with minimal artifacts.
B.2 Class Activation Maps
We present additional Grad-CAM visualizations based ViT-B/16 features from the ImageNet validation set in Fig. 6. Except for the "Low-Res" column, where features remain at their original 1 14x14 resolution, all feature maps upsampled to 224x224 before Grad-CAM extraction. The explainability maps generated by our upsampling approach are noticeably sharper and more accurate, exhibiting fewer artifacts compared to those from alternative methods. Notably, CARAFE and LiFT fail to produce meaningful explanations in this setting, suggesting that the training of these methods does not transfer effectively these ViT-based features.
B.3 Depth Estimation
Fig. 7 presents additional examples of linear probe transfer learning for depth estimation on the COCO-Stuff dataset. Feature maps of size 32 x 32, extracted from DINOv2-S/14 backbone, are upsampled to 448 448 using the various baseline methods. A lincar probe is then trained on
16

---

Low Low-Ras
Biliner
Imagelmmage
Strided
GARAFE
SAPA
DySample
ReSFU
LIFT
FootUp
JAFAR
Training Free
Task Dependent
Task Agsostic
Figure 5: PCA Feature Visualization. DINOv2 ViT-S/14 features at 32 x 32 resolution from the ImageNet validation set upsampled to 448 448.
Large Image
DySample
FeetUp
Mica Cocodd
Courd Goads
Figure 6: Class Activation Maps comparison.
these features to predict depth, using supervision from a Depth-AnythingW model. The results demonstrate that both FeatUp variants produce high-quality features well-suited for transfer learning in depth estimation tasks.
Bi inser
Lange Image
Stiike
CARAFE
SARA
DySomple
RaSFU
LFFT
FeetUp
JAFAR
Geaund-Tath
Depeolnn
Figure 7: Depth Estimation Visualization.
17

---

B.4 Semantic Segmentation
Fig. 8 presents examples linear probe transfer learning for semantic segmentation on the COCO- Stuff dataset. Feature maps of size 32 32, extracted from DINOv2-S/14 backbone, upsampled to 448 x 448 using the various baseline methods. JAFAR produces more coherent segmentation results, offering improved delineation of both object boundaries and background regions.
Ree
Stride
CARAFE
SARA
Dysammle
RASFU
FeatUp
Ground
Trahinn Free
Tak Depesdan
Tark Agannic
Figure Semantic Segmentation Visualization.
B.5 Attention Maps Visualization
To illustrate the behavior the upsampling module, we visualize attention maps in Fig. 9. For each example, query location is selected, and its attention distribution over the low-resolution keys is shown. The maps reveal that attention tends to concentrate on the region semantically related to the query, even though all keys accessible. This suggests that fully global attention may be strictly necessary, and that localized variants could provide significant runtime and memory gains. Nevertheless, we argue that the mechanism should not be overly local, since feature maps produced by vision encoders often suffer from spatial misalignments and positional artifacts. Allowing each query broader receptive field helps counteract these inconsistencies and improves upsampling quality, Conceptually, this resembles the principle of non-local means in image denoising, where pixel is refined using information from wider neighborhood rather than only its immediate surroundings.
Figure 9: Attention maps from JAFAR. Each map shows the attention distribution of query location (white point) over the low-resolution keys.
C
Additionnal Comparisons With Task-Agnostic Baselines
C.1 FeatUp & LiFT
In Tab. 1, we reported results obtained by training FeatUp and LiFT within our own codebase. To complement this evaluation, we present an additional comparison in Tabs. 8 and 10 using the official released checkpoints, FeatUp on DINOv2 ViT-S/14 and LiFT on DINO ViT-S/16, respectively.
Across all datasets and output resolutions, with the exception of ADE20K at 448 for FeatUp. JAFAR consistently delivers significant improvements semantic segmentation performance. We also evaluate LiFTTiterative baseline, which stacks two LiFT x2 upsamplers described in the original LiFT paper. However, this iterative approach does not outperform the simpler method of applying single LiFT x2 upsampler followed by bilinear interpolation.
18

---

Table Semantic Segmentation Comparison between JAFAR and FeatUp. JAFAR is evaluated using FeatUp's original feature extractor and compared against the official JBU checkpoint on the DINOv2 ViT-S/14 backbone.
224
448
Model
ADE20K
Cityscapes
VOC
ADE20K
Cityscapes
VOC
Bilinear
24.50
38.18
64.10
28.19
49.35
70.64
FeatUp
29.26
42.84
71.90
32.87
53.17
77.57
JAFAR
30.04
48.52
75.36
32.29
56.45
79.06
Table 8: Semantic Segmentation Comparison between JAFAR and LiFT.
JAFAR is evaluated using LiFT's original feature extractor and compared against the checkpoint the DINO ViT-S/16 backbone.
224
448
Model
ADE20K
Cityscapes
VOC
ADE20K
Cityscapes
VOC
Bilinear 15.89
32.52
32.15
16.44
36.56
33.13
LiFT 16.97
36.31
35.30
16.98
40.01
35.07
LiFT-iterative
15.62
37.68
31.25
14.96
39.97
29.43
JAFAR
21.08
39.93
47.36
21.44
43.93
48.18
C.2 LoftUp
We present an additional comparison with LoftUp [54], recent baseline which relies on segmentation masks from SAM [55] during training and employs two-stage pipeline that includes a self distilillatto phase. For a fair evaluation, we tested JAFAR within LoftUp's 's official codebase using DINOv2 ViT-S/14 backbone.
Table 9: Semantic Segmentation Comparison between JAFAR and LiFT. JAFAR is evaluated using Loftup's original feature extractor and compared against the check- point on the DINOv2 ViT-S/14 backbone.
Cityscapes
COCO
Resolution
mloU (f)
Acc (1)
mloU (1)
Ace (t)
LoftUp
15.30
75.50
25.33
54.06
562
JAFAR
19.09
79.34
28.79
57.86
JAFAR + distillation
18.35
79.05
28.60
57.49
LoftUp
32.02
86.28
48.89
73.09
1122
JAFAR
34.56
87.19
50.67
74.56
JAFAR + distillation
33.63
87.47
50.76
74.60
LoftUp
50.83
91.55
59.79
80.04
224
JAFAR
51.45
91.25
59.76
79.93
JAFAR distillation
51.84
91.53
59.90
80.04
LoftUp
62.49
93.69
62.25
81.43
4483
JAFAR
61.49
93,46
62.02
81.30
JAFAR + distillation
62.30
93.76
62.36
81.45
As shown in Tab. 9, JAFAR outperforms LoftUp at lower upsampling resolutions (56 and 112) and delivers comparable performance higher resolutions (224 and 448). Different from LoftUp, JAFAR uses a simpler and more efficient single-stage strategy: it operates entirely at low resolution and does not rely on external annotations. Nevertheless,
self-distillation mechanism introduced in LoftUp is complementary to approach and can be seamlessly integrated into JAFAR's pipeline. To demonstrate this, we implemented a similar distillation objective and report the results as JAFAR + distillation in Tab. 9, While the gains are minimal lower resolutions, this enhancement provides
19

---

clear boost at higher resolutions (224 and 448). Lastly, JAFAR is considerably more lightweight, with only 700K parameters compared to 4.3M in LoftUp.
D Additional Analysis
D.1 Upsampling Ratios
To evaluate the robustness of JAFAR upsampling across different ratios, we measure its performance on linear probing semantic segmentation task at multiple scales, including extreme ratios (8² - 8962). The results indicate that JAFAR consistently sustains strong performance as the upsampling ratio grows, underscoring its ability to generalize well beyond the training range.
Table 10: JAFAR robustness across upsampling scales on linear probing segmentation.
Input Resolution
Dataset
Upsampler 562
1122
224
4482
8962
Bilinear
31.87
31.81
31.90
31.61
31.92
Cityscapes
JAFAR
36.54
36.04
35.71
35.77
35.54
82
Bilinear
64.79
64.95
64.89
64.91
64.9
vOC
JAFAR
72.7
73.13
72.77
72.69
72.55
Bilinear
47.77
47.77
47.78
47.83
47.84
Cityscapes
JAFAR
53.18
53.22
52.35
51.76
51.36
162
Bilinear
75.41
75.42
75.51
75.50
75.50
VOC
JAFAR
80.57
80.76
80.58
80.36
80.24
Bilinear
59.56
59.53
59.41
59.54
59.15
Cityscapes
JAFAR
59.65
61.39
61.38
60.86
60.79
32
Bilinear JAFAR
80.64
80.72
80.77
80.83
80.82
VOC
81.51
82.53
82.67
83.55
83.24
D.2 Positional Encoding
As shown in Tab. 11, RoPE plays critical role in maintaining consistent spatial alignment between queries and keys across varying spatial resolutions, with benefits becoming more pronounced at higher resolutions.
Table 11: Ablation of RoPE on linear probing segmentation (mloU).
Cityscapes
VOC
JAFAR
562
112
224
4482
562
1122
2242
4482
w/o RoPE
18.15
27.06
34.80
36.09
35.36
62.38
67.58
67.22
w/ RoPE
21.41
35.80
52.42
60.86
40.41
72.78
80.57
83.55
E
Performance
We compare in Tabs. 12 and 13 the runtime and memory usage respectively of various methods with a batch size and input resolution of 448, across multiple target resolutions. The experiments are conducted on single A100 GPU.
20

---

Table 12: Runtime (ms) for different models and resolutions.
Model
# Params (M)
56
112
221"
448
Upsamplers
FeatUp
8.0
14.9
64.9
JAFAR
5.7
94.0
LiFT
1.2 4.3
0.9
0.9
1.0
1.5
LoftUU
3.8
8.9
24.5
145.1
Other configurations
Large Image (x8)
6.2
34.7
348
5 5558
Strided (1)
11.4
136.0
2 482
48 090
Table 13: Memory usage (GB) for different models and resolutions.
Pass
Model
# Params (M)
562
1122
224²
4482
Bilinear
0.0
0,4
0.5
0.5
0.8
Forward
FeatUp
0.2
0.6
0.8
1.6
4.8
JAFAR
0.7
0.6
0.6
1.1
7.7
LoftUp
4.3
0.6
0.7
1.8
12.3
FeatUp
0.2
0.7
0.9
2.1
7.4
Backward
JAFAR
0.7
0.7
1.1
3.5
26.0
LoftUp
4.3
0.7
1.3
4.6
26.5
In Tab. 14, report GFLOPs and training memory comparisons for the BeV segmentation setup. JAFAR delivers 6%-1%% 15% higher loU while requiring only 3% 10% more GFLOPs than the version without upsampling, underscoring the trade-off between accuracy and computation.
Table 14: Comparison of upsampling methods on BeV segmentation pipelines.
Model
Metric
No Upsampling
Bilinear
FeatUp
JAFAR
LoftUp
Memory (MiB)
4539
4981
5305
5453
5305
S [45]
GFLOPs
594.00
595.10
595.73
612.24
621.06
mloU
31.75
33.67
33.95
36.59
Memory (MiB)
1507
1833
1961
2521
2397
PointBeV [46]
GFLOPs
154.30
154.32
154.90
171.41
180.25
mloU
34.89
36.01
35.38
37.20
Memory (MiB)
2143
2489
2639
3135
2971
BeVFormer [47]
GFLOPs
352.40
354.70
357.72
376.74
379.82
mloU
33.72
34.18
34.01
36.54
F
Limitations
Although our method offers significant advantages over existing upsampling approaches, also has certain limitations. Because our approach employs global attention mechanism, each query attends to all keys, and the number of keys increases with the resolution of the input image processed by the foundation vision encoder. As result, the computational and memory costs become prohibitive when dealing with large key sets (e.g., 64 x 64 or higher). While large receptive field is desirable, attending to every key is not strictly necessary. Alternative attention mechanisms with more localized receptive fields [56, 57, 58] may achieve prders-of-magnitude efficiency improvements and represent promising direction for future work.
21

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our experiments reflect the scope of the paper and the claims made in the abstract and in the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We include in the conclusion two limitations. The method requires to train an upsampler per backbone and some artifacts are still remaining in the upsampled feature maps.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
22

---

Answer: [NA].
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide detailed description of the method in Sec. 3 well as the training dataset and hyperparameters for the method in cref Sec. 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The datasets used in the experiments publicly available. Code will be released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips...
onooonnlooollll icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wctpublic//aaccccccccccccccc /gwpides///////eeeeee /Coeee/mmdSsbmii SarteeSubmissioons ssionPolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detail in section Sec. the datasets splits used evaluate the method as well as the value of default hyper- parameters, optimizer and image shapes.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: No eπrоr bars are reported as standard validation sets are used to evaluate the method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
24

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mention in Sec. 4 the type of graphic card used to run the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //tpurip///urip...... http//eurrppcccppulliieecc ittpeiieeeccccaciiillliieeeeee
Answer:
[Yes]
Justification:
We reviewed the Code of Ethics and conformed to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification:
We do not deem that this work has negative societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
25

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not deem that this paper poses such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Authors of used assets (data) in partieular, credited in the paper. Code will be released and original owners of assets will be properly credited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
26

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Code will be released and documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA].
Justification: This paper does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
27

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The method development does not involve LLMs as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: /ttpr//ps..//ooofeeeeee////2255/ for what should or should not be described.
28


Reviews:
[{"review_id": "dB68HIEwKA", "reviewer": "Reviewer_hVFB", "summary": "The paper propose a lightweight and flexible feature upsampler designed to enhance the spatial resolution of visual features from any Foundation Vision Encoder to any target resolution. JAFAR features an attention-based upsampling module that aligns the spatial representations of high-resolution queries with semantically enriched low-resolution keys via Spatial Feature Transform modulation.", "questions": "see weakness", "limitations": "no", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThis paper proposes an interesting approach that enables adaptive super-resolution for arbitrary resolutions. The use of RoPE to generalize to unseen resolutions, combined with attention mechanisms and zero-shot training, is quite innovative.\n\nCompared to previous methods, the proposed approach not only addresses the challenge of arbitrary-resolution upsampling but also achieves superior performance.\n\nWeaknesses:\n\nI would like to see an ablation study to evaluate whether RoPE indeed improves the model's generalization ability to different spatial resolutions.\n\nThe concept of zero-shot super-resolution has been explored as early as 2017 in the SR community (e.g., ZSSR [1]). While not entirely new, applying this idea to a new super-resolution task can still be considered a modest contribution.\n\n[1] “Zero-Shot” Super-Resolution using Deep Internal Learning", "was_revised": true, "final_justification": "My concerns are addressed. I keep my positive score.", "rebuttal": "We thank the reviewer for the helpful comments and appreciate the positive feedback. Please note that, for the purpose of the rebuttal, all probing layers of the following tables were trained for 10 epochs (instead of 20, as reported in the main paper) to accelerate training while preserving fair comparison.\n\n**1. I would like to see an ablation study to evaluate whether RoPE indeed improves the model's generalization ability to different spatial resolutions.**\n\nIn response to the reviewer’s recommendation, we provide the following ablation study to evaluate the impact of RoPE on JAFAR’s generalization ability in semantic segmentation probing across multiple spatial resolutions $(56, 112, 224, 448)$. To ensure consistency, the upsampling factor is fixed ($\\times 14$) across all resolutions.\n\n| **Cityscapes** (mIoU)  | 56    | 112   | 224   | 448   |\n|----------------|-------|-------|-------|-------|\n| JAFAR w/ RoPE  | 21.41 | 35.8  | 52.42 | 60.86 |\n| JAFAR wo/ RoPE | 18.15 | 27.06 | 34.80 | 36.09 |\n\n\n|     **VOC**  (mIoU)   |   56  |  112  |  224  |  448  |\n|:--------------:|:-----:|:-----:|:-----:|:-----:|\n|  JAFAR w/ RoPE | 40.41 | 72.78 | 80.57 | 83.55 |\n| JAFAR wo/ RoPE | 35.36 | 62.38 | 67.58 | 67.22 |\n\nThis ablation study highlights that RoPE plays a critical role in maintaining consistent spatial alignment between queries and keys across varying spatial resolutions, with its benefits becoming more pronounced at higher resolutions. We will include this ablation in the supplementary material to underscore the critical role of RoPE in enabling effective generalization to varying spatial resolutions. \n\n\n**2. The concept of zero-shot super-resolution has been explored as early as 2017 in the SR community (e.g., ZSSR [1]). While not entirely new, applying this idea to a new super-resolution task can still be considered a modest contribution.**\n\nWhile we acknowledge that our approach shares a conceptual similarity with ZSSR in image super-resolution (i.e using a downsampled version of the target to predict the target) we emphasize several important differences, through which we introduce what we believe are significant contributions.\n\nA key distinction lies in the nature of the target: while high-resolution images are easy to obtain, acquiring high-resolution feature maps is significantly more challenging. While ZSSR typically downsamples images by modest factors, modern vision encoders reduce spatial resolution much more aggressively, by factors of $14$, $16$, or even $32$. This discrepancy severely limits the range of scaling factors that can be simulated during training, making it much harder to match the large upsampling ratios needed at inference time.\nOne might consider increasing input image resolution to produce higher-resolution feature maps. However, this approach introduces substantial problems: high-resolution inputs push the vision encoder far outside its training distribution, and although resizing positional embeddings can compensate to some degree, it quickly reaches its limits (l.34 [1, 2]). We evaluated this approach as the **Large Image** baseline in Tables 1–3 and Figure 4, which consistently underperforms compared to the Bilinear Upsampling baseline. \n\nDespite these limitations, our experiments (Tables 1–4, Figure 4) consistently show that training the proposed cross-attention architecture at low upsampling ratios (between $\\times 2$ and $\\times 4$) and resolutions leads to strong generalization at much higher scales ($\\times 14$). This robust behavior, both surprising and significant, represents a key empirical contribution of our work.\n\n\n[1] Yang, Jiawei, et al. \"Denoising vision transformers.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n\n[2] Fan, Qihang et al. “ViTAR: Vision Transformer with Any Resolution.” ArXiv abs/2403.18361 (2024): n. pag."}, {"review_id": "IfB15ds0R7", "reviewer": "Reviewer_ne7u", "summary": "In this paper, the authors propose a lightweight and flexible feature upsampler named JAFAR. It employs an attention-based structure to upsample features from any base visual encoder to arbitrary output resolution. The authors evaluate the performance of JAFAR on a diverse set of downstream tasks.", "questions": "See weaknesses.", "limitations": "The paper lacks an analysis of the limitations of its research.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n(1) The authors propose a task-independent, lightweight, and training-friendly feature upsampling method that can enhance visual features at any resolution, breaking through the limitations of existing methods in resolution, generalization, and interpretability.\n\n(2) The authors compare multiple existing upsampling methods and conducted empirical comparisons in multiple tasks (classification, depth estimation, semantic segmentation) and at different resolutions, demonstrating the performance advantages of JAFAR.\n\n(3) JAFAR achieve a tradeoff between parameter quantity and inference time, with only 0.7M parameters and low computational cost at most output resolutions, making it suitable for practical deployment.\n\n(4) The paper is well-written and well-organized. \n\n**Weaknesses:**\n\n(1) Although the method works well, the authors lack theoretical derivation and explanation of the JAFAR upsampling mechanism, and do not clearly explain why its structure can improve spatial resolution while retaining semantic information.\n\n(2) The paper's method can achieve different upsampling ratios. Will model performance deteriorate as the upsampling ratio increases?\n\n(3) Can the paper's method be further extended to image super-resolution tasks?", "was_revised": true, "final_justification": "The authors have fully addressed my concerns during the author response phase. Therefore, I have raised my rating from 4 to 5.", "rebuttal": "We thank the reviewer for the helpful comments and appreciate the positive feedback. Please note that, for the purpose of the rebuttal, all probing layers of the following tables were trained for 10 epochs (instead of 20, as reported in the main paper) to accelerate training while preserving fair comparison.\n\n\n**1. Although the method works well, the authors lack theoretical derivation and explanation of the JAFAR upsampling mechanism, and do not clearly explain why its structure can improve spatial resolution while retaining semantic information.**\n\nWhile our method is primarily empirical, its core design shares conceptual similarities with non-local means filtering used in image denoising. In non-local means, each pixel is denoised by aggregating information from all other pixels in the image, weighted by their similarity to the target pixel, rather than relying solely on local neighborhoods, as in traditional filters. This principle of leveraging semantically similar but spatially distant information is directly applicable to our setting and helps explain the effectiveness of our approach. We will include this analysis in the revised version of the paper.\n\nIn the context of feature upsampling, this is particularly useful because low-resolution feature maps produced by vision encoders often suffer from spatial misalignments and positional artefacts. Traditional local upsampling methods like bilinear interpolation simply propagate these inconsistencies to higher resolutions without correcting them.\n\nJAFAR addresses this limitation by learning a similarity-based weighting scheme via a cross-attention mechanism, where high-resolution queries, derived from the original image, attend to low-resolution keys from the same image encoding, semantically enriched through the SFT module. As shown in Table 5, this design greatly enhances the semantic relevance of the learned similarities. In contrast, the **Linear Projection** baseline, where queries are taken from a shallow image encoding and keys directly from the semantic feature map, performs noticeably worse. This is likely due to the mismatch between the two representation spaces, which hinders the quality of the computed similarities. Our proposed mechanism enables each high-resolution query location to dynamically gather information from semantically aligned key regions, effectively mimicking non-local means in a more flexible, learned fashion. \n\nTo further highlight the value of the **SFT mechanism**, we will include the following line in Table 5 demonstrating the performance gain from semantic feature injection. We will re-run the experiment with 20 epochs to include it in the main paper.\n\n|               | Cityscapes | VOC   | ADE20K |\n|---------------|------------|-------|--------|\n| JAFAR         | 60.86      | 83.55 | 41.04  |\n| JAFAR wo/ SFT | 56.76      | 82.54 | 40.27  |\n\n**2. The paper's method can achieve different upsampling ratios. Will model performance deteriorate as the upsampling ratio increases?**\n\nTo evaluate the robustness of JAFAR across varying upsampling ratios, we conducted an experiment (shown in the tables below) assessing its performance at multiple scales, including extreme ratios. The results demonstrate that JAFAR maintains strong performance even as the upsampling ratio increases, confirming its ability to generalize well beyond the training range. We will include these tables in the supplementary material to further support the method’s robustness and scalability.\n\n**Input Feature Size: 8**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 7$) | 112 ($\\times 14$) | 224 ($\\times 28$) | 448 ($\\times 56$) | 896 ($\\times 112$) |\n|:---------------------:|:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|\n|        Bilinear       |      31.87      |       31.81       |       31.90       |       31.61       |        31.92       |\n|         JAFAR         |      36.54      |       36.04       |       35.71       |       35.77       |        35.54       |\n\n| **VOC** (mIoU) | 56 ($\\times 7$) | 112 ($\\times 14$) | 224 ($\\times 28$) | 448 ($\\times 56$) | 896 ($\\times 112$) |\n|:--------------:|:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|\n|    Bilinear    |      64.79      |       64.95       |       64.89       |       64.91       |        64.9        |\n|      JAFAR     |       72.7      |       73.13       |       72.77       |       72.69       |        72.55       |\n\n\n**Input Feature Size: 16**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 3.5$) | 112 ($\\times 7$) | 224 ($\\times 14$) | 448 ($\\times 28$) | 896 ($\\times 56$) |\n|:---------------------:|:-----------------:|:----------------:|:-----------------:|:-----------------:|:-----------------:|\n|        Bilinear       |       47.77       |       47.77      |       47.78       |       47.83       |       47.84       |\n|         JAFAR         |       53.18       |       53.22      |       52.35       |       51.76       |       51.36       |\n\n| **VOC** (mIoU) | 56 ($\\times 3.5$) | 112 ($\\times 7$) | 224 ($\\times 14$) | 448 ($\\times 28$) | 896 ($\\times 56$) |\n|:--------------:|:-----------------:|:----------------:|:-----------------:|:-----------------:|:-----------------:|\n|    Bilinear    |       75.41       |       75.42      |       75.51       |       75.50       |       75.50       |\n|      JAFAR     |       80.57       |       80.76      |       80.58       |       80.36       |       80.24       |\n\n\n**Input Feature Size: 32**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 1.75$) | 112 ($\\times 3.5$) | 224 ($\\times 7$) | 448 ($\\times 14$) | 896 ($\\times 28$) |\n|:---------------------:|:------------------:|:------------------:|:----------------:|:-----------------:|:-----------------:|\n|        Bilinear       |        59.56       |        59.53       |       59.41      |       59.54       |       59.15       |\n|         JAFAR         |        59.65       |        61.39       |       61.38      |       60.86       |       60.79       |\n\n| **VOC** (mIoU) | 56 ($\\times 1.75$) | 112 ($\\times 3.5$) | 224 ($\\times 7$) | 448 ($\\times 14$) | 896 ($\\times 28$) |\n|:--------------:|:------------------:|:------------------:|:----------------:|:-----------------:|:-----------------:|\n|    Bilinear    |        80.64       |        80.72       |       80.77      |       80.83       |       80.82       |\n|      JAFAR     |        81.51       |        82.53       |       82.67      |       83.55       |       83.24       |\n\n\n**3.  Can the paper's method be further extended to image super-resolution tasks?**\n\nWe have not applied our method to image super-resolution tasks, so we cannot confidently assess its effectiveness in that domain. However, upsampling latent representations produced by super-resolution pipelines that operate in a latent space [1] might be feasible. For example, we could think of applying JAFAR to the latents of a pre-trained VAE, then decoding them to obtain higher-resolution images. That said, this adaptation would likely require significant modifications, including the use of more localized cross-attention mechanisms and the introduction of image-specific loss terms. Given these considerations, we view this as a non-trivial but promising direction for future research.\n\n**4. The paper lacks an analysis of the limitations of its research.**\n\nThank you for pointing this out. We briefly mention some limitations in the conclusion but we will expand on the existing discussion in the main text and include a dedicated limitations section in the supplementary material. This section will specifically address the computational cost and necessity of global versus local cross-attention mechanisms, incorporating the points raised during the rebuttal.\n\n[1]Chen, Yinbo, et al. \"Image neural field diffusion models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024."}, {"review_id": "OdWPJy5Yt4", "reviewer": "Reviewer_AAhc", "summary": "This paper presents JAFAR, a light and trainable feature upsampler that can upsample visual features from any foundational vision encoder to an arbitrary target resolution. JAFAR employs an attention-based module to promote semantic alignment between high-resolution queries and semantically enriched low-resolution keys using Spatial Feature Transform (SFT) modulation. This paper also demonstrates that training JAFAR with low upsampling ratios and resolutions can generalize well to significantly higher output scales. Experiments show JAFAR can recover fine-grained spatial details and consistently outperforms existing feature upsampling methods across a diverse set of downstream tasks.", "questions": "Referring to the Weaknesses section, I have the following questions: \n\n1. Why does global interpolation perform better in upsampling tasks? Could other parametrized upsamplers also work effectively within the JAFAR framework?\n\n2. What's the real-world use case of high-resolution JAFAR features (*e.g.*, at original image size)? Can JAFAR be integrated into end-to-end training or finetuning pipelines? What's the overall additional computational cost in these cases?\n\nI will adjust my rating according to the author's response.", "limitations": "Limitations are briefly discussed in the Conclusion section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### **Strengths**\n\n1. The writing is smooth and easy to follow.\n2. The proposed JAFAR is both architecture and task-agnostic, which can be easily applied to various vision foundation models and downstream tasks.\n3. Comprehensive qualitative comparisons illustrate JAFAR's superior performance over other upsampling methods.\n\n### **Weaknesses**\n\n1. **The contribution of this paper is unclear.** This paper follows FeatUp [1] to train a model and task-agnostic framework for feature upsampling to any resolution. Key components such as feature consistency loss and similarity-based upsampling have been well-explored in prior works [2, 3], and the major contribution is the global interpolation mechanism using cross-attention. However, the motivation and necessity of using **global interpolation** for feature upsampling is unclear, since local structures are generally more important for upsampling while considering only local structures can also help reduce computational costs.\n2. **Concern of practicality for stand-alone feature upsamplers**. While JAFAR demonstrates improved performance over other (training-free, task-dependent and task-agnostic) upsampling approaches in linear probing settings, it's often impractical to use high-resolution features directly in downstream tasks. More evaluations and analysis beyond the linear probing settings are necessary. What's the advantage of stand-alone feature upsamplers over integrated upsampling modules?\n\nMinor remark: I suggest that the author carefully review the equations in Section 3.1 to ensure that the vector dimensions are consistent for multiplication (Equation (1)), and use different notations for matrix multiplication and element-wise operations (Equations (1) and (2)).\n\n\n\n[1] Fu, Stephanie, et al. \"FeatUp: A Model-Agnostic Framework for Features at Any Resolution.\" *The Twelfth International Conference on Learning Representations*.\n\n[2] Yang, Jiawei, et al. \"Denoising vision transformers.\" *European Conference on Computer Vision*. Cham: Springer Nature Switzerland, 2024.\n\n[3] Lu, Hao, et al. \"SAPA: Similarity-aware point affiliation for feature upsampling.\" *Advances in Neural Information Processing Systems* 35 (2022): 20889-20901.", "was_revised": true, "final_justification": "The authors have addressed my major concerns during the rebuttal.", "rebuttal": "We thank the reviewer for the helpful comments and appreciate the positive feedback. Please note that, for the purpose of the rebuttal, all probing layers of the following tables were trained for 10 epochs (instead of 20, as reported in the main paper) to accelerate training while preserving fair comparison.\n\n**Weaknesses**\n\n**1. The contribution of this paper is unclear. This paper follows FeatUp [1] to train a model and task-agnostic framework for feature upsampling to any resolution.**\n\nWe would like to clarify that FeatUp does not support feature upsampling to arbitrary resolutions. As noted in lines 97–101, the implicit variant of FeatUp can upsample feature maps only to the original image resolution and requires training a separate upsampler for each image, limiting its generality. The JBU variant, in contrast, stacks $k$ JBU upsampling blocks and is thus limited to a fixed upsampling factor equal to a $2^k$. Our method, however, is the only baseline that supports continuous, task-agnostic feature upsampling to any intermediate resolution between the original feature map and the image resolution.\n\n**2. Key components [...] computational costs.**\n\nWe would like to clarify both the role of the attention mechanism in our method and the contributions beyond prior work.\n\n**Cross-attention for continuous-resolution upsampling:**\n\nWhile similarity-based upsampling has been explored in prior works such as [1], our use of a cross-attention mechanism enables direct control over the output resolution through the spatial dimensions of the queries, determined by the pooling mechanism. This allows feature upsampling not only to the original image resolution but also to arbitrary intermediate resolutions in a task-agnostic manner, a capability not demonstrated by previous methods.\n\n**Importance of SFT:**\n\nAs demonstrated in Tab 5, a key contribution of our method is the integration of the SFT module, which softly injects semantic information into the keys. This mechanism enhances spatial resolution while preserving important semantic content from the low-resolution features.\n\n**Architectural flexibility and generalization:**\n\nThanks to the pooling mechanisms, we are able to train the model at low upsampling ratios and resolutions, while achieving strong generalization to significantly higher upsampling ratios at inference time, demonstrating the robustness and flexibility of our proposed architecture. This robust behavior represents a key empirical contribution of our work.\n\nRegarding the global nature of the cross-attention mechanism, we agree that attending to all keys is not strictly necessary. Attention variants with more localized receptive fields [2, 3, 4], may offer efficiency gains and could be explored in future work. That said, we deliberately avoid overly local mechanisms, as feature maps produced by vision encoders often exhibit spatial misalignments or positional artefacts. Granting each query a larger receptive field via global attention helps mitigate these inconsistencies and improves upsampling quality. Conceptually, this could be related to non-local means in image denoising, where a pixel is refined using information from a broader neighborhood rather than just immediate surroundings.\n\n**3. Concern of practicality for stand-alone feature upsamplers. [...]**\n\nPlease Refer to **Question 3** and **Question 4** below.\n\n**4. Minor remark**\n\nWe thank the reviewer for pointing this out. We will revise Section 3.1 to ensure that all tensor dimensions are consistent and clearly defined in each equation. In particular, we will explicitly indicate the reshaping of query and key tensors before the dot-product in Equation (1) and clarify the distinction between matrix multiplication and element-wise operations by using a $\\odot$ symbol to denote element-wise operation in equation 2).\n\n**Questions**\n\n**1. Why does global interpolation perform better in upsampling tasks?**\n\nFeature maps produced by vision encoders often suffer from imperfections such as spatial misalignments (semantics does not align cleanly with object boundaries), and positional artefacts. Local interpolation methods such as bilinear upsampling are insufficient to correct these issues, as they lack the capacity to incorporate broader contextual information and simply propagate these inconsistencies to higher resolutions without correcting them. In contrast, our use of global attention gives each query access to a wide receptive field, allowing it to integrate information from semantically similar regions across the feature map. This helps mitigate spatial inconsistencies and enhances upsampling quality.\n\n**2. Could other parametrized upsamplers also work effectively within the JAFAR framework?**\n\nA key strength of JAFAR lies in the flexibility of its architectural design, which enables a clear decoupling between training and inference, a feature not commonly found in many existing upsampling approaches. Specifically, the use of adaptive pooling modules plays a dual role:\n- During training, these modules allow supervision at low resolution, thereby avoiding the need for high-res ground truth features, which are typically unavailable.\n- At inference, they support generalization to unseen upsampling factors by adjusting the spatial resolution dynamically.\n\nExisting upsamplers lack this level of flexibility. For example, FeatUp cannot dynamically adjust its output resolution and must be explicitly trained to generate high-res features, followed by learning a separate downsampler to match low-resolution supervision. LiFT, on the other hand, follows a training pipeline similar to JAFAR’s, but its CNN-based architecture inherently limits its ability to generalize to unseen upsampling factors.\n\nIn response to the reviewer’s request, and to demonstrate that JAFAR outperforms baselines within the same training framework, we train them to perform $\\times 2$ upsampling and evaluate them in a linear probing segmentation setup, with a subsequent bilinear upsampling applied at inference.\n\n| **Cityscapes** (mIoU) | 56    | 112   | 224   |\n|-----------------------|-------|-------|-------|\n| JAFAR                 | 18.68 | 33.72 | 52.02 |\n| LiFT                  | 18.60 | 31.74 | 47.57 |\n| FeatUp                | 18.48 | 31.67 | 48.29 |\n| SAPA                  | 10.20 | 23.16 | 36.96 |\n\n| **VOC** (mIoU) | 56    | 112   | 224   |\n|----------------|-------|-------|-------|\n| JAFAR          | 35.73 | 68.81 | 79.90 |\n| LiFT           | 31.57 | 64.33 | 73.69 |\n| FeatUp         | 33.44 | 64.23 | 75.1  |\n| SAPA           | 20.14 | 53.72 | 69.29 |\n\n**3 What's the real-world use case of high-resolution JAFAR features [...] ?**\n\nHigh-res features have clear real-world utility in zero-shot dense prediction tasks, such as open-vocabulary semantic segmentation and object localization. Several recent works [5, 6, 7] rely on dense visual features from frozen vision encoders. To match the resolution of the input image, these methods often require an upsampling step, and each currently adopts its own upsampling strategy (e.g., bilinear upsampling, stride modifications, or multi-crop inference). JAFAR offers a simple, effective, and task-agnostic alternative that generalizes across resolutions and tasks. We believe it could serve as a unified upsampling module for these and future zero-shot pipelines, improving both performance and design simplicity. Furthermore, many end-to-end pipelines leverage high-res features for improved performance [8, 9].\n\n**4. Can JAFAR be integrated into end-to-end training or finetuning pipelines? What's the overall additional computational cost in these cases?**\n\nWe would like to highlight Section 4.3.5 (Table 4) of the paper, where JAFAR is integrated into three different BeV end-to-end vehicle segmentation pipelines. In these experiments, JAFAR is trained jointly with the rest of the architecture and consistently outperforms the baselines. This demonstrates that JAFAR can be effectively incorporated into end-to-end training or fine-tuning workflows, leading to clear performance improvements. In summary, JAFAR is both practical and versatile: it can function as a plug-and-play component in zero-shot pipelines or be integrated into end-to-end training setups, making it a valuable contribution for a broad range of vision applications.\n\n[1] Lu, Hao, et al. \"SAPA: Similarity-aware point affiliation for feature upsampling.\" Advances in Neural Information Processing Systems 35 (2022): 20889-20901.\n\n[2] Liu, Ze, et al. \"Swin transformer: Hierarchical vision transformer using shifted windows.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[3] Hassani, Ali, et al. \"Neighborhood attention transformer.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[4] Pan, Xuran, et al. \"Slide-transformer: Hierarchical vision transformer with local self-attention.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[5] Mukhoti, Jishnu, et al. \"Open vocabulary semantic segmentation with patch aligned contrastive learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[6] Couairon, Paul, et al. \"Diffcut: Catalyzing zero-shot semantic segmentation with diffusion features and recursive normalized cut.\" Advances in Neural Information Processing Systems 37 (2024): 13548-13578.\n\n[7] Jose, Cijo, et al. \"Dinov2 meets text: A unified framework for image-and pixel-level vision-language alignment.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[8] Li, Kaiyu, et al. \"Segearth-ov: Towards training-free open-vocabulary segmentation for remote sensing images.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[9] Zhu, Junzhe et al. “DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo.” ArXiv abs/2412.05268 (2024): n. pag."}, {"review_id": "nIu3ArMvTI", "reviewer": "Reviewer_RrSa", "summary": "This paper proposes a novel feature upsampling module named JAFAR, which can upsample the low-resolution output features from any foundation vision encoder (such as DINOv2 or CLIP) to arbitrary high resolutions, enabling downstream pixel-level tasks such as semantic segmentation, depth estimation, and CAM visualization. Unlike existing task-dependent or fixed-ratio task-agnostic upsampling methods, JAFAR designs a heterogeneous Query-Key alignment strategy based on a cross-attention mechanism, where the Query retains the high-resolution texture information from the original image, and the Key is modulated from low-resolution semantic features via an SFT module. Experiments demonstrate the effectiveness of the proposed method.", "questions": "See Weakness", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\nJAFAR provides a unified and efficient feature upsampling solution for multiple dense vision tasks. It consistently improves performance without relying on downstream supervision, making it a high-quality plug-and-play module.\n\nCompared to LiFT (which only supports 2× upsampling) or FeatUp (which has limited performance), JAFAR naturally supports arbitrary resolution alignment and is verified to maintain alignment performance even at unseen high resolutions (e.g., 448) during testing.\n\nThe paper conducts systematic evaluations across multiple tasks and benchmarks, and the results show that the proposed method achieves superior performance.\n\nWeaknesses\nAlthough the paper frequently emphasizes “lightweight” and “efficient,” it does not provide numbers such as FLOPs, parameters, or inference latency. Also, there is no comparison with other task-agnostic methods (like FeatUp or LiFT) in terms of GPU/CPU runtime or memory usage. Moreover, the authors do not evaluate the challenges of high-scale upsampling, where the attention computation grows quadratically with resolution. Compared to FeatUp’s layer-wise interpolation, is JAFAR more prone to out-of-memory issues under extreme upsampling ratios?\n\nThe paper does not analyze robustness to input image quality. Since JAFAR heavily depends on the original image as Query guidance, its performance may degrade when the image is blurry, occluded, or under low light. There is no robustness evaluation on low-light, blurred, JPEG-compressed, or occluded images. In other words, if degradations like occlusion or Gaussian blur are applied to the Query image, will the output deteriorate? Is the module too dependent on image quality? This is a critical issue for real-world deployment of downstream tasks.\n\nThe paper should include visualizations such as attention maps or Query-Key matching diagrams to show what the model captures from the image. Adding these would make the method more intuitive.\n\nAlthough the paper claims support for 32→448 upsampling, have the authors tried more extreme cases such as 8→512 or 64→1024? It would be helpful to discuss the generalization limits of the module (e.g., patch size, image aspect ratio) or failure cases.\n\nI am curious whether large upsampling ratios will cause Query-Key misalignment or sparse attention issues. Also, is RoPE more stable under large-scale resolutions compared to learnable positional bias or sinusoidal encoding?", "was_revised": true, "final_justification": "The author's response has addressed my concerns, and I raise my score to 5.", "rebuttal": "We thank the reviewer for the helpful comments and appreciate the positive feedback. Please note that, for the purpose of the rebuttal, all probing layers of the following tables were trained for 10 epochs (instead of 20, as reported in the main paper) to accelerate training while preserving fair comparison.\n\n**1. Although the paper frequently emphasizes “lightweight” and “efficient,” it does not provide numbers such as FLOPs, parameters, or inference latency. Also, there is no comparison with other task-agnostic methods (like FeatUp or LiFT) in terms of GPU/CPU runtime or memory usage.**\n\nWe would like to highlight that the supplementary material includes a detailed comparison of JAFAR against other task-agnostic methods (Tab 10-11), such as FeatUp and LiFT, reporting parameter counts, inference latency, and GPU memory usage. As detailed in the supplementary material, JAFAR demonstrates runtime and memory usage comparable to FeatUp’s layer-wise interpolation strategy for upsampling resolutions up to 224×224. \n\n**2. Moreover, the authors do not evaluate the challenges of high-scale upsampling, where the attention computation grows quadratically with resolution. Compared to FeatUp’s layer-wise interpolation, is JAFAR more prone to out-of-memory issues under extreme upsampling ratios?**\n\nAt higher resolutions, JAFAR incurs additional computational overhead; however, it remains scalable and stable, successfully handling feature upsampling resolutions up to 896×896 without encountering out-of-memory issues.\nThat said, extreme upsampling ratios naturally lead to increased GPU memory consumption. This limitation could be alleviated by replacing global attention with more localized alternatives, such as those used in [1, 2, 3, 4]. We will make sure to explicitly include these considerations in the limitation section of the supplementary material.\n\n**3. The paper does not analyze robustness to input image quality. Since JAFAR heavily depends on the original image as Query guidance, its performance may degrade when the image is blurry, occluded, or under low light. There is no robustness evaluation on low-light, blurred, JPEG-compressed, or occluded images. In other words, if degradations like occlusion or Gaussian blur are applied to the Query image, will the output deteriorate? Is the module too dependent on image quality? This is a critical issue for real-world deployment of downstream tasks.**\n\nWe thank the reviewer for highlighting this important point regarding robustness to image quality. While we did not explicitly emphasize this in the main paper, a point we will address in the revised version, we would like to clarify that, during training, we incorporated light Gaussian blur and noise augmentations to improve robustness to image degradations. \nTo better assess the impact of this regularization, we include below an ablation study evaluating JAFAR’s performance under noise degradation. These results demonstrate that the model maintains reasonable performance and that the augmentation strategy contributes positively to its robustness. We appreciate the reviewer’s feedback and will make sure to integrate these insights more explicitly in the final version of the paper. \n\n|               | Cityscapes | Cityscapes + Noise |\n|---------------|------------|--------------------|\n| JAFAR w/ aug  | 60.86      | 59.00              |\n| JAFAR wo/ aug | 60.44      | 58.44              |\n\n|               | VOC   | VOC + Noise |\n|---------------|-------|-------------|\n| JAFAR w/ aug  | 83.55 | 82.98       |\n| JAFAR wo/ aug | 83.07 | 82.73       |\n\n**4. The paper should include visualizations such as attention maps or Query-Key matching diagrams to show what the model captures from the image. Adding these would make the method more intuitive.**\n\nWe thank the reviewer for this valuable recommendation. We agree that visualizations of the Query-Key attention mechanism would provide a more intuitive understanding of the method, helping to visualize how the queries attend to keys for the upsampling. Following the recommendation, will include additional visualizations of the cross-attention mechanism in a dedicated supplementary section. \n\n**5. Although the paper claims support for 32→448 upsampling, have the authors tried more extreme cases such as 8→512 or 64→1024? It would be helpful to discuss the generalization limits of the module (e.g., patch size, image aspect ratio) or failure cases.**\n\nWe conduct below an experiment evaluating the performance of JAFAR in a linear segmentation probing setup under various upsampling ratios including extreme scenarios. This experiment demonstrates that JAFAR maintains robust performance even for extreme upsampling cases. We thank the reviewer for this recommendation and will further discuss such cases in the supplementary material.\n\n**Input Feature Size: 8**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 7$) | 112 ($\\times 14$) | 224 ($\\times 28$) | 448 ($\\times 56$) | 896 ($\\times 112$) |\n|:---------------------:|:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|\n|        Bilinear       |      31.87      |       31.81       |       31.90       |       31.61       |        31.92       |\n|         JAFAR         |      36.54      |       36.04       |       35.71       |       35.77       |        35.54       |\n\n| **VOC** (mIoU) | 56 ($\\times 7$) | 112 ($\\times 14$) | 224 ($\\times 28$) | 448 ($\\times 56$) | 896 ($\\times 112$) |\n|:--------------:|:---------------:|:-----------------:|:-----------------:|:-----------------:|:------------------:|\n|    Bilinear    |      64.79      |       64.95       |       64.89       |       64.91       |        64.9        |\n|      JAFAR     |       72.7      |       73.13       |       72.77       |       72.69       |        72.55       |\n\n\n**Input Feature Size: 16**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 3.5$) | 112 ($\\times 7$) | 224 ($\\times 14$) | 448 ($\\times 28$) | 896 ($\\times 56$) |\n|:---------------------:|:-----------------:|:----------------:|:-----------------:|:-----------------:|:-----------------:|\n|        Bilinear       |       47.77       |       47.77      |       47.78       |       47.83       |       47.84       |\n|         JAFAR         |       53.18       |       53.22      |       52.35       |       51.76       |       51.36       |\n\n| **VOC** (mIoU) | 56 ($\\times 3.5$) | 112 ($\\times 7$) | 224 ($\\times 14$) | 448 ($\\times 28$) | 896 ($\\times 56$) |\n|:--------------:|:-----------------:|:----------------:|:-----------------:|:-----------------:|:-----------------:|\n|    Bilinear    |       75.41       |       75.42      |       75.51       |       75.50       |       75.50       |\n|      JAFAR     |       80.57       |       80.76      |       80.58       |       80.36       |       80.24       |\n\n\n**Input Feature Size: 32**\n\n| **Cityscapes** (mIoU) | 56 ($\\times 1.75$) | 112 ($\\times 3.5$) | 224 ($\\times 7$) | 448 ($\\times 14$) | 896 ($\\times 28$) |\n|:---------------------:|:------------------:|:------------------:|:----------------:|:-----------------:|:-----------------:|\n|        Bilinear       |        59.56       |        59.53       |       59.41      |       59.54       |       59.15       |\n|         JAFAR         |        59.65       |        61.39       |       61.38      |       60.86       |       60.79       |\n\n| **VOC** (mIoU) | 56 ($\\times 1.75$) | 112 ($\\times 3.5$) | 224 ($\\times 7$) | 448 ($\\times 14$) | 896 ($\\times 28$) |\n|:--------------:|:------------------:|:------------------:|:----------------:|:-----------------:|:-----------------:|\n|    Bilinear    |        80.64       |        80.72       |       80.77      |       80.83       |       80.82       |\n|      JAFAR     |        81.51       |        82.53       |       82.67      |       83.55       |       83.24       |\n\n\n**6. I am curious whether large upsampling ratios will cause Query-Key misalignment or sparse attention issues.**\n\nAcross our experiments, we did not observe any noticeable query–key misalignment or sparse attention issues, even at high upsampling ratios. As noted above, JAFAR demonstrates strong robustness under extreme upsampling conditions. To further support this, we will include additional visualizations in the supplementary material that show upsampled feature maps at various resolutions, highlighting the consistency and stability of the attention mechanism across scales.\n\n**7. Also, is RoPE more stable under large-scale resolutions compared to learnable positional bias or sinusoidal encoding?**\n\nIn this work, we did not investigate alternative positional embeddings. Following the rationale presented in [5], we adopted RoPE due to its ease of integration, absence of positional artefacts, and strong transferability across spatial resolutions.\n\n\n[1] Liu, Ze, et al. \"Swin transformer: Hierarchical vision transformer using shifted windows.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[2] Hassani, Ali, et al. \"Neighborhood attention transformer.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[3] Pan, Xuran, et al. \"Slide-transformer: Hierarchical vision transformer with local self-attention.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[4] Fan, Qihang, et al. \"Rmt: Retentive networks meet vision transformers.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024.\n\n[5] Darcet, Timothée, et al. \"Cluster and predict latent patches for improved masked image modeling.\" arXiv preprint arXiv:2502.08769 (2025)."}]

Author rebuttal:

Meta-review: This paper received overall positive reviews. Reviewers recognized it as an interesting approach to enable adaptive super-resolution for arbitrary resolutions, validated across multiple tasks. The rebuttal effectively addressed the initial concerns, resulting in a version suitable for acceptance. The AC concurs with the majority and considers this a solid contribution. It is recommended that the clarifications and supporting evidence from the rebuttal be incorporated into the camera-ready version. Looking forward to seeing this work presented at NeurIPS.
Decision: Accept (poster)