A Systematic Evaluation of Out-of-Distribution Generalization in Crop Yield Prediction
Authors: Aditya Chakravarty
Venue: Accepted by TMLR ()

Abstract: Accurate crop yield forecasting under shifting climatic conditions is essential for food security and agricultural resilience. While recent deep learning models achieve strong performance in in-domain settings, their ability to generalize across space and time—critical for real-world deployment—remains poorly understood. In this work, we present the first systematic evaluation of temporally-aware crop yield prediction models under spatio-temporal out-of-distribution (OOD) conditions, using corn and soybean data across more than 1,200 U.S. counties. We benchmark two representative architectures, GNN-RNN and MMST-ViT, using rigorous evaluation strategies including year-ahead forecasting, leave-one-region-out validation, and stratified OOD scenarios of varying difficulty based on USDA Farm Resource Regions. Our comprehensive analysis reveals significant performance gaps across agro-ecological zones, with some models showing negative R² values under distribution shift. We uncover asymmetric transferability patterns and identify the Prairie Gateway region as consistently challenging for generalization. These findings challenge prior generalizability claims and provide practical insights for deploying agricultural AI systems under climate variability.

Published in Transactions Machine Learning Research (03/2026)
A Systematic Evaluation of alization in Crop Yield Prediction
Gener-
Aditya Chakravarty Independent Research
chakravarty..dttyya2888gg aditya28gmaiii.com
Reviewed on OpenReview: http:://pearrrreeeeee//////ss/ea
Abstract
Accurate crop yield forecasting under shifting climatic conditions is essential for food security and agricultural resilience. While recent deep learning models achieve strong performance in in-domain settings, their ability to generalize across space and tine-critical for real-world deployment-remne remains poorly understood. In this work, we present the first systematic evaluat ion of temporally-aware crop yield prediction models under spatio-temporal out-of-distributinn (OOD) conditions, using corn and soybean data across more than 1,200 U.S. countics. We benchmark two representa- tive architectures, GNN-RNN and MMST- ViT, using rigorous evaluation strategies validation, and stratified
including year-ahead forecasting, leave-one-region-- OOD scenarios of varying difficulty based on USDA Farm Resource Regions. Our comprehensive analysis reveals significant performance gaps across agro-ecological zones, with some models showing negative R values under distribution shift. We uncover asymmetric transfcrability patterns and identify the Prairic Gateway ro- gion consistently challenging for generalization. These findings challenge prior generalizability claims and provide practical insights for deploying agricultural Al systems under climate variability.
1 Introduction and Related Work
Crop yield prediction is critical lask in agricultural planning, food security, and economic forecasting, particularly in the face of climate change (Houghton et al. 1990). Traditional process- based models,
while useful, often suffer from high inaccuracios due to strong assumptions about management practices and high computational costs (Leng and Hall 2020 Studies have demonstrated the
growing impact of climate change agricultural productivity (Orti--oobea et al., 2018). (Zhao et al. 2017). These challenges have driven the adoption of machine learning (ML) and deep learning (DL) approaches, which leverage largo-scale environmental and satellite data for more accurate crop yield predictions. Recent IPCC reports underscore the growing and detrimental effects climate change on global agricultural yields, further emphasizing the need for adaptive strategies and accurate predictive models (PPCC 2021 b. Recent deep lcarning models improve crop yield

---

Published in Transactions Machine Learning Research (03/2026)
prediction by capturing spatio-temporal dependencies, but true spatial distrii-iittiiutionn (OOD) generalization remains unexplored. Bridging this gap is critical for forccasting yields in regions lacking historical data or facing climate-driven shifts. This study addresses this critical gap through systematic evaluation of model robustness under realistic spatio-temporal distribution shifts.
DL models in yield prediction Deep learning (DI.) models for crop yield prediction leverage two complementary types of data sources: remote sensing imputs (satellite imagery, UAV data, vegetation indicos) and mnttoorooogical/errinnnn
inputs (weather variables, soil paramoters). While earlier works often relied on one or the other, slate-of-the-ard models increasingly fuse both modalities, as they capture different aspects of crop growth dynamics. This distinction useful for understanding model design, though in practice the two input types are complementary rather than mutually exclusive.
Early neural network models for crop yield prediction demonstrated promising results, outperforming conventional regression techniques (Drummond et al., 2003), (Liu et al., 2001). Among the meteoro- logical and environmental data- based approaches, a key advancement was the CNN-RNN framework, which integrates multi-year metcorological and environmental data improve yield forecasts (Khaki et al. 2019), (Khaki and Wang) 2021). This method established the importance of historical weather data, demonstrating that using multi-year sequences of climate variables significantly enhances prediction accuracy.
Building upon CNN-RNN architectures, newer methods incorporate graph neural networks (GNNs) to model geographical dependencies. The GNN-RNN model extends CNN-RNN by incorporat spatial relationships among counties, ena enabling the model to leverage information from neighboring regions to refine yield predictions (Fan et al.) (022) using long-term metcorological data. This method has shown improvements over CNN-RNN models in various evaluations, demonstrating the benefits of integrating spatial context into deep learning frameworks.
On the other hand, remote sensing- based approaches utilize satellite and UAV-derived data to estimate crop yields (Tseng et al. 2021). These methods analyze features extracted from spectral indices, vegetation health, and high-rosolution imagery to make yield predictions. While effective, many of these models overlook the direct impact of meteorological variations on crop growth (Sainte Fare Garnot and Landrieu 2021).
More recently, hybrid approaches have emerged that integrate both remote sensing data and meteorological variables for enhanced crop yield predictions. As climate change increasingly affects global agriculture, models that explicitly incorporate long-term climate trends have gained attention Ortiz-Bobea et al.) 2018), (Zhao et al.) 2017). Unlike earlier DL models that focused primarily on short-term weather patterns, transformer- based architcctures explicitly model long-tom climate shifts, enabling stakeholders to assess their elfects on crop yields (Lin el al. 2023 By leveraging attention mechanisms, these models both short-term weather fluctuations and long-ten climate trends, offering an alternative approach GNN-based methods.
2 Motivation and Contribution
Despite growing interest in deep learning for crop yield prediction, systematic evaluation of model robustness under realistic distribution shifts remains limited. Existing studies typically evaluate models within the same goographic regions used for training, providing incomplete picture of
2

---

Published in Transactions Machine Learning Research (03/2026)
real-world deployment scenarios where models must generalize to new climatic conditions, unseen regions, or future time periods. This evaluation gap is particularly concerning for agricultural applications, where climate change is continuously shifting the underlying data distribution and reliable forecasts are needed for regions with limited historical data.
We conduct systematic comparative analysis across both temporal-only and spatio-temporal OOD settings, using data from over 1,200 counties in the top 20 U.S. corn and soy producing states. Counties are grouped into scientifically grounded clusters based on geographic and climatic characteristics, enabling rigorous testing of generalization. Furthermore, we design deployment scenarios categorized by cluster similarity to assess model robustness across OOD shifts of varying severity, with direct implications for climate-resilient agriculture and food security. By using publicly available datasets and scientifically robust clustering techniques, this work not only enhances the understanding of model generalizability in practical agricultural forecasting scenarios but also informs strategic model deployment decisions under climate variability. Our findings provide insights into the relative strengths and limitations of MMST-ViT and GNN-RNN and offer guidance for selecting optimal predictive approaches under different apatial-tempora distribution shifts.
3 Dataset and features
We used the CropNet dataset (see Lin et al., 2024 1, which is a large-scale, publicly available,
multi-modal dataset specifically designed for climate chango-aware crop yield predictions across the contiguous United States from 2017 to 2022. Table S summarizes the CropNet dataset components.
4 Experiment Design
This study investigates two forms of out-of-distribution (OOD) generalization: temporal OOD and spatio-temporal OOD, Temporal OOD refers to year-ahead prediction, where models are trained on data from 2017 2021 and tested on 2022 in the same region. Spatio-temporal OOD corresponds to more challenging: the test set is not only for 2022 but also from geographic region entirely excluded during training. This dual-axis domain shift across both time and model generalizability.
forms a stringent test of
Each experiment in Table is designed to isolate a specific aspect of OOD robustness: UMAP validates the clustering basis; reproduction establishes basclines; LCCOCV measures spatial general- ization; ablations quantify each modality's contribution; the pairwise matrix reveals asymmetric transfer; and deployment scenarios simulate real-world conditions of varying difficulty.
Data Preparation
USDA Farm Resource Regions for Robust Regional Classification and OOD Evaluation The counties in our study were classified into seven clusters based on the USDA Farm Resource Regions (Heimlich) 2000), a widely recognized and scientifically validated framework for agricul- tural classification. These regions are delincated using climate, soil properties, and topographic characteristics, ensuring that counties within the same region share similar agro-environmenttll conditions. Unlike arbitrary spatial partitioning, this classification method captures the biophysical
3

---

Published in Transactions Machine Learning Research (03/2026)
Table 1: Experimental design and guiding questions for evaluating OOD generalization. Each experiment is necessary to isolate a specific aspect of OOD robustness; together they form a comprehensive benchmarking protocol.
Experiment
Description
Question
UMAP embedding
Cluster counties using UMAP on weather variables
Spatial structure; Are USDA FRRR valid basis for OOD analysis?
Reproduce prior
GNN-RNN and MMST-VIT runs us- ing original configs and datasets
Do original performance claims hold under re-evaluation?
models
Leawe-one-lluster-
Tune models via LCO-CV and sess generalizability across clusters
Which regions fit/unfit for spatio- temporal generalization
out CV
Ablation: No mes- sage passing
Disable message passing in the GNN-RNN model
How important is spatial message passing in GNN-RNN?
Ablation:
No
Remove Sentinel input from MMST- ViT, only use HRRR data
What role does remote sensing play in model accuracy?
Sentinel-2 input
Region pairwise ma- trix
Evaluate models on all train-test gion pairs
How does performance vary across region combinations?
Real-world deploy- ment scenarios
Define OOD cases using LCO-CV results, geugraphy and UMAP
How well do models generalize in
realistic deployment seltings?
and agronomic factors that influence crop production, making a robust basis for regional analysis Spangler al. 2020). Figure illustrates the spatial distribution of study areas.
By leveraging the USDA Farm Resource Regions, ensure scientifically grounded approach to OOD evaluation, essential for building robust agricultural AI models that remain reliable under future climate uncertainties. For all experiments, data from 2017 to 2021 is used for training, while models are evaluated using 2022 data. All experiments were conducted on a system with NVIDIA RTX 4090 GPU (24 GB VRAM) and 64 GB RAM.
Architecture Tuning and Chea Evaluation Model sensitivity analysis was conducted during the leave-one-chuster-out cross-validation (LCO-CV) phase to select optimal hyperparamcters and identify regions with strong generalization potential. For GNN-RNN, we varied the number of layers (2, 3, 4), aggregation type (mean, pool, gen), and dropout rate (0, 0.5, 0.7). For MMST-ViT, we explored embedding dimensions (128, 512) and dropout rates (0, 0.3, 0.5). The best model configurations from LCO-CV were carried forward into year- evaluations.
In this setting, models were trained data from 2017-2021 and tested 2022, with each train-test region pair evaluated independently. For MMST-ViT, this included re-running both contrastive pretraining and fine-tuning. Diagonal entries in the resulting performance matrices represent within- region year-ahead forecasting: diagoaa entries quantify spatio-temporal transfer. To simulate realistic deployment, train regions were grouped into larger sets than the test regions (3-to-1 splits), with lest candidates selected based on generalization insights from LCO-CV. The evaluation spans seven USDA Farm Resource Regions (EU, HL, MSP, NGP, NC, PG, SS), with RMSE and standard deviation reported over three random seeds for both corn and soybean.
4

---

Published in Transactions Machine Learning Research (03/2026)
Color
Prairie-Gateway
Northern-Crescent
Heartland
Mississippi-Pone
Southern- Seaboard
Souther--eeboard
Eastern-U Uplends
Northern-Great-PPann Plaans
Figure Spatial Distribution of Study Areas and USDA Farm Resource Regions

---

Published in Transactions Machine Learning Research (03/2026)
5 Results
5.1
Reproduction of Published GNN-RNN and MMST-ViT Results
To validate our implementation pipeline, we first reproduced the GNN-RNN model Fan et al. 2022) using the same architecture and hyperparameters as reported in the original paper. As only data from Illinois (IL) and Iowa (IA) were publicly available, our experiments were limited to these two states. We trained the model on data from 1981-20 1981-2017 and evaluated on the 2018 season.As shown in Table [ our reproduced RMSE/Std-dev values differ by only 0.04 for soybean and 0.05 for corn from those reported. The R and correlation values are also within 0.01 0.09 of the published results.
Table 2: Comparison of GNN-RNN and MMST-VIT performance between our reproduced results and those reported in the original publications.
Model (Dataset)
Crop
Present study
Reported in original paper
RMSE/Std-dev
R²
Corr
RMSE/Std-der
It²
Corr
GNN-RNN
Soy bean
0.43
0.72
0.83
0.47
0.73
0.86
(only IL IA)
Corn
0.41
0.58
0.73
0.46
0.67
0.82
RMSE
R²
Corr
RMSE
R²
Corr
MMST-VIT (IL + MS + LA)
Soybean
6.98
0.81
0.87
5.00
0.89
0.89
Corn
17.61
0.78
0.84
10.00
0.89
0.89
We also reproduced the MMST-ViT model (Lin et al. 2023)
using the same dataset (IL, IA, MS, and LA), input modalities inen2 and HRRR), and training workflow, including both the contrastive pretraining and fine-tuning stages. All implementation details and data preprocessing steps were kept identical to those outlined in the original paper. Nevertheless, our reproduced RMSE values were higher by 1.98 for soybean and 7.61 for corn (in bu/acre), with R values lower by up to 0.11. Correlation differences remained within 0.05. While some variation is expected due to factors such as random initialization nondeterministic GPU operations, the magnitude of the discrepancy raiscs the possibility that not all aspects of the original experimental setup were fully documented or reproducible from the available information. All experiments in the present study were repeated three times with different random seeds, and we report the mean performance across runs. In contrast, our GNN-RNN model trained without soil and management variables achieved performance nearly identical to the original paper that included these features, indicating that such inputs may contribute little additional value in this specific setting.
5.2 Cross validation, Ablation outcomes, and real-world train-test scenarios
Overall, performance in both settings was substantially weaker than in-distribution evaluations. Year-ahead prodictions (temporal OOD) showed moderate degradation in R² and correlation, but performance remained usable in several regions. In contrast, the spatio-temporal LORO selting resulted in significantly worse outcomes. Across many configurations, models exhibited negative R² and correlation values, indicating that prodictions were not only inaccurate but in some cases inversely related to ground truth.
Eastern Uplands (EU) was the most consistently predictable region across both crops and OOD settings. Heartland (HL) and Northern Great Plains (NGP) also stood out, with multiple GNN-RNN
6

---

Published in Transactions Machine Learning Research (03/2026)
and MMST-ViI configurations achieving positive R² values. For soybean, HL achieved R² > 0 in settings such sys_type-gen, n_ 1a, and MMST-ViT with e=512, drop=0. NGP exceeded R 0.25 in several GNN configurations. For corn, NGP again achieved the highest out-of-region R (> 0.45) under MMST-ViT with e=128, drop=0.5. 5. Model design choices had clear effects on generalization. For GNN-RNN, higher dropout rates were consistently associated with weaker OOD performance. In particular, for soybean in NGP, R² dropped from approximately 0.25 (no dropout) to below -0.4 when dropout=0.1 7 was used. Increasing the depth to n_ layers=4 was sometimes beneficial, espocially in HI. and NGP, but did not yield consistent improvements across all regions.
For MiST-ViT, configurations with smaller embedding size (e-128) and dropout (drop=0) generally led to the best results. Larger embedding sizes and stronger regularization often resulted in poorer generalization, with several cases showing negative R² and correlation valucs-particulally in Prairie Galeway and Southern Seaboard. Given these observations, we selected the following configurations for downstream use and OOD-focused evaluations: for GNN-RNN, n_layers=4, dropout=O, and agg_ tys_type=eenn or pool; for MMST-ViT, e=128, drop=0. Table R summarizes the training and testing times associated with two model architectures. There is tremendous gap between the two owing to the model design.
Table 3: Training time comparison of MMST-VIT and GNN-RNN on a single RTX 4090 GPU.
GNN-RNN achieves a 135x speedup over MIST-ViT.
Model
Pretraining Time
Fine-tuning Time
Total Training Time
MMST-VIT GNN-RNN
23 hours
8.5 hours
31.5 hours
14 minutes
Sentinel-2 Data Ablation in MMST-VIT Replacing Sentinel-2 inputs with dummy values led to consistent performance drops across crops and regions. This indicated that satellite imagery contributes improves MMST-ViT performance even when weather data is available.
Message Passing Ablation in GNN-RNN Removing spatial message passing-effective reducing the model to CNN-RNN baseline without graph-based context led to noticeable decline in generalization performance. This underscores the importance of graph connectivity in leveraging local context for spatial genoralization.
Table summarizes the final training and lesting clusters associated with each hypothesized OOD difficulty level. The easy, medium, and hard splits are defined heuristically, based on a qualitative assessment of leave-cluster-out (LCO) vross-validation results, UMAP emboddings, and geographical proximily.
5.3 Cross-region transferability and pairwise RMSE patterns
Tables and 6 report the full 7 x 7 pairwise RMSE matrices for GNN-RNN and MMST-ViL respectively, across both soybean and corn. These matrices reveal asymmetric transfcrability patterns that are directly relevant. to OOD understanding: the off-diagonal entries quantify how much performance degrades when training and test regions differ. For soybean with GNN-RNN, diagonal values (2022 ycar-ahead RMSE) are lowest for HL (6.15), MSP (6.09), and NGP (7.11),
7

---

Published in Transactions Machine Learning Research (03/2026)
Table 4: Real-world scenarios and corresponding USDA Farm Resource Region splits.
Scenario
Train Region
Test Region
Case (Easy)
Trairie-Gatewa Heartland
Bastern-Uplands
Mississippi-Pont Portal
Case (Medium)
Northern Cresseent + Prairie-
Heartland
Gateway + Northem-Great-PPlee
Case (Hard)
Trairie-Gatever + Southern-
Northern-Great-
Seaboard + Mississippi-Porall
Plains
indicating relatively stable temporal generalization. However, the off-diagonal entries reveal strong asymmetries. Models trained HL generalize reasonably well to MSP (8.88) and NGP (7.73), but not to PG (19.30) or SS (11.50). Similarly, NGP-HL (8.04) and NGP-MMSP (9.70) are relatively low, suggesting latent affinity among these midwestern regions. In contrast, training on PG consistently rosults in high RMSE values across all test regions (c.g.. PG-EU: 14.71, PG-MSP: 15.54, PG-SS: 13.03), indicating poor generalization. Notably, SS shows low within-region error (7.96), but generalizes poorly to others. In corn with GNN-RNN, HL again shows strong within-region performance (22.67), with moderately good transfer to MSP (32.44) and NGP (32.20). Training on NC leads to particularly low RMSE in EU (22.29), suggesting that NC-trained models may capture transferable features for FU. In contrast. PG again acts as an unreliable source region, with high RMSE when predicting EU (48.36), HL (33.31), and SS (45.18). PG also exhibits the worst transferability in the soybean sctup, indicating structural dissimilarity.
Table 5: RMSE for Soybean (left) and Corn (right) using GNN-RNN model; diagonal entries (bold) represent year-ahead predictions for 2022. Color scale is shown below the table.
EU
HL
MSP
NGP Soybean
NC
PG
EU
HI.
MSP
NGP Corn
NC
PG
88
Train Test
Train
Test
EU
5.46
7.24
11.55
9.01
9.63
21.42
9.70
EU
26.69
38.92
43.55
32.86
22.29
97.42
41.77
HL
8.39
6.15
8.88
7.73
8.37
19.30
11.50
H1,
33.05
22.67
32.44
32.20
33.66
56.21
49.68
MSP
17.22
11.17
6.09
9.79
9.72
26.
10.59
MSP
10.98
23.48
31.03
30.87
37.03
53.01
39.05
NGP
8.94
8.04
9.70
7.11
11.88
23.37
12.48
NGP
33.81
25.83
34.33
21.65
57.52
51.49
44.54
NC
12:25
12.39
12:67
10.29
7.25
28. 28.23
12.51
NC
40.13
41.67
30.56
34.64
24.42
92.17
51.94
PG
11.71
11.43
15.54
14.62
13.59
11.71
13.03
PG
48.36
33.31
42.21
11.59
52 /17
12.94
15.18
SS
13.55
10.69
12.17
12.86
9.77
11.20
7.96
SS
42.13
29.91
39.46
11.47
32.35
53.41
25.09
Color scale
Soybean RMSE (bu/acre):
<<
8-11
11-14
214
Corn RMSE (bu/acre):
28
28 40
40
55
>55
Turning to MMST-VIT, the year-ahead results are less promising overall. For soybean, HI, MSP, and SS show relatively low diagonal RMSE (10.18, 8.71, and 7.15, respectively), but cross-region RMSEs are considerably higher than those observed with GNN-RNN. For example, HL-NGP is 25.01, MSP-PG is 23.15, and NC-SS is 14.26. Soybean predictions from PG are unreliable across the board, consistent with GNN-RNN findings. Corn performance with MMST-ViT is similarly degraded. Only a few region pairs achieve RMSE below 30, e.g., EU-NC (29.14) and SS-EU (30.22). Diagonal entries for HL (33.37), NC (37.93), and SS (35.48) are modestly acceptable, but generalization across regions is again poor. PG remains the least reliable source region with
8

---

Published in Transactions Machine Learning Research (03/2026)
Table 6: RMSE for Soybean (left) and Corn (right) using MMST-ViT model; diagonal entries are bold represent year-ahead predictions for 2022 and colors indicate performance. Color scale same as Table
EU
HL
MSP
NC
NGP
PG
SS
EU
HL
MSP
NC
NGP
PG
SS
Train Test
Soybean
Train Test
Corn
EU
8.93
11.16
9.69
9.38
14.51
22.15
15.25
EU
26.06
38.83
35.52
29.14
34.29
59.27
45.90
HL
25.01
20.38
HL
52.35
89.87
MSP
9.62
11.91
8.71
9.72
16.87
23.15
10.78
MSP
35.21
39.39
41.32
35.96
49.60
69.85
57.16
NC
11:31
10.45
10.13
14.19
19.38
24.76
14.26
NC
36.32
37.78
15.17 17
37.93
10.91
66.12
15.27
NGP
24.12
25.63
25.67
16.59
11.33
17.76
12.17
NGP
44.35
73.04
601.21
51.72
43.61
39.47
36.58
PG
16.18
13.87
12.03
13.37
14.48
PG
38.09
61.08
60 93
14.85
16
S8
13.55
17.65
10.35
9.31
8.19
18.75
7.15
SS
30.22
51.23
33.45 18
32.
30.57
51.83
35.48
extremely high test RMSEs across all targets (e.g., PG-HL: 64.08, PG-SS: 46.92). Similarly, NGP shows large within-region error (43.61) and high off-diagonal errors, e.g., NGP-HLL 73.04, confirming weak model fit.
Across both crops, the GNN-RNN model shows significantly better cross-region generalization than MMST-ViT. For soybean, average RMSEs for GNN-RNN across test regions are consistently 3 points lower than those for MMST-ViL The corn results reveal even larger disparities, with MMST-ViT frequently exceeding 50 RMSE cross-region tests (e.g., HL- 89.87, NGP-HHL: 73.04), while GNN-RNN maintains more bounded errors.
Several consistent trends emerge. First, HL and NGP are relatively strong source and target regions across both crops, and especially under the GNN-RNN model. Second, PG exhibits poor generalization in both directions, suggesting significant structural dissimilarity. Third, MMST-ViT shows more variance in generalization and higher errors across nearly all region pairs, possibly due to overfitting or insufficient spatial inductive bias. Finally, within-region year-ahead predictions generally more accurate than cross-region evaluations, though the gap varies by model and crop. Taken together, these results support the use of GNN-RNN for spatio-temporal generalization, particularly with training from HL or NGP, and caution against deploying MMST-ViT in OOD settings without further adaptation or regularization.
5.4 Performance in Realistic Deployment Conditions
Performance Gap Rationalc and Calculation. To assess the practical utility of the models in deployment scenarios, we evaluate how well they generalize across unseen geographies- an essential capability in real-world agricultural settings where labeled data may not be available in all regions. Specifically, we compute the performance gap between two settings: (i) Different region scenario, where the test region held out during training (e.g., trained in regions A+B for 2017 to 2021, and tested in region C for 2022), and (ii) the same-region year-ahead, where both training and test data belong to the same region but separated temporally (e.g., trained region A for 2017-2021 and tested in region A for The performance gap is defined as the relative increase in RMSE (see appendix).
Table summarizes results across three OOD cascs as defined in Table ! GNN-RNN consistently achieves lower absolute RMSE across both corn and soybean predictions, even under OOD set- tings. This makes it stronger candidate for deployment where minimizing prediction error critical. However, the performance gap is more variable for GNN-RNN, particularly in harder 0OD
9

---

Published in Transactions Machine Learning Research (03/2026)
Table 7: OOD vs. same-region RMSE (bu/acre) across crops, models, and scenarios. The scenario cases are detailed in Table
RMSE (same-region year-ahead)
RMSE( region year-ahead)
Performance Gap (%)
Crop
Model
Scenario
Case 1
9.04
8.93
1.23
Soybean
MMST-VE VIT
Case 2
11.63
10.18
14.24
Case 3
12.19
11.33
7.39
Case 1
30.92
26.06
18.65
Corn
MMST-ViT
Case 2
34.93
33.37
4.67
Case 3
50.26
43.61
15.25
Case
6.92
5.46
26.75
Soybean
GNN-RNN
Case 2
9.75
6.15
58.53
Case 3
11.11
7.11
56.20
Case
27.62
26.69
3.48
Corn
GNN-RNN
Case 2
27.75
22.67
22.40
Case 3
32.07
21.65
48.13
cases indicating higher sensitivity to distribution shift. MMST-ViT, while slightly less accurate overall, exhibits more stable performance gaps across regions and crops.
6
Discussion
Limitations Despite their complementary strengths, both models and the underlying datasel exhibit notable limitations. CropNet repackages publicly available sources (Sentinel-2, HRRR, USDA) without introducing domain-cxpert curation. While the dataset does provide NDVI imagery, neither GNN-RNN nor MMST-ViT incorporate this modality in their current implementations. Other widely-used indices such EVT (Huete et al. 2002) are also absent, and the dataset remains limited to county-level aggregation, precluding field-scale yield estimation.
Impact of Sentinel-2 bands While CropNet is the first terabyte-scale, multi-modal benchmark for county-level yield forecasting, its design choices impose important limitations. Only four of Sentinel- 2's twelve spectral bands are provided-omitting red-edge bands crucial for detecting vegetation stress (Krisp and Scheinert 2021) and all imagery is Level-1C (un-corrected), leaving atmospheric effects unaddressed (Topping et al. 2019). Furthermore, spatial aggregation to km x km grid cells erases intra-county heterogeneity, precluding field-level analysis.
Sentinel-2 imagery resolution and uneven grid size The number of grids per county is also highly variable, ranging from fewer than 5 to over 130, which introduces unequal spatial sampling across countics and likely contributes to model underperformance by biasing learning toward large, extensively sampled areas. The heterogeneity grid coverage likely exacerbates cross-region transfer challenges well, since counties with few grids contribute noisier less informative features to the learned representations, reducing gencralization robustncss under distribution shifts.
10

---

Published in Transactions Machine Learning Research (03/2026)
Higherrresolution approaches like using MODIS (1 km resolution) could mitigate this by enabling more uniform representation of small or fragmented counties. Recent studics have shown that temperature anomalies, more than precipitation variability, are primary driver of yield variation globally (Tizumi and Sakai, 2020), making the loss of local variability particularly consequential. Additionally, other global datasets such as EarthStat have demonstrated the utility of higher spatial resolution and field-level yield mapping (Monfreda et al. 2008). which CropNet does not currently support.
Modelling scope MMST-ViT does not integrate soil properties or management variables, which are known to substantially influence yicld outcomes. In contrast, GNN-RNN has optional support for these factors, though they were not used in the core experiments presented due to data unavailability. The five-year (2017-2021) training window may be insufficient capture long-term climatic transitions, and the lack of integration with process-based or operational baselines (Lobell et al., 2015) hinders comprehensive assessment. Furthermore, the evaluation omits process-based crop models such as DSSAT (Jones et al. 2001) or APSIM (Robertson et. al. 2012 which are frequently used in agricultural forecasting pipelines.
Our comprehensive out-of-distribution evaluation reveals that the GNN-RNN model-combining a lightweight GraphSAGE encoder with an LSTM-exhibits the most robust spatial transferability across over 1,200 U.S. counties. Even when deployed in USDA Farm Resource Regions unseen during training, it often maintains reasonable RMSEs and positive correlations, indicating that its learned county embeddings capture both local geographic structure and temporal climate trends. Prior work using deep Gaussian processes for yield prediction also emphasizes the importance of robust uncertainty modeling under spatial shift (You et al.) 2017b). By contrast, both GNN-RNN and MMST-ViT suffer severe degradation under ppatio-temporal domain shifts (leaeeonn-rrggion-out evaluations), with frequent instances of negative R² and inverse correlation. Notably, LMST-ViT's performance implodes on truly unseen geographies- despite strong in-domain accuracy on the four states originally reported (II., IA, MS, (AA)~suggstiigg it relies heavily on memorized region-specific statistics rather than learning transferrable patterus.
Generalization Patterns
Poor performance in PG The Prairie Gateway (PG) consistently emerges as the hardest out-of-distributioo target in our leave-one-chuseer-on out evaluation, driven by several factors: a) its hot, semi-arid climate with chronically high vapor-pressure deficit induces yield responses not represented the humid training regions; b) widespread supplemental irrigation, unobscrvod in HRRR inputs, leads models trained on rain-fed systems to misinterpret precipitation anomalies; e) large internal heterogeneity in crop types, elevation (500-1700 m), and soil properties makes it difficult for single fcature mapping to generalize across PG counties; d) USDA yicld series in PG are sparser and noisier, increasing label uncertainty; and e) frequent compound hot dry extremes and late-season weather hazards violate the conditional independence assumptions embedded in source-region models. The omission of Sentinel-2's red-edge bands, which are particularly sensitive to early vegetation stress and chlorophyll content (Krisp and Scheinert, 2021). may disproportionately impair model performance in semi-arid or drought-prone regions such as Prairie Gateway, where early stress detection is critical for yield outcomes.
The observed regional disparities in generalization performance raise important socictal considerations. If predictive models are systematically more reliable in
Broader Impact Statement
11

---

Published in Transactions Machine Learning Research (03/2026)
historically well-resourced, rain-fed regions (e.g., Heartland, Northern Crescent) and less reliable in more vulncrable, semi-arid regions (o.g., Prairic Gateway), there is risk of exacerbating existing inequalities in agricultural decision support. Regions already facing greater climate risk, variable yields, and water scarcity could be further marginalized by inaccurate forecasts. Therefore, ensuring equitable model performance across diverse agro-climatic zones is not only a technical challenge but also an essential component of promoting fair access to the benefits of AI-driven agricultural innovation.
Mitigation approaches Several avenues exist to mitigate the poor transferability into challenging regions such as PG: a) incorporating additional covariates such as irrigation intensity, satellite- derived evapotranspiration, or groundwater depth to explicitly represent water management; b) using domain-aware normalization schemes that adjust weather feature scaling regionally, reducing feature distribution shift; c) applying domain-adversarial training feature alignment techniques to harmonize latent representations across climate regimes; d) supplementing training with a small, stratified sample of PG counties to expose the model to the semi-arid, irrigated production regime during learning; and e) combining data-driven predictions with biophysical priors, such as water- limited potential yield envelopes, to maintain physical plausibility under extreme conditions. We note that GNN-RNN model can incorporate soil and management variables.
7 Conclusion
This study provides the first systematic evaluation of spatio-temporal out-of-distribution generaliza- Lion in crop yield prediction, benchmarking stale-ofthhe-rl deep learning models under realistic climate-driven distribution shifts. Through comprehensive experiments across 1,200+ U.S. counties using scientifically grounded regional splits, we reveal significant limitations in current approaches that were previously obscured by in-domain evaluation protocols.
Our key findings include: (1) substantial performance degradation under spatial transfer, with models frequently achieving negative R2 values in unscen regions; (2) asymmetric transfcrability patterns, where certain regions (Prairie Gateway) consistently resist. generalization while others (Eastern Uplands) transfer well; (3) architectural differences in robustness, with GNN-RNN demonstrating superior cross-region stability compared to MMST-VIT despite requiring 100x less computational resources; and (4) the critical importance of incorporating spatial relationships, as demonstrated through ablation studies.
These negative results provide valuable insights for the agricultural Al community, highlighting the gap between controlled evaluation settings and real-world deployment challenges. Our evaluation framework, based on USDA Farm Resource Regions, offers standardized protocol for future robustness assessments in agricultural ML. The consistent failure modes ponttity-patiiuuaall in semi-arid, irrigated regions point toward specific research directions for improving model generalization under climate variability.
While deep learning shows promise for improving crop yield forccasting, our findings underscore the necessity of rigorous oul-of-distribution evaluation and transparent reporting of generalization limitations. Future work should prioritize domain adaptation techniques and incorporate physical constraints to improve robustness across diverse agro-ccological conditions.
12

---

Published in Transactions Machine Learning Research (03/2026)
References
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597-1607, 2020. URL
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesníkov, Dirk Weissenborn, Xiaohua. Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. doi: 10.48550/arXi 2010.11929.
S. T. Drummond, K. A. Sudduth, A. Joshi, S. J. Birrell, and N. R. Kitchen. Statistical and neural methods for spec-fp yicld prodiction. Transactions of the ASAE, 46(1):5, 2003.
Joshua Fan, Junwen Bai, Zhiyun Li, Ariel Ortiz-Bobea, and Carla P. Gomes A gmn-rnn approach for harnessing geospatial and temporal information: Application to crop yield prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11873 11881, 2022. doi: 10.1609/ w....1..111.....
W. Hamilton et al. Inductive representation learning on large graphs. In Advances in Neurul Information Processing Systems (NewrIPS), pages 1024-1034, 2017.
Ralph E. Heimlich. Farm resource regions. Technical Report Agriculture Information Bulletin
No. 760, United States Department of Agriculture, Economic Research Service, 2000. URL https://www. ers usda, wttp:/ww........opppaaaaaaaaaaaa
S. Hochreiter and J. Schmidhuber. Long short-term memory, Neural Computation, 9(8):1735-11 1780,
1997. doi: www.:///neco.199..30.
J. Houghton, G. Jenkins, J. Ephraums, et al. Climate change. Technical report, Cambridge University Press, Cambridge, GB, 1990.
Alan Huete, Curtis Justice, and William van Leeuwen. Modis vegetation index (mod13) algorithm theoretical basis document. NASA Earth Science Research Laboratory, 2002.
Toshihiko Tizumi and Tomoko Sakai. Local temperature is more important than precipitation anomalies in determining wheat yield variability at the global scale. Climatic Change, 162(1):
119-136, 2020. doi:
IPCC. Climate change 2021: The physical science basis. contribution of working group i to the sixth assessment report of the intergovernmental panel climate change. 2021.
J. W. Jones, G. Hoogenboom, C. H. Porter, K. J. Boote, W. D. Batchelor, L. A. Hunt, P. W. Wilkens, U. Singh, A. J. Gijsman, and J. T. Ritchie. DSSAT cropping system model. Agronomy Journal, (3(2):240 253, 2001.
Saeed Khaki and Lizhi Wang. YieldNet: A convolutional neural network for simultaneous corn and soybean yield prediction from remote sensing data. Remote Sensing, 13(3):448, 2021.
Saeed Khaki, Lizhi Wang, and Sotirios V. Archontoulis. A cnn-rnn framework for crop yield prediction. Frontiers in Plant Science, 10:1750, 2019. doi: (0.3389//flss2211.00777
13

---

Published in Transactions Machine Learning Research (03/2026)
Marcus Krisp and Sebastian Scheinert. Red-edge spectroscopy of vegetation and the importance of swir bands. Remote Sensing Letters, 12(9))815-824, 2021.
G. Leng and W. Hall. Predicting spatial and temporal variability in crop yields: an inter- comparison of machine learning, regression and process-based models. Environmental Research Letters, 1544)0044027 2020. doi: 10.1088/ (L1088/1748-10 00018///749992///11 9326/3376777
Fudong Lin, Summer Crawford, Kaleb Guillot, Yihe Zhang, Yan Chen, Xu Yuan, Li Chen, Shelby Williams, Robert Minvielle, Xiangming Xiao, Drew Gholson, Nicolas Ashwell, Tri Setiyono, Brenda S. Tubana, Lu Peng, Magdy A. Bayoumi, and Nian-Feng Tzeng. MMST-ViT: Climate change-aware eтоp yield prediction via multi-modal spatial-temporal vision transformer. In Proceedings of the IEEE/CVE International Conference on Compuler Vision (ICCV), pages 1000-1010, 2023.
Fudong Lin, Kaleb Guillot, Summer Crawford, Yihe Zhang, Xu Yuan, and Nian-Feng Tzeng. An open and large-scale dataset for multi-modal climate change aware crop yield predictions. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 5375 5386, 2024.
Liu, C. Goering, and L. Tian. A neural network for sctting target corn yields. Transactions of the ASAE, 44(3):705, 2001.
David B. Lobell, David Thau, Craig Seifert, Edward Engle, and Boyana Little. A scalable satellite- based crop yield mapper. Remote Sensing of Environment, 164:324-33 333, 2015. doi: 10.1016/j.rs 2015.04.021.
Carmen Monfreda, Navin Ramankutty, and Jonathan A. Foley. Farming the planet: 2. geographic distribution of erop areas, yields, physiological types, and net primary production in the year 2000. Global Biogeochemical Cycles, 22(1)2 2008. doi: 2000099/007
A. Ortiz-Bobea, F. Knippenberg, and R. G. Chambers. Growing climatic sensitivity of agriculture linked to technological change and regional specialization. Science Advances, reaa)33a) 2018. doi: 10.1126 /sci///saav..... /st///ddaa.a..444333.3. aat4343.
M. J. Robertson, P. S. Carberry, G. L. Hammer, A. M. Whitbread, S. T. Cawthray, M. A. Foale, D. P. Holzworth, N. I. Huth, B. A. Keating, A. D. Moore, P. L. Poulton, and W. R. Tahnk. APSIM: Agricultural production systems simulator. Environmentol Modelling & Software, 39: 205-2155 205-215,40 2012.
Vivien Sainte Fare Garnot and Loic Landrieu. Panoptic segmentation of satellite image time series with convolutional temporal attention networks. In International Conference on Computer Vision (ICCV), 2021.
Kaitlyn Anita Spangler, Emily K. Burchfield, and Britta Lee Schumacher. Past and current dynamics of U.S. agricultural land use and policy. Frontiers in Sustainable Food Systems, 4:1-15, 2020. doi: http://ssss..22...00. URL hhttps://www www.:aww..oooteriin..co wrw.a/rccaaaartttrtt....oom wtt.c//eeeee...3///ffff... 2020.00104/fu11
Don V. Topping, Ja-Hong Chun, Harsha Thimmappa, Changqi Huang, Xuelei Yang, Feng Gao, Dennis Helder, and Bo Tan. The impact of atmospheric correction on sentinel-2 1lc 12a reflectance. Remote Sensing of Enwironment, 227:123-1323 2019.
14

---

Published in Transactions Machine Learning Research (03/2026)
Gabriel Tseng, Ivan Zvonkov, Catherine Nakalembe, and Hannah Kerner. Cropharvest: A global satellite dataset for crop type classification. In Neural Information Processing Systems (NeurIPS), 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, 2017. doi: 10.48550 htt.///aXxiv...........
Wenhai Wang, Enze Xie, Xiang Li, Deng- Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In International Conference on Computer Vision (ICCV), pages 548-558, 2021. doi: 10. 2A1109/CCCV48922.2022 2009900099
You, X. Li, M. Low, D. Lobell, and S. Ermon. Deep gaussian process for crop yield prediction based on remote sensing data. In Thirty-First First AAAT Conference on Artificial Intelligence, 2017a.
Jia You, Xin Li, Morgan Low, David B. Lobell, and Stefano Ermon. Deep gaussian process for crop yield prediction based on remote sensing data. In Proceedings of the Thirty-Fire AAAI Conference on Artificial Intelligence, volume 31, pages 4559 4566, 2017b. URL https: 7/
C. Zhao, B. Liu, S. Piao, X. Wang, D. B. Lobell, Y. Huang, M. Huang, Y. Yao, S. Bassu, P. Ciais, et al. Temperature increase reduces global yields of major crops in four independent estimates. Proceedings of the National Academy of Sciences, 114((5):9326 9331, 2017. doi: 10. 17177//1
A Appendix / supplemental material
Table Dataset parameters used in the study
Dataset
Parameters
SENTINEL-2
RGB imagery 9x9 km grids, 40 meters resolution
USDA
Production (Corn and Soybean). Yield (bu/acre)
HRRR
Averaged Temperature, Maximal Temperature, Minimal Temperature, Precipitation, Relative Humidity, Wind Gust, Wind Speed, Downward Shortwave Radiation Flux, Vapor Pressure Deficit
The CropNet dataset is publicly available (cited [in et al., 2024]) and accessible at https:// huggingface... aungginngaace.co/dataatttttt CropNet//CropNet http///atsstts/rppleee////////// All code used in this study is from open-source repositories, with links provided as foolnotes in the main paper: GNN-RNN (https://github com/ JunvenBai/CNN-RIg MMST-ViT
/https://gits //ttt:///gttttttt//e htttss///gghuu...c//uuuu///////
Linear baseline using Lasso regression We applied the Lasso (Least. Absolute Shrinkage and Selection Operator) regression method predict crop yield using structured imputs derived from weather, soil, and management data. The model trained standardized output variables
15

---

Published in Transactions Machine Learning Research (03/2026)
using only the training data. A grid search was conducted over a predefined range of regularization strengths (a 00.01,0.1,1, 0,1, 10, 100, 1000, 10000)); and each configuration random seeds to account for training variability. The best- best-performing configuration was selected based on the average test set performance, evaluated using root mean squared error (RMSE), coefficient of determination (I²²), and Pearson correlation coefficient. Finally, predictions were rescaled to the original units.
evaluated across three
GNN-RNN model The GNN-RNN model is a deep learning framework designed to capture both spatial and temporal dependencies in crop yield prediction. Traditional yield prediction models often assume that counties are independent, ignoring the geographic correlations that influence crop growth. To address this limitation, (Fan al.
2022 proposed the GNN- -NN architocture. It builds upon the CNN-RNN framework (Khaki et al.,
2019) by introducing graph- based component that refines county-level embeddings using neighboring counties' data before passing them to the LSTM. Experimental results indicate that GNN-RNN outperforms CNN-RNN multiple yield prediction tasks, highlighting the importance of incorporating geospatial information in deep learning models Fan et al.) 2022). The GNN-RNN model consists of three key components:
Feature Embedding Extraction: Each county's climatic and soil fcatures are processod separately before being fused into an annual embedding. The model employs ID CNN to extract temporal features from weekly weather and land surface data, while a separate CNN processes static soil quality features. These embeddings concatenated to form county-year-leell representation Fan et al. 2022).
Graph Neural Network (GNN) for Spatial Dependencies: The model represents counties as nodes in a graph, where edges connect neighboring counties based on geographic adjacency. A GraphSAGE-
GNN (Hamilton et al. 2017) aggregates information from a county's 's surrounding areas, allowing local predictions to be influenced by broader regional trends (Fan jet al., 2022 The GNN-processed embeddings incorporate spatial dependencies, ensuring that the model captures geographic influences on crop yield.
Recurrent Neural Network (RNN) for Temporal Dependencies:A Long Short-Tern Memory (LSTM) network (Hochreiter and Schmidhuber 1997) processes multi-year embeddings for each county, modeling long-term climatic effects and historical yield trends (Fan et al. 2022 The final output embedding from the LSTM is used to predict county-level crop yields for the current year.
MMST-VIT Model Unlike the GNN-RNN model which relied solely on meteorological and static environmental fcatures AMST-ViT integrates remote sensing data (satellite imagery) and weather-based climate indicators. The of transformer-based atlention mechanisms allows the model to capture both fine-grained spatial dependencies and long-term temporal trends. The MMST-ViT model consists of three key components:
Remote-Sensing imagery Fcature Extraction: Pyramid Vision Transformer (PVT) WWag
et al. 2021 is used to analyze remote sensing imagery. To miligate the tendency of transformer- based to overfit, the Pyramid Vision Transformer (PVT) backbone pretrained
using contrastive learning framework similar to SimCLR (Chen et al. 2020 This
16

---

Published in Transactions Machine Learning Research (03/2026)
pretraining strategy enhances the model's ability to generalize by learning robust feature representations from unlabeled data.
Spatial Transformer for Geographic Dependencies::MMSTIVIL employs self-attention mech- anisms to implicitly learn spatial dependencies. This approach is motivated by (Dosovitskiy et al. 2021), who showed that vision transformers can outperform CNNs by capturing global spatial information without predefined adjacency constraints. The self attention mechanisms allows the model to learn spatial correlations across regions (counties, in this study).
Temporal Transformer for Climate Trends: To explicitly incorporate climate trends, Temporal Multi-Head Attention module models long-term climate variability, aligning with recent work emphasizing the growing influence of climate change agricultural yields (Ortiz-Bobea et al. 2018). Unlike LSTM-based approaches, which have been widely used for time-series forecasting in agriculture (You et al. 2017a),, the transformer-based framework allows MMST-ViT to capture long-range dependencies more efficiently (Vaswani et al. 2017).
Table 9: Scnsitivity analysis parameters* and model hyperparancters for the GNN-RNN model. Parameters marked with * were varied individually during sensitivily analysis. Bold values indicate the baseline configuration used in the main experiments.
Hyperparameter
Values Tested or Used
Sensitivity Parameters (Ablated)
Number GNN Layers* (Mayers)
2, 3, 4 pool, mean,
Aggregation Type"
Dropout Rate*
0, 0.5, 0.7
Fixed Hyperparameters
Temporal Sequence Length (At)
5 years
Learning Rate
le
Batch Size
64
LR Scheduling
Cosine To = 100, Mmin = le-6)
Epochs
200
Optimizer
Adam
Loss Function
Log-cosh
A.1 UMAP weather embeddings
To evaluate the robustness of our models in realistic out-of-distribation (OOD) deployment scenarios, we leverage historical weather data and visualize its structure using UMAP embeddings. Specifically, we extract weekly weather variables from 2017 to 2022 and compute their low-dimensional UMAP reprosentations. Each embedding is then colored according to its USDA Farm Resource Region (FRR) to assess the coherence of the USDA-based clustering.
As shown in Figure 2 the weather embeddings naturally cluster according USDA Farm Resource Regions, confirming that the USDA FRRs naturally form distinct clusters based weather patterns.
17

---

Published in Transactions Machine Learning Research (03/2026)
Table 10: Sensitivity analysis parameters" and model hyperparameters for the GNN-RNN model. Parameters marked with were varied individually during seusitivity analysis. Bold values indicate the baseline configuration used in the main experiments.
Hyperparameter
Values Tested Used
Sensitivity Parometers (Ablated)
Embedding Dimension*
128, 512
Dropout Rate*
0, 0.3, 0.5
Shared Training Configuration
Optimizer
AdamW
Weight Decay
0.05
Learning Rate Schedule
Cosine decay with warmup
Pretraining
Epochs
200
Initial Learning Rate
le-4
Warmup Epochs
20
Batch Size
64
AdamW (2)
(0.9, 0.95)
Augmentation
Random crop, horizontal flip, Gaussian blur, color jitter
Fine-tuning (Supervised)
Epochs
100
Initial Learning Rate
le-3
Warmup Epochs
5
AdauW (31, (2)
(0.9. 0.999)
18

---

Published in Transactions Machine Learning Research (03/2026)
UMAP Projection of Scaled Duta with Encoded Colors
10
1IMAP Dimenston
6-
Color Categories
Easten-Uglannk
2-
Heartland
Mistisiipi-PPoall
Northen-Crresent
Northen-GreeatPPee Plains
Praini-Gatewwy
Soutbemn- Seaboard
10
UMAP
Figure 2: UMAP visualization of weekly weather embeddings (2017-2022), colored by USDA Farm Resource Regions. The clustering structure confirms that USDA FRRs provide meaningful partitioning of agricultural regions based on weather characteristics.
This provides further evidence that USDA FRRs are a meaningful and scientifically grounded way partition the data for OOD evaluation. To systematically evaluate model generalization under varying levels of distribution shift, define three out-of-distribution (OOD) scenarios- Caaari 1, Case 2, and Case 3 based on qualitative synthesis of multiple indicators. Specifically, we consider: (i) the distribution of clusters in UMAP space, (ii) their geographical proximity, and (iii) performance patterns observed in leave-cluster-out (LCO) cross-validattin. While the cases are not explicitly labeled by difficulty, we hypothesize that Case likely corresponds to an easier distribution shift, involving test regions that are closely aligned with the training distribution in both UMAP space and geography. Case 2 appears to reflect a moderate shift, with partial separation in UMAP space and moderate geographic distance. Case 3 seems to represent. the most challenging scenario, involving substantial separation in UMAP space, geographic remoteness, and typically larger performance drops in LCO validation.
19

---

Published in Transactions Machine Learning Research (03/2026)
Table 11: Performance metrics for Seaee-OO--Region--ee year-ahead evaluation for soybean yield prediction using GNN-RNN model. [1,d,type] corresponds to number of layers, dropout and aggregation type of the graph convolutional network.
Method Excl RMSE Incl
R Incl
Corr Incl
RMSE Exel
R Excl
Corr Excl
Reg
Lasso
PG
10.90
0.43
0.02 0.03
0.69 0.05
17.03 + 1.27
-0.11 0.04
0.19 = 0.06
baseline
EU
11.44
0.62
0.14 0.04
0.64 0.06
10.90 + 0.87
0.14 0.04
0.67 0.05
HL
10.58 + 0.82
0.12 + 0.03
0.52 0.07
15.87 + 1.28
-2.47 + 0.15
0,31 = 0.03
MSP
10.41 = 0.71
0.32 + 0.05
0.63 + 0.06
12.08 + 1.07
-1.35 0.12
-0.15 = 0.02
NC
12.39 = 0.91
0.07 + 0.03
0.60 0.05
8.17 = 0.45
0.10
0.03
0.44 = 0.03
NGP 11.08
1 0.77
0.18 1 0.04
0,60 + 0.06
11.32 1 1.14
-1.42 1:2:10 0.13
0.04 = 0.00
SS
10.86
+ 0.68
0.26 4 0.05
0.61 + 0.06
10.62 1 0.95
0.10 -1.06
-0.04 + 0.00
1=2,1=0
FU
8.60 0.42
0.52
0.05
0.73 0.05
6.96 0.81
0.65 0.05
0.84 + 0.05
.pool
HL 9.60 + 0.65
0.28 + 0.04
0.56 + 0.06
8.31 + 0.76
0.05 = 0.03
0.55 = 0.06
MSP 8.33 = 0.51
0.57 + 0.05
0.76 + 0.05
10.22 + 0.83
-0.68 + 0.08
-0.10 0.00
NC 9.07 + 0.58
0.50 + 0.05
0.73 + 0.05
10.28 + 0.91
-0.42 + 0.06
-0.18 0.00
NGP 8.77 + 0.63
0.48 0.05
0.72 + 0.06
9.53 + 0.84
-0.71 + 0.09
-0.6 0.01
PG 7.06 + 0.45
0.59 + 0.06
0.78 0.05
23.26 + 2.18
-0.56 + 0.05
0.32 = 0.01
SS
8.88 = 0.57
0.51 + 0.05
0.73 = 0.05
10.83 = 0.99
-0.64 0.06
-0.39 = 0.01
1=2,d=0 EU
8.74 + 0.53
0.50 0.05
0.72 0.06
5.97 1 0.62
0.74 1 0.05
0.86 1 0.05
.mean
HL 9.08 + 0.48
0.36 + 0.04
0.60 + 0.06
7.94 + 0.81
0.13 + 0.04
0.58 = 0.06
MSP 7.90 + 0.47
0.61 0.06
0.79 0.05
9.37 + 0.75
-0.41 + 0.07
-0.07 0.00
NC 9.16 + 0.70
0.49 + 0.05
0.71 + 0.06
9.43 + 0.50
-0.20 + 0.05
0.06 + 0.00
NGP 8.26 = 0.73
0.54 + 0.05
0.74 0.06
7.51 + 0.62
-0.06 + 0.04
0.29 = 0.01
PG 7.04 0.46
0.59 + 0.06
0.77 0.05
23.57 2.25
-1.12 0.07
0.27 = 0.02
SS 8.98 = 0.67
0.50 + 0.05
0.72 0.06
11.78 + 1.14
-153 + 0.07
-0.39 = 0.01
1=2,d=0 EU 9.09 + 0.63
0.46 + 0.05
0.69 + 0.06
6.75 1 0.54
0.67 1 0.05
0.83 = 0.06
.8rm
HL 8.90 + 0.71
0.38 + 0.04
0.63 + 0.06
7.74 + 0.58
0.17 + 0.04
0.64 + 0.05
MSP 8.43 + 0.55
0.56 + 0.05
0.75 + 0.06
9.39 0.93
-0.42 + 0.07
-0.00 0.00
NC 8.93 = 0.49
0.52 + 0.05
0.72 = 0.06
10.05
-0.36 = 0.03
-0.13 0.00
NGP 9.00 = 0.52
0.46 + 0.05
0.71 0.06
8.98 4.98+080
-0.52 0.04
-0.03 0.00
PG
7.25 0.65
0.57 0.06
0.77 0.05
25.33 + 1.86
-1.45 +
0.07 = 0.00
SS
8.08 + 0.66
0.59 0.06
0.78 0.05
11.15 + 1.22
-1.27 + 0.06
-0.39 + 0.01
1=3,-=0 EU
9.29 + 0.57
0.44 + 0.05
0.71 + 0.06
7.72 + 0.75
0.57 + 0.05
0.83 + 0.06
.pool
HL 9.88 +: 0.83
0.24 1: 0.04
0.54 1: 0.06
8.58 1 0.84
-0.02 1 0.03
0.56 1 0.03
MSP 8.26 + 0.74
0.57 + 0.05
0.76 0.06
10.63 + 1.08
-0.82 0.10
-0.02 + 0.00
NC 8.63 + 0.60
0.55 + 0.05
0.74 0.06
9.55 + 0.96
-0.23 + 0.06
0.09 = 0.00
NGP 8.07 + 0.51
0.56 0.05
0.76 0.06
6.38 + 0.58
0.23 + 0.05
0.56 + 0.03
PG 7.57 0.49
0.53 0.05
0.74 0.06
24.20 2.31
-1.23 0.07
0.25 = 0.02
8.65 0.56
0.53 0.05
0.73 0.06
11.36 + 1.15
-135 0.09
-0.43 0.03
1=4,d=0 EU
8.91 + 0.67
0.48 1 0.05
0.71 + 0.06
6.36 1 0.49
0.71 1 0.05
0.85 1 0.05
.pool
HL
9.53 + 0.65
0.29 + 0.04
0.58 + 0.06
8.07 0.90
0.10 + 0.04
0.59 = 0.06
MSP
8.15 + 0.49
0.58 + 0.06
0.78 0.05
10.45 + 1.10
-0.76 0.08
-0.10 + 0.00
NC
9.12 + 0.66
0.49 1 0.05
0.74 0.06
9.77 - 0.85
-0.29 0.07
-0.03 0.00
NGP 8.20 0.59
0.55 0.05
0.77 0.06
6.29 0.63
0.25 0.05
0.57 0.06
PG 7.49 0.77
0.54 0.05
0.75 0.06
24.99 + 2.13
-1.38 0.09
0.31 0.02
Continued next page
20

---

Published in Transactions Machine Learning Research (03/2026)
Continaed from premious page
SS
8.95 + 0.64
0.50
0.05
0.71 0.06
12.07 + 1.22
-1.66 0.07
-0.41
0.02
1=2,0=0.5EU
8.35 + 0.63
0.55 + 0.05
0.74 + 0.06
6.33 + 0.53
0.71 + 0.05
0.85 + 0.06
.pool
HL 9.83 = 0.71
0.24 + 0.04
0.54 + 0.06
8.47 + 0.90
0.01 + 0.03
0.54 = 0.06
MSP 9.24 0.51
0.47 + 0.05
0.69 0.06
9.38 + 0.91
-0.42 + 0.08
0.10 = 0.00
NC 8.95 + 0.62
0.51 + 0.05
0.72 0.06
10.26 1.01
-0.42 + 0.07
-0.06 + 0.00
NGP
8.58 0.56
0.51 0.05
0.72 0.06
8.70 0.81
-0.43 0.08
0.26 0.00
PG
7.65 0.64
0.52 0.05
0.73 0.06
23.08 + 2.25
-1.03 0.08
0.25 0.01
SS
8.09 0.71
0.59 0.06
0.77 0.05
1.26 11.39
-1.37 0.07
-0.35 0.01
1=2,d=07EEU
8.81 i 0.73
0.49 1 0.05
0.70 + 0.06
6.49 1 0.57
0.69 1 0.05
0.84 = 0.06
.pool
HL 9.61 + 0.55
0.28 + 0.04
0.55 + 0.06
8.08 + 0.68
0.10 + 0.04
0.53 + 0.06
MSP 8.65 + 0.66
0.53 + 0.05
0.74 0.06
9.61 + 1.06
-0.49 + 0.06
0.01 + 0.00
NC 9.48 0.55
0.46 0.05
0.75 0.06
10.97 = 1.28
-0.62 0.07
40.03 0.00
NGP 9.67 0.75
0.37 0.04
0.66 0.06
10.38 = 0.96
-1.23 0.05
40.15 0.00
PG 7.27
0.56 0.06
0.76 0.05
24.01 2.33
-1.20 0.06
0.18 0.00
7.94
0.61 0.06
0.79 0.05
10.54
0.07
-0.29 0.01
Ablation CNN-
EU 9.93 + 0.71
0.36 + 0.04
0.63 0.06
7.59 + 0.65
0.58 + 0.05
0.84 + 0.06
RNN
HL 9.06 0.67
0.36 0.04
0.66 0.06
8.81 0.74
-0.07 0.02
0.71 0.04
MSP 9.33
0.46 0.05
0.71 0.06
9.59 0.89
0.03 -0.48
0.14 0.01
NC
0.75
0.37
0.66
9.07
0.14 0.00
NGP 9.00 0.68
0.46 0.05
0.70
8.11 0.79
0.01 -0.24
0.47 0.02
PG 10.23 0.77
0.14 0.04
0.57 0.07
19.41 + 2.17
-0.44 1:44:00 0.01
0.19 = 0.00
7.98 0.59
0.60 0.06
0.79 0.05
10.20 - 1.15
-0.90
0.02
-0.18 0.00
Table 12: Performance metrics for Ceee--nOn-RRiinn-n year-ahead evaluation for soybean yield prediction using MMST-ViT model. [emb,drop] corresponds to the embedding dimension and dropout of the model.
Method Exel RMSE Incl
R Incl
Corr Incl
RMSE Excl
R Excl
Corr Excl
Reg
Lasso
PG
10.90 + 0.43
0.02 + 0.03
0.69 0.05
17.03 + 1.27
-0.11 + 0.04
0.19 + 0.06
baseline
EU
11.44 + 0.62
0.14 i: 0.04
0.64 0.06
10.90 1 0.87
0.14 0.02
0.67 = 0.05
HIL
10.58 1 0.82
0.12 0.03
0.52 0.06
15.87 1 1.28
-2.47 1 0.15
0.31 0.01
MSP
10.41
0.71
0.32 0.05
0.63 0.06
12.08 +
-1.35 0.12
40.15 0.00
NC
0.91
0.07 0.03
0.60 0.05
0.45
0.10 0.00
0.44 0.01
NGP
11.08
0.77
0.18
0.60 0.06
11.32 +
-1.42
0.00
10.86
0.68
0.26 0.05
0.61 0.06
10.62 + 0.95
-1.06 0.08
-0.04 0.00
c=512,
EU
13.06 1 0.47
0.48 0.05
0.69 0.06
9.16 0.42
0.62 1 0.06
0.79 + 0.05
drop=0
13.46
0.61
0.30
0.04
0.55 0.06
11.67 + 0.53
0.10 0.00
0.32 0.02
MSP
0.49
0.52
0.06
0.72 0.05
12.71 0.58
-0.41 0.03
-0.09 0.00
NC
0.57
0.05 0.46
0.68 0.06
9.46 0.39
0.65
0.04
0.81 0.05
NGP
0.48
0.64 0.06
11.64 + 0.51
0.04
0.00
0.43
0.57
0.76 0.05
20.96 1 0.91
0.06
0.00
12.49
0.55
0.50
0.06
0.71 0.06
15.32 + 0.67
-1.24 0.04
-0.37 0.00
Continaed next
21

---

Published in Transactions Machine Learning Research (03/2026)
Continued from premious page
e=512,
EU
14.99 1 0.68
0.42
0.05
0.65 0.06
11.58 i 0.52
0.39 i 0.06
0.77 = 0.06
drop=0.3
ILL
17.18
0.77
0.23 0.04
0.48 0.06
10.19 + 0.47
0.09
0.04
0.30 0.01
MSP
0.62
0.49 0.06
0.70 0.06
24.07
-1.35
0.15
40.21 0.01
NC
15.04
0.66
0.40 0.05
0.63 0.06
13.00 + 0.59
-0.29 0.06
-0.12 0.00
NGP
14.72 0.63
0.38 0.05
0.62 0.06
14.66 +
0.07
-0.16 0.00
PG
13.48 0.60
0.50 0.06
0.71 0.06
23.44 = 1.01
-1.12 0.13
0.27 = 0.00
14.75 0.65
0.46 0.06
0.68 0.06
14.29 + 0.63
0.12 -0.99
-0.32 0.01
128,
EU
14.01
0.63
0.45 0.06
0.67 0.06
12.59 + 0.57
0.60 0.07
0.78 0.06
drop=0
HL 0.60
0.31
0.04
0.56 0.06
13.82 + 0.62
0.05 -0.79
-0.12
4..10 0.00
MSP
0.58
0.50
0.71 0.06
17.04 +
0.00
NC
0.61
0.43 0.05
0.66 0.06
12.89 + 0.58
0.07
0.06
NGP
14.17
0.59
0.40 0.05
0.63 0.06
9.20 0.42
0.05
0.64 0.05
PG
13.63
0.62
0.52 0.06
0.72 0.06
16.16 + 0.72
-0.85 0.12
0.20 = 0.00
S5
0.60 13.85
0.48 0.06
0.69 0.06
14.25 + 0.64
-0.99 0.13
-0.32 0.00
128,
EU
13.41
0.59
0.47 0.06
0.68 0.06
12.33 + 0.56
0.61 0.07
0.78 0.06
drop=0.3
13.34 0.58
0.32 0.04
0.57 0.06
13.27 1 0.60
-0.15 0.05
-0.11 0.00
MSP
13.32 0.57
0.51 0.06
0.72 0.06
13.74 + 0.41
-0.41 1 0.07
+0.00 0.00
NC 13.53
0.60
0.44 0.05
0.66 0.06
12.17 + 0.55
0.69 0.07
0.83 = 0.06
NGP
13.48 + 0.59
0.41 0.05
0.64 0.06
10.63 + 0.48
0.30 0.05
0.55 0.04
PG
12.63
+: 0.56
0.54 0.06
0.74 0.06
18.61 + 0.83
-1.01 + 0.13
0.23 - 0.00
SS
13.47
+ 0.61
0.49 0.06
0.70 0.06
11.41 + 0.52
0.65 1 0.07
0.81 + 0.05
e=128,
EU
16.03
0.72
0.40 0.05
0.63 0.06
14.75 + 0.66
0.59
0.06
0.77 0.06
drop=0.5
18.58 0.83
0.20
0.04
0.45 0.06
11.03 + 0.50
0.09
0.04
0.30 = 0.01
MSP
16.16
0.73
0.47 0.06
0.68 0.06
13.01 + 0.59
-0.41
0.07
-0.09 0.00
NC
15.67
+ 0.71
0.41
0.05
0.64 0.06
17.10 + 0.77
-0.79 1 0.12
-0.15 0.00
NGP
16.00
1 0.72
0.38 1: 0.05
0.62 0.06
14.61 1 0.66
-0.41
0.07
-0.16 1166:00 0.00
PC
14.29
1: 0.65
0.51 0.06
0.72 0.06
26.92
1 1.21
-1335
0.15
0.21 - 0.00
SS
15.36
+: 0.69
0.43 1 0.06
0.66 0.06
21.10
+ 0.95
-1.12 1 0.13
-0.32 + 0.01
Ablation HRRR-
15.30
0.67
0.32
0.04
0.57 0.06
11.50 + 0.52
0.54
0.07
0.74 0.06
only
HL
14.92
1: 0.63
0.28
0.04
0.53 0.06
12.75 1 0.58
-0.12 1 0.05
-0.08 0.00
MSP 15.67
1: 0.71
0.41
1 0.05
0.64 0.06
14.03
1 0.63
-0.38
0.07
-0.09 1 0.00
NC
15.11
0.68
0.36
0.05
0.60 0.06
11.63 + 0.53
-0.62
0.07
-0.54
NGP 15.18
0.69
0.34 + 0.05
0.58 0.06
8.82 + 0.41
-0.25 - 0.05
-0.23 12 0.00
PG 14.00
+ 0.61
0.44 + 0.05
0.66 0.06
25.71 + 1.17
-0.88 + 0.12
-0.19 0.00
SS 15.21 1 0.69
0.39 1: 0.05
0.62 + 0.06
13.60 1 0.62
-0.58 1 0.06
-0.22 0.00
OOD Scenario Definition and Performance Gap Metric
We define three out-of-distribnn dishrfidittiibtiinn (OOD) scenarios using USDA Farm Resource Regions to evaluate spatiotemporal generalization. These scenarios are heuristically designed based on geographic separation and UMAP-basod latent space dissimilarity. Each case involves training on crop yield data from 2017 2021 within selected regions and evaluating on an unseen region in 2022. To quantily the degradation in performance under distribution shift, we define normalized performance gap metric based on root mean squared error (RMSE):
22

---

Published in Transactions Machine Learning Research (03/2026)
RMSEOODD Model trained on 2017-2021 data from given Troin Region, evaluated on 2022 data from disjoint Test Region
Model trained on 2017-2021 data and evaluated on 2022 data from the same region used during training.
RMSEnD:
The relative performance degradation is computed
Performance Gap (%) RMSEOOD - RMSEED X 100
RMSE
This normalized performance gap provides fair comparison of model robustness across regions with inherently different data complexity or signal quality, It adjusts for baseline model performance in each region, removing confounding effects due to region-specific difficulty (e.g., climatic variability, sparse observations, or noisy yield labels). By anchoring performance to an in-distribution baseline, this metric isolates the effect of distribution shift itself, making it particularly well-suited for evaluating OOD generalization in worldworld, spatially heterogeneous dataset application such as crop yield prediction.
23


Reviews:
[{"review_id": "OvAigk3RaL", "reviewer": "Reviewer_g7ua", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a systematic evaluation of out-of-distribution generalization in crop yield prediction models under climate variability. The authors benchmark two deep learning architectures (GNN-RNN and MMST-ViT) using corn and soybean data from over 1,200 U.S. counties spanning 2017-2022 (the CropNet Dataset), employing USDA Farm Resource Regions as a basis for defining spatial clusters and OOD scenarios. \nThe main contributions of the paper are: (1) demonstrating significant performance degradation under spatial distribution shift; (2) identifying asymmetric transferability patterns; (3) showing GNN-RNN achieves superior cross-region stability compared to MMST-ViT despite requiring substantially less computational resources; and (4) establishing a standardized evaluation framework for agricultural ML.\n\nI find that the research question considers an important setting for model robustness: realistic spatio-temporal distribution shifts. The use of USDA Regions provides a grounded regional partitioning, and the transparent reporting of negative results is valuable.\n\nNonetheless the work presents some key weaknesses. The presentation lacks clarity in dataset composition, model adaptation procedures, and experimental setup. Missing explicit citations and insufficient methodological detail make the paper difficult to follow.", "claims_and_evidence_yn": "No", "claims_explanation": "While the paper presents extensive experimental results, several clarity issues prevent full assessment of the claims:\nThe dataset description in Section 3 is insufficient. It remains unclear whether Sentinel-2 imagery constitutes a separate dataset requiring integration with CropNet, or whether it is already included. The relationship between weather data, USDA yield data, and Sentinel-2 imagery needs explicit clarification.\nSection 5.1 states \"we first reproduced the GNN-RNN model using the same architecture and hyperparameters as reported in the original paper\" without explicitly citing which paper at that specific point. While Fan et al. 2022 appears later, the immediate context lacks this citation.\nThe experimental setup lacks critical details about model-to-dataset adaptation. How is the GNN-RNN graph structure defined? How are Sentinel-2 images processed and integrated with weather variables in MMST-ViT? The paper jumps to results without adequately describing these procedures.", "audience_yn": "Yes", "audience_explanation": "The research addresses a specific case of OOD generalisation in spatiotemporal settings on real data. \nThe negative results (models achieving negative R2 under certain transfers) could provide valuable insights about failure modes that are often underreported (but they need to be explored more in detail). \nHowever, the current presentation significantly limits accessibility. The paper requires substantial revision to clarify the experimental setup, dataset details, and model adaptation procedures before acceptance.", "requested_changes": "Critical Changes (required for acceptance):\n\n1. Clarify dataset composition and model integration: There is a value in explaining the dataset and which attributes it contains. Describe the data pipeline: preprocessing steps, spatial and temporal alignment, and final input format for each model.\nFor GNN-RNN, describe how the graph structure is constructed—what defines an edge, typical neighbor count, and handling of isolated counties. For MMST-ViT, describe how the Sentinel-2 patches are matched to counties (given variable grid counts per county), how the four spectral bands are processed, and how imagery features are fused with HRRR weather variables.\n2. Add explicit citations throughout Section 5.1: When referencing \"the original paper\" for GNN-RNN, immediately cite Fan et al. 2022. Similarly, cite Lin et al. 2023 when discussing MMST-ViT reproduction.\n3. Expand Section 4: For a practical paper the authors should provide a clear description of: (a) exact train/validation/test splits, (b) hyperparameter selection procedure, (c) evaluation metrics computation, (d) number of random seeds, and (e) computational requirements. Consider adding a figure illustrating the experimental pipeline. Best of all would be to share a code to reproduce the result of the paper.\n4. Add a baseline experiment training on all regions: Include an experiment where models train on all seven USDA regions (2017-2021) and test on year 2022 across all regions. This establishes a baseline when no spatial shift is present and quantifies the cost of spatial generalization.\n\nWhile reading the work I also think that this work would improve with some baselines and additional explorations\n\n5. Provide feature importance analysis: Identify which input features (weather variables, soil properties, spectral bands) are most predictive through ablation studies, attention weight visualization, or permutation importance. This would explain why certain regions transfer poorly and provide actionable insights.\n6. Quantify degree of distribution shift: Beyond categorical USDA regions, provide quantitative measures of how \"out-of-distribution\" each test region is using statistical distance metrics (MMD, Wasserstein distance) on weather features or UMAP embedding distances. Correlate these with performance degradation to understand whether performance scales predictably with shift magnitude.\n8. What are the main reasons why Prairie Gateway specifically fails? Can the authors comment a bit on this?", "broader_impact_concerns": "The paper includes an appropriate Broader Impact Statement discussing how regional disparities in model performance could exacerbate existing inequalities in agricultural decision support. \nNo additional ethical concerns beyond those identified by the authors.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TRcXGofbe3", "reviewer": "Reviewer_fVyG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper is an evaluation of how the two models GNN-RNN and MMST-VIT perform under temporal and spatio-temporal OOD in the CropNet dataset containing data from 1200 US counties split into 7 clusters.\n\n### Strength:\n1. The paper’s motivation to examine how crop yield prediction models are affected by OOD shifts across time and space is relevant.\n2. Designing three different OOD difficulty scenarios based on UMAP in Table 4 is a good starting point.\n\n### Weakness:\n1. The paper is structured as a sequence of reproduction studies, ablation studies, and discussions on the limitations of existing models and the dataset itself, but lacks a coherent narrative flow. Some experiments do not appear directly relevant to the stated motivation of investigating temporal and spatio-temporal ood generalization.\n2. This lack of flow stems partly from missing or unclear references between sections. For example, Tables 5 and 6 appear in the main content but are never explicitly referenced. Section 5.2 claims to analyze temporal OOD performance in terms of R², yet none of the tables in the main content seem aligned with this discussion.\n3. The purpose of conducting a cross-region transferability study for each region is unclear, given that the paper’s stated goal is to study OOD behavior. Why not instead use the region groupings introduced in Table 4?\n4. The discussion section should focus more specifically on identifying which aspects of the model or dataset cause the observed lack of generalization under OOD settings.\n5. Using only RMSE and reporting its degradation without any confidence analysis weakens the qualitative findings.", "claims_and_evidence_yn": "No", "claims_explanation": "refer weakness", "audience_yn": "Yes", "audience_explanation": "refer strength", "requested_changes": "refer weakness", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mV0LNx8U8F", "reviewer": "Reviewer_DhdB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work evaluates two deep learning architectures for crop yield prediction in out-of-distribution settings. They find limited robustness of the two methods, especially if models are trained only on few data from few geographic areas and then applied to other geographic areas.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The work runs extensive experiments to assess the performance of the two methods and ensure their robustness to distribution shifts.\n\nOne minor problem was that the authors were unable to reproduce the results reported in the MMST-ViT paper. I would suggest the authors to reach out to MMST-ViT authors and ask for additional information to reproduce the original results. However, given the extensive description i trust the authors of this work to have put significant work into trying to reproduce MMST-ViT results, albeit so far unsuccessfully.", "audience_yn": "Yes", "audience_explanation": "This paper evaluates two methods which have been previously published at ML venues (AAAI & ICCV). It \"systematically studies the robustness or generalizability of the published methods\" and is thus within scope of TMLR.", "requested_changes": "1. Remove \"climate-aware\" from the title. This work has little to nothing to do with climate change, so using this term is misleading.\n2. Change the classification of page 2: it makes little sense to classify methods as to either use remote sensing or meteorology, as those two are rather complementary types of features and should thus be used simultaneously.\n3. I don't get Table 1. Perhaps it should be streamlined to match the storyline and particularly make clear why all the experiments are necessary to have a robust OOD evaluation -- and not just state what the experiments do.\n4. Table 5: add the LOO results to this table as an additional row: how good are the models, if trained on all other regions except for the test region?\n5. Table 5 & 6: you need to give the colorscale\n6. Phrase some sentences more carefully throughout the work, e.g. \"While deep learning shows promise for climate-resilient agriculture\" in the conclusion, seems a bit strong wording: crop yield prediction is not really the same as \"climate-resilient agriculture\".", "broader_impact_concerns": "Broader impact statement is OK.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)