Title: MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation
Benchmark: MMM-RS (Multi-modal, Multi-GSD, Multi-scene Remote Sensing)
Summary: MMM-RS is a large, multi-modal multi-GSD, and multi-scene remote sensing text-to-image generation benchmark. It is publicly available, aggregated and filtered from existing datasets, and contains information-rich captions. 
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: remote sensing text-to-image generation
Phenomenon defined by authors: No
Phenomenon definition: Remote sensing image generation is the ability to prompt a multimodal model to generate a high-quality remote sensing images. 
Definition scope: Comprehensive
Definition integrity: Single cohesive phenomenon
Definition integrity detail: Not applicable
Phenomenon contested: Widely-agreed
Task definition: A multimodal model is given an information-rich text prompt and must generate the described remote sensing image. 
Task item definition: An remote sensing image and an information-rich text prompt, specific to its image modality. For example, the prompt may contain satellite type, weather type, category, resolution, subject, etc.  
Task source: Modified from another benchmark (e.g. translation into another language), LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task source detail: "The MMM-RS dataset is derived from 9 publicly available RS datasets: MRSSC2.0 [16], Inria [19], NaSC-TG2 [45], GID [28], WHU-OPT-SAR [14], HRSC2016 [40], TGRS-HRRSD [42], fMoW [5], and SEN1-2 [25]" (4). It contains images across three modalities: RGB, Synthetic Aperture Radar, and Near Infrared. Multi-scene remote sensing images are synthesized at different scales and weather conditions using physics models and multimodal models. The process is outlined in Figure 4 on Page 6. 
Task ecology: Complete real task (e.g. providing medical advice to real people interactively)
Dataset size: 2,103,273
Dataset size notes: Optional train set is defined at 200,000 samples. 
Dataset metadata available: No
Dataset metadata detail: =
Sampling method: Targeted items (creators defined a task space and chose tasks within it strategically), Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test, Train
Response format: Image
Metric definition: Distribution (perplexity, calibration, correlation)
Metric definition detail: The paper uses Frechet Inception Distance (FID) and Inception Score (IS).
Metric aggregation: Simple Mean
Metric statistics: Simple mean
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: The benchmark is itself realistic
Human baseline: No
Authors discuss validity: Yes
Author validity detail: The authors highlight that a remote sensing text-to-image generation dataset should be multimodal across data and image types, and be information-rich. 
Taxonomy: Grounding / 
New taxonomy: General Capability / Grounding