COSMIC: Compress Satellite Image Efficiently via Diffusion Compensation
Authors: Ziyuan Zhang
Han Qiu
Zhang Maosen
Jun Liu
Bin Chen
Tianwei Zhang
Hewu Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Image compression
Satellite images
Generative model
Probabilistic diffusion models
Primary area: generative_models

Abstract: With the rapidly increasing number of satellites in space and their enhanced capabilities, the amount of earth observation images collected by satellites is exceeding the transmission limits of satellite-to-ground links. Although existing learned image compression solutions achieve remarkable performance by using a sophisticated encoder to extract fruitful features as compression and using a decoder to reconstruct. It is still hard to directly deploy those complex encoders on current satellites' embedded GPUs with limited computing capability and power supply to compress images in orbit. In this paper, we propose COSMIC, a simple yet effective learned compression solution to transmit satellite images. We first design a lightweight encoder (i.e. reducing FLOPs by 2.5~5X) on satellite to achieve a high image compression ratio to save satellite-to-ground links. Then, for reconstructions on the ground, to deal with the feature extraction ability degradation due to simplifying encoders, we propose a diffusion-based model to compensate image details when decoding. Our insight is that satellite's earth observation photos are not just images but indeed multi-modal data with a nature of Text-to-Image pairing since they are collected with rich sensor data (e.g. coordinates, timestep, etc.) that can be used as the condition for diffusion generation.  Extensive experiments show that COSMIC outperforms state-of-the-art baselines on both perceptual and distortion metrics.

COSMIC: Compress Satellite Images Efficiently via Diffusion Compensation
Ziyuan Zhang' Han Qiu Maosen Zhang
Jun Liu
Bin Chen² Tianwei Zhang* Hewu Li
1 Tsinghua University, China
Harbin Institute of Technology, Shenzhen, China
Nanyang Technological University, Singapore
(ziyuan-z23, zhangms24) aomails.tsinghiee. edu. en, fqiuhan.jumeliu chenbin20210hit. edu. tianwei. zhangōntu. 1ihemu@cernet..er
junelin)@tsinghua. edu.
Abstract
With the rapidly increasing number of satellites in space and their enhanced capabil- ities, the amount of earth observation images collected by satellites is exceeding the transmission limits of citiite-tt-grunng links. Although existing learned image compression solutions achieve remarkable performance by using a sophisticated encoder extract fruitful features as compression and using a decoder to recon- struct, it is still hard to directly deploy those complex encoders on current satellites' embedded GPUs with limited computing capability and power supply compress images in orbit. In this paper, we propose COSMIC, simple yet effective learned compression solution to transmit satellite images. We first design a lightweight encoder (i.e. reducing FLOPs by 2.6 5x) on satellite to achieve high image compression ratio save satellite-to to-ground links. Then, for reconstructions on the ground, to deal with the feature extraction ability degradation due to simplifying encoders, we propose diffusion-baan decoding. Our insight is that satellite's 's earth observation photos not just images but indeed multi-modal data with a nature of Text-to-Inee to-Image pairing since they are collected with rich sensor data (e.g. coordinates, timestamp, etc.) that can be used as the condition for diffusion generation. Extensive experiments show that COSMIC outperforms state-of oftth-f-the-art baselines both perceptual and distortion metrics. The code publicly available at hhttps://githhi: com/ Joanna 0421/C0SMIC
model to compensate image details when
Introduction
The revival of the aerospace industry [19 38), coupled with reduced costs of launching rockets [2]. has fueled exponential increase in the number of nanosatellites, resulting in massive growth in images collected in-orbit. For example, the Sentinel-3 missions can collect maximum of 20 TB raw data satellites (mainly earth observation images) every day 20). However, the data transmission capability between satellites and ground stations has clear upper bounds [8 45 [7]. This situation of the rapid growth of images collected by satellites versus the limited transmission capability the ground requires effective image compression on satellites before transmission back to Earth.
Current industrial compression solutions for satellite images rely JPEG [48), JPEG2000 [46), CCSDS 123 [29] (c.g. satellite BiISAT-I (56)). These solutions are outperformed by various learned compression methods [22 in various cases. Existing learned image compression methods (39 36 53
use sophisticated encoders to extract fruitful features and then use decoder to decompress
32
*Corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Earth observation missions (e.g. NASA's Landsat Program 44 involve the use of satellite photos to monitor and collect data for tasks like forestry [5), agriculture [4 land degradation (15), land Traditional earth observation Although we notice novel promising trend of deploying embedded GPUs on satellites in both academia [T6]
and industry (e.g. satellite Phi-Sat-1 [25]. Chaohu-1 [3] and Forest- 1 4 which brings the potential opportunity of using learned compressors on satellites. It is still hard to directly adopt existing learned compression solutions for satellites since their sophisticated encoders are still too complex for GPUs on satellites (e.g. NVIDIA Jetson Xavier NX on Forest-1 4 which have limited computing capacity and power supply M. We have two insights to fill the above gaps. (1) We first design lightweight encoder on satellites with higher priority of compression ratio than feature extraction ability. (2) At the receiver's s end on the ground, we deal with this simple encoder's s feature extraction ability degradation by compensating image contents when decoding. We choose diffusion as compensation due to its powerful generation capability and, more importantly, satellites' earth observation photos are not only images but enjoy a multi- modal nature in which rich real-time sensor information is the description of the corresponding photo. For instance, in Figure I the coordinates (e.g. latitude and longitude) denote the location of the image which describes its main category (e.g. sea, city, etc.) and the timestamp describes the image's 's lightning- day or night.
T7 6
In this paper, propose to COmpress Satellite iMage via dIffusion Compensation (COSMIC), novel learned image compression method for satellites. COSMIC has two key components, i.e., (1) a lightweight encoder for compression satellite and (2) a sophisticated decompression process with decoder and a diffusion model on
Satelitte earth doservation imago
Downlink
Description.
the ground with sufficient GPUs. First, we de-
Satellta's is sensar information collected with the image
sign a lightweight convolution architecture to ex-
Ground Sample Desancc
0.5884
tract local features and apply convolution to ob-
neoes
2B 900T" N. 77 REPP W
Cloud Cover
31
tain an attention map of global features, to real- ize a lightweight image compression encoder in terms of FLOPs. Please note that lightweight en- coders usually extract fewer key features which increases the difficulty for the decoder decom- press. Thus, our second component, decompres- sion, has two parts including decoding with corresponding decoder and, more importantly, a compensation model. Inspired by the multi-modal nature of the satellite's images, we aim to build text-to-image pairs (image and its sensor information like Figure D and use diffusion as the compensation model.
Ground
Timestarpp
2015 the Day: 28 Hour
station
Entropy Gosling
1000110111000100001110000000
Figure An example of the satellite's earth ob- servation image and this image's corresponding sensor data as a description.
We compared COSMIC with 6 state-of-the-art (SOTA) baselines, 3 of which are based on generative models, considering both distortion and perceptual metrics, In addition, we constructed two image compression test sets based on satellite images by considering ordinary scenes and unique tile scenes in satellite imagery. Extensive experiments have proven that COSMIC significantly reduces the encoder's ' complexity 2.6 5x fewer FLOPs while achieving better performance on almost all metrics than basclines. Our contributions can be summarized as follows.
We propose a novel idea that uses a lightweight image compression encoder on satellites and leverages satellite images' text-to-image pairing nature for compensation when decompressing. We propose a novel compensation model based on stable diffusion to compensate image details when decompressing with the unique sensor data of satellite images as descriptions.
We analyzed the characteristics of satellite images in detail and incorporated them into the training and inference stages. In addition, we constructed two datasets under satellite image transmission scenarios, taking into account the typical satellite image transmission tasks like tile scenes.
2 Background
2.1 Earth observation missions on satellites
use and land cover 40). biodiversity (34), and water resource 35 missions rely on pipeline which satellites take photos and then send them back to ground stations for analysis. Recently, along with the reduced cost of launching rockets and manufacturing
77

---

the photos on satellites are rapidly increasing which brings novel challenges for transmitting photos back to the ground. A recent promising approach to deploy embedded GPUs on satellites to support DNN models to either filter useless data before transmission or make partial processing tasks on satellites. For instance, ESA's satellite Phi-Sat-1 [25] first deploy Intel VPU on satellite to support DNN models for filtering useless photos (e.g. covered by clouds) that can save 30%+ transmission volume. OroraTech has launched AI nanosatellites with the NVIDIA Jetson Xavier NX for wildfire detection [4], and Orbital Sidekick uses NVIDIA Jetson AGX Xavier as the AI engine at the edge of the satellite to detect gas pipeline leaks (2]. However, due to the inelastic computational capabilities of onboard satellites and limited power supply only from the sunshine (i.e., up to 15 Watt for GPUs on satellites [4], a certain amount of images are still needed to be transmitted back to the ground. This brings an urgent need for satellite-specie image compression methods.
nanosatellites [2]]
2.2 Neural image compression methods
Learned image compression methods have achieved remarkable rate-distortion performance com- pared with classical information theory-based image compression methods, to each of which consists of an encoder &, a quantization Q. and a decoder D. The encoder, as the most critical part, extracts key features from the image as the latent representation. Higher quality representation extracted by the encoder means less content loss at compressing which is more likely to reconstruct a higher quality image when decompression. Thus, SOTA approaches explore introducing more complex modules into the encoder. [37 integrates the transformer into the CNN encoder and uses the Mansffrmer-NNN mixture block to extract rich global features. The other approaches aims to reduce the complexity of the decoders that are deployed on edge devices like smartphones. For instance, [54] adopts shallow or even linear decoding transforms to reduce the decoding complexity, compensated by more powerful encoder networks and iterative encoding.
Generative models for decompression. Although VAE-based methods have achieved good per- formances, optimizing solely for mean square error (MSE) can lead to excessive image smoothing, resulting visual artifacts. More recent works [36] models (e.g. diffusion) to achieve better visual results. 5 uses conditional diffusion model as the image compression decoder which improves visual results.
53
30 have combined VAEs with generative
30 and [24] decouple the compression
task and augmentation task, sending the output of the VAE codec to diffusion to predict the residual.
Satellite image compression method. There are some compression methods specifically for remote sensing images [57 23 (50). (50) uses discrete wavelet transform to divide image features into high- frequency features and low-frequency features, and design frequency domain incoding-decoding module to preserve high- feeh-freeuency information, thereby improving the compression performance. (3 explore local and non-local redundancy through a mixed hyperprior network improve entropy model estimation accuracy. Few of these works focus on onboard deployment. 2 use the CAE model to extract image features and reduce the image dimension to achieve compression, and deploy the model on VPU. However, this method only considers the reduction of image dimension and does not consider the arithmetic coding process in the actual transmission process, resulting in the image compression rate can only be adjusted by changing the model architecture.
Limitations to use for satellites. Most of the above approaches don't 't consider lightweight compres- sion encoders which makes them impractical to deploy on satellite's 's embedded GPUs constrained by computing capacity and power supply. The approach used on VPU impractical as the image compression rate is highly related to model architecture. Besides, none of them pay attention to the modtl-modal nature of satellite earth observation images to introduce conditions to further improve decompression quality.
3 Prelimiaries
3.1 Problem formulation
We formulate the basic process of learned image compression. The encoder E uses a non-linearly transformation to convert the input image x into the latent representation which is subsequently discretized and entropy-coded by quantization @ under a learned hyper prior Ç. Under the stochastic Gaussian model, each discrete code [y]; can be expressed as Gaussian distribution with mean in and variance oi given hyper prior Çi: p (lv1 = N/ of). The decoder D reconstructs the
3

---

discrete representation Ly] to the image X. The model can be optimized by the loss function (Eq.
Li = R + AD = E - log, p (1)) log, (c) + AE [d (x, x)]
(1)
where R is the bit rate of latent discrete coding, D is the distortion between the original and the reconstructed image (measured by MSE), and À controls the trade-off between rate and distortion.
3.2 Diffusion model
Diffusion model is type of generative model that can generate images from Gaussian noise through multi-step iterative denoising. These models include two Markov processes. First, the diffusion process gradually applies noise to the image until the image is destroyed and becomes complete Gaussian noise. Then, in the reverse stage, it learns the process restoring the Gaussian noise to the original image. During the inference stage, given random noise sample xT ~ '' (0, 1), the diffusion model can denoise through steps and gradually generate a photorealistic image Xo. At each step
E {0, 1, ..... T), intermediate variable X, be expressed as x, v1 B,x,- + Bict, where S. E (0. 1) is the variance hyperparameter of Gaussian distribution, and satisfies 31 < B2 <... < Sr; G / (0. 1) is the Gaussian noise at step In the diffusion model, noise prediction network (Eo) is used to predict the noise at step t, and - can be obtained from X
The diffusion model can also understand the content of the given conditions, such as text and images, and generate images consistent with the conditions. In this case, the noise prediction network takes three parameters: intermediate sample X,, timestep t, and given condition Ç as input. To guarantee the noise predicted by the noise prediction network at the t-th step of the reverse process has the same distribution the noise introduced into the image during the diffusion process, diffusion model usually use L2 to optimize the network following Vo ||c, - Eg (X,, t, <)ll-
Stable diffusion [42 is proposed to reduce the training cost, which implements the diffusion process in cow-dimensional latent space while retaining the digensinnnii information in the original pixel space for decoding. In this article, we aim to leverage the powerful generation ability of stable diffusion and its ability to maintain consistency with given condition to provide compensation for the information lost by the image compression encoder.
4 Method
4.1 Framework of COSMIC
As illustrated in Figure 2 COSMIC consists of two components: a compression module and a compensation module. To adapt to the satellite scenarios, the compression module includes lightweight image compression encoder & and an entropy model deployed on the satellite (Sec. (.2). as well as image compression decoder D deployed at the ground station. To fix the content detail loss caused by the lightweight encoder, a compensation module is proposed, which is entirely deployed at the ground station. has an encoder E, aiming to extract compensation information Zo from original images, which is received by decoder D compensation for latent representation y/ extracted by & during training (Sec. 4.3) During the inference phase, the noise prediction network generates compensation information 20/ from noise simulate 2g compensation (Sec. 4.4).
4.2 Lightweight image compression encoder
To make the image compression encoder & practical on satellites, we first make it lightweight. The main idea is to reduce the amount of calculation required for image compression terms of FLOPs. We followed the classic architecture of image compression encoder [10], which is structured with downsampling convolutions with a stride of 2 and generalized divisive normalization (GDN) 9 arranged alternately (Figure 2 (a)). Since GDN provides the best performance for image compression when the number of channels is 192 [8), the increase in convolution filters will exponentially increase the calculation amount of convolution. To reduce the amount of computation required for image downsampling, we propose a lightweight convolution block (LCB), as Figure 2 (d), which uses depthwise convolution to replace the ordinary convolution with a convolution kernel size of 5 x 5. To interact between different channels of the feature map, depthwise convolution followed by 1 x convolution with a full number of channels. Inspired by [27]. which proves that there is a lot of

---

(a) Compression Model
(c) CAM
GDN
GDN
CAM
L.C 12
GDN
CAM
1yyper Entropy Modal
Inpun
LCB
LCB
Triuge Compression Enciddrr 8"
5x1 CoTY
Ourpat Imagee
Image
Decoder
Cancat
Tiranscony
72
GDN
Tuanscon
12
dec
Sampling
(d) LCB
Imput Imagee
Image Ensonder
Diffusion
Process
Noise Prediction Network
W Book GAEka
Geb
Z4
oony
Coordinkaa
Tapouss
Metadata
Encoder
cony
Ofieade Angle
Lapet Acanadi
Sun Armuah
(b) Compensation Model
Only used at training
Figure 2: COSMIC framework. (a) Compression module for satellite images: a lightweight encoder and a compensation-basel decoder (Sec. 4.3) (b) In the noise prediction network, each Cross- Attention (CA) block receives embedding of the Metadata Encoder (ME) (Sec. (.4). and the Vanilla Convolution (VC) blocks use latent image discrete encoding to guide the prediction of noise for each diffusion step. (c) & (d) Convolution attention module and lightweight convolution block (Sec.
redundancy in the features extracted by convolution, in LCB, only half of the output feature maps are obtained through 1 1 convolution, and the remaining half of the output feature maps are obtained through linear transformations with cheap cost using redundant features.
Transformer- based methods [58] 37
outperform CNN-based methods the attention capture non-local information of the images. However, the computational complexity of self-attention has quadratic relationship with the size of the input feature map, which is not computationally friendly. Inspired by we use two one-dimensional convolutions in series. The first convolution is used to extract horizontal information. On this basis, the second convolution is used to vertically synthesize the previously extracted horizontal information to obtain the global attention map. Meanwhile, the other branch uses LCB with stride to capture local information, shown in Figure (c).
4.3
Compensation-guide image compression
After giving a lightweight design of the image compression encoder E, we note that the representation ability of this encoder is inevitably degraded. Specifically, the features contained in the latent representation Ly] received by the ground station are not enough to let the decoder reconstruct high-quality image. Thus, explore compensation for the degradation in encoding.
is well-known that stable diffusion has powerful generation capabilities for specified content from noise under the guidance of text information 42 The ground station can obtain not only the latent representation compressed by the encoder but also rich sensor data like the geographical location, time, camera parameters, etc. along with each image. We use these sensor data as conditions to guide diffusion generation fix the missing image details. The training divided into two stages. In the first stage, train the compression model. As shown in (Figure ( (b)), since the Image decoder D needs two parts information (i.e. y/ and Zo in Figure 2 for decoding, we introduce another image encoder E to extract compensation information zo from the original image. the first stage, &, & and DD are trained together. The reconstructed image & can be expressed as in Eq.
* D (concal (transcony (Lyl), (xx))
In the second stage of training, we freeze the parameters of E, E and D, and train the noise prediction network, with the goal of making the information generated by the diffusion model as close to Zo possible, denoted zol, to generate the compensation information required by the decoder.
5

---

During the inference phase, the trained diffusion model can generate compensation information Zo/. Therefore, we no longer need È. The Zo generated by the diffusion model replaces the Zo extracted by E to help the image decoder decompress the image.
4.4 Conditional diffusion model for loss compensation
As pointed out in Sec. D earth observation images collected by satellites indeed multi-modal data. Here we consider that a satellite image Xg, when transmitted to the ground station, contains a discrete image coding Lyl paired with its sensor information denoted numerical metadata m. For m E I just like diffusion handles timestep t, we use sinusoidal embedding (Esin) to encode them to 9j E 1x (j 1, .,),,, where d is the dimension of the clip embedding, as concatenate the metadata embedding together as description of an image. This process is expressed as in Eq. 3
final = MLP (concat ((eain (m1) ..., Esin (mm)]))
This final metadata condition will be incorporated into the latent representation using CrossAttention (CA) blocks to guide the generation process.
Stable diffusion that stable diffusion generates image details that are not extracted by the satellite image encoder E, and still retain the overall structure of the image. To address this problem, coding ly] into the Vanilla Convolution (VC) blocks of the noise prediction network to provide the structure information and improve content consistency, which can be described in Eq. 4
originally used for generative tasks, which have randomness. Here, expect
inject the discrete image
f f + projection; (Ly1),
where is the i-th feature map of the U-Net backbone. and projection; is the upsampling convolution used to align the dimensions between [y] and f. Guided by image coding and the metadata as description, we use MSE loss minimize the distance between target distribution and learned distribution in latent space as in Eq.
Lidm = (. [||4 - co (z, (,,,,,yy.ce lyl. |1²]
(5)
5 Experiments
5.1 Setup
Dataset. We use the function Map of the World (IMoW) [13], which has 62 categories, and in which each image is paired with different types of metadata features, as our training data and test data. resolution of 256 x 256 pixels. For information
For training data, we randomly crop the image collected by satellite sensors as metadata, we choose Coordinates, Timestamp, GSD, Cloud cover, Off-nadir Angle, Target Azimuth, Sun Azimuth, and Sun Elevation provided by fMoW.
For testing data, we constructed two test sets with different resolutions. One is from the fMoW test set, where ensure comprehensive representation of categories, we randomly selected one image from each category, cropped it to resolution of 256 x 256, and used as a standard test set. For another test set, we considered the actual scenario of the satellite construct tile test set. Since the image captured by the satellite is a large geographic region, the computing resources on the satellite are limited, and large-size images cannot be processed directly, so images should first be cut into smaller sub-images, this process is known as tiling [31 [1 17 [1]. In this paper, for images of size 2306 x 2306, we first divide each image into 81 smaller patches of size 256 x 256 each, and then compress and decompress each patch individually, as illustrated in Figure 5 (a). After obtaining all the decompressed patches, we reassemble them as one image for further evaluation.
For distortion comparison, we use the Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity Index Measure (MS-SSMM) to validate the pixel fidelity and measure brightness, contrast, and structural information at different scales. For perceptual comparison, we choose Learned Perceptual Image Patch Similarity (LPIPS) and Fréchet Inception Distance (FID).
Metrics. We use 4 metrics for quantitative measures following previous works 39
36 53
Model training details. The training has two stages. First, we train the image compression encoder E, image encoder & and image decoder D together using Lie for 100 epochs with batchsize of 32.

---

Second, we freeze the parameters of the model trained in the first stage, use the pretrained stable diffusion model for the noise prediction network, and finetune it using Liam for 10 epochs with a batchsize of 4. All the training experiments are performed on 10x NVIDIA GeForce RTX 3090 using Adam optimizer with Ir we utilize the DDIM sampling 43 with 25 steps.
10-4 4 and E {0.00067, 0.0013, 0.0026, 0.005]. During inference,
Baselines. We consider baselines including traditional methods, VAE-based methods, and genera- tive model based methods. Elic (28) proposes a multi-dimension entropy estimation model, which can effectively reduce the bit rate and improve the coding performance. Hific 39 pays more attention to the perception of the model reconstruction effect, obtaining visually pleasing reconstructed images. Based on Hific, COLIC [36] considers the semantic information of the image when designing the loss function, and treats structure and texture respectively. CDC [53] is the first work to use the diffusion model as an image compression decoder, performing the reverse process of diffusion in pixel space to reconstruct the image. HI RS [50] is an image compression method, especially for remote sensing images, which processes the figh-frequeccy part and the low-frequency part of the images separately to better preserve the important high-frequener features of remote sensing images. For these 5 baselines, we retrain their models with the IMoW dataset for a fair comparison. Besides, we choose JPEG2000, the industrial solution for satellite image compression [55] for comparison.
5.2 Comparison with baselines
RD performance. Figure 3 shows the comparison results with baselines on two test sets. The dotted line in the figure represents the baselines, and the solid line represents COSMIC. We demonstrate results from two perspectives, i.e., distortion and perception. Across all transmission rates, COSMIC surpasses the baselines in terms of LPIPS, and FID. At low hpp. the MS-SSIM of COSMIC is lower compared the baseline. This is mainly because the hpp decreases, the encoder extracts less information, and during the decompression process, there is a greater reliance on diffusion- -based generation (more details in Sec. 5.4). Additionally, due the degraded feature extraction of the lightweight encoder, the feature obtained at low bpp is insufficient to guide the diffusion process in generating high-fidelity images. As the bpp increases, the latent coding contains more features, resulting in a significant improvement in the MS-SSIM of COSMIC, demonstrating SOTA performance. Note that the sensor data is transmitted to the ground by default in earth observation missions. Besides, the volume of these sensor data is negligible compared with images so we do not consider them when counting bpp. We show more results of more metrics and baselines including other VAE-based methods in Appendix B which COSMIC achieves the SOTA performance on all 6 perceptual metrics.
PSNR 1
MS-SSIM 1
LPIPS
FID
32.0
0.39
100
8.6
fMoW testset
3.58
35.0-
+Bc
HLRS
5.37
COMNE
0.56
25.0
3.55
0.10
40
24.0
5.56
0.35
20
0.5
0.4
32.5
0.22
tile testset
28.5
LLKS
0.23
25.0
0.10
40
25.0
0.23
70
8pp
hpp
bpp
bpp
Figure 3: Trade-off between bitrates and different metrics on COSMIC and baselines. The f (L) means higher (lower) better. The first row is for the fMoW test set (image size 256 x 256). The second is for the tile test set by comparing between the stitched images and their original ones.
Visual results. Figure 4 shows the example of reconstructed images at low bitrates and high bitrates. For fair comparison, we only show the results of optimizing for image perception. Fig shows an example of high-resolution image reconstruction by COSMIC and baselines. Due to the tiling and
7

---

stitching process in image compression, we pay particular attention to the seams where different small patches form larger image. JPEG2000 exhibits noticeable misalignment at the image seams. Hific and COLIC can not accurately restore the details of the seam. For example, in the picture outlined orange, the car headlight at the seam has been reconstructed into a red dot. The diffusion- CDC reconstruction also exhibits some color differences between different sub-images and shows noticeable misalignment, such as the caves at the seam in the picture outlined in red. Compared to baselines, COSMIC maintains higher similarity in structure and color between different sub-images, resulting in significant visual improvements.
Original
JPEG2000
COLIC
HIFIC
CDC
COSMIC
[0 0.20p7/PNN222..29
[0.19 10.19gpPSNR227.66 62]
[0285 PSNR28851
Original
P2
COLIC
HIFIC
CDC
COSMIC I A7
10.7 to (0PP [0.0M//5R229..3 [086mpPSNR:302]]
Figure 4: Decompressed fMoW images (full images in supplementary material). 1s row: comparison under low bitrates, COSMIC shows better visual effects. Compared with CDC, COSMIC still gets slightly better visual reconstruction with less bitrates. 2nd row: comparison under high bitrates.
5.3 Encoder efficiency analysis
We evaluate the lightweight en- coder of COSMIC in terms of FLOPs in Table [ Compared with baselines, the on-satellite FLOPs (including the encoder and entropy model) of COSMIC has been significantly reduced by roughly 2.6 5x while the overall performance of COSMIC can still outperform baselines un- der similar bitrate.
Table 1: Comparison of the on-satellite FLOPs with baselines on
the tile test set. Best performances highlighted in bold.
Method
FLOPs (G)
PSNKT
MS SS--
LPIPS.
FID.
bppl
CDC
31.98
0,982
45.49
COLIC 36
26.4
29.04
0.975
0.0530
32.56
0.64
Hific 39
26.4
29.11
0.977
0.0384
27.01
0.60
Blic 28
21.78
33.31
0.983
0.0683
60.80
0.54
HI.
11.87
31.30
0.979
0.0782
27.38
0.56
COSMIC
4.9
32.11
0.980
0,0359
13.50
0.59
5.4 Ablation study
w/o DC. To demonstrate the compensatory role of diffusion in the image reconstruction process, we remove the diffusion compensation module, and the results shown in Figure 6 We remove the compensation module and retrain the model to show the quantitative metrics. The result shows that the diffusion model plays an important role decompressing images get better perceptual metrics. To show the compensatory role of the diffusion model more clearly, we directly remove the diffusion model and only use the output information the lightweight encoder E reconstruct the image. We find that at low bitrate, diffusion compensation is more important. Due to the insufficient feature extraction capability of the lightweight encoder, many image content details are lost to save the transmission rate, and diffusion needs to reconstruct much of the image content guided by the limited output of the encoder, along with the metadata. As the bitrate increases, more features extracted by the encoder can be retained, and at this point, diffusion only needs to compensate for some image details that the encoder failed to capture. The visual results are shown Figure 7 The results indicate that using the diffusion model as compensation is very useful, especially with small bitrate.

---

COSMIC
Illustraioon for tile testset
Original
JPEG2100
COLIC
HIFIC
CDC
COSMIC (0:3)T/NN:22475
Original
(020bg//SNN:22277)
(02061///5N::220000 [041hbg1/SR2268
(b) Visual result
Figure 5: (a) Illustration of the tile test set. A high-resolution image is divided into many small sub- images (or patches), each which is compressed individually. The reconstructed sub-images are then placed back in their original positions and stitched together to form a high-resolution reconstructed image. (b) On the tile test set, we provide two detailed views a stitching area (outlined orange and red). The visual comparison between COSMIC and the bascline shows that COSMIC achieves the best visual effects in terms of texture alignment and consistency in color brightness.
Table 2: Effect on image classification model.
Classes
Original
JPEG2K [46]
Elic [28]
COLIC [36
HIFIC [9]
CDC [53
HL RS 50
COSMIC
10
98.95%
-4211%
-5.27%
-1.06%
-1.06%
-2.11%
-2.11%
-1.06%
15
97.92%
-4.17%
-3.84%
-0.70%
-0.70%
-1.40%
-1.39%
-0.70%
98.42%
-3.16%
-3.68%
-0.53%
-0.53%
-1.06%
-2.1%
-1.06%
w/o CAM. To show the CAM module can capture non-local information, which can help the encoder get higher quality representation. We remove the CAM module and show the results in Figure The result shows that CAM module is effective and can achieve better RD performance.
w/o ME. To show that sensor data guide the generation of diffusion, encoder and only use image encoding for diffusion generation. Please note that there are mainly two kinds of sensor data here including the camera parameters such as the off-nadir angle and target azimuth used during image capture and the data on cloud cover, illumination, and ground sample distance, etc. Figure indicates that sensor data helps guide the diffusion in reconstructing the image.
remove the metadata
Influence of decoding steps. We further investigate the impact of different denoising step counts in the reverse diffusion process the reconstructed image quality, shown in Figure 8 We find that as the number of denoising steps increases, the perceptual metrics of the generated images gradually improve, which is consistent with the exploration of the relationship between denoising steps and FID scores in DDIM [4 While the perceptual results improve, distortion metrics such as PSNR experience slight decline, showcasing the trade-off between perceptual quality and distortion. In practical deployment and downstream tasks, the number denoising steps can be selected based on different emphases on distortion and perceptual performance to suit actual applications.
5.5
Compression influence downstream tasks
To confirm that COSMIC will not affect the downstream remote sensing tasks, such as image classifi- cation, we choose the image classification task in 7 to show the effect caused by compression, as shown in Table 2 The same test images are compressed at low bitrate level and decompressed, then processed through the classification model to obtain the accuracy changes on different numbers of

---

PSNR
MS-SSIM
LPIPS
Wh
3.50
COOMIC
0.37
0.150
20
3.40
0.122
0.100
3.45
Figure 6: Ablation study with different variants of COSMIC. "w/o DC indicates that diffusion compensation is not used during decoding. "w/o CeM'ddnotes the CAM module is removed in the encoder. "w/o ME" denotes that we remove the metadata encoder.
classes. The result shows that JPEG2000 and Elic reduce the accuracy by the most and COSMIC is outstanding learning methods.
Original
w/ DC
w/ DC
Original
DC
w DC
[0:4RRPPSNR 17.1] [0ARk2)/SNR22633]
[0.800)pP5N222077]
Figure 7: Contrast visual results. "w/ DC means using of diffusion for compensation during decoding, while "w/o DC indicates that diffusion compensation not used during decoding.
PSNR
MS-SSIM
LPIPS
FID
10p4
0.025
0.580
20.0
27.5
0.970
0.955
25.2
26
0.055
0.500
17.5
bpp
Figure 8: Compression performance with different numbers of decoding step.
Conclusion
We present COSMIC, novel approach to compress images for satellite earth observation missions. We first design a lightweight encoder to adapt to the limited resources on Satellites. Then, we introduce a conditional latent diffusion model by using the sensor data of satellite as instructions to compensate the missing details due to the lightweight encoder's degradation of feature extraction. Extensive results indicate that COSMIC can only achieve SOTA compression performance for typical satellite tasks but also guarantee the accuracy of satellite images' downstream tasks.
Limitations & Future work. Even though COSMIC compensates for encoder limitations using diffusion, at extremely low bpp (c.g., less than 0. 1bpp), the information provided by latent image coding may not be enough to support diffusion in generating high-fidelity images. We think the main reason is that we only finetune the pretrained Stable Diffusion model, which lacks sufficient prior knowledge of satellite images. As prospective solution, will train diffusion model specifically for satellite images or use historical satellite images as a reference for further improvements.
Acknowledgement. This work was supported by the National Key R&D Program of China (2022YFB3105202) National Natural Science Foundation of China (62106127, 62301 189, 62132009), and key fund of National Natural Science Foundation of China (62272266).
10

---

References
[1] 1.
jetson-xaver-nni
[2]
[3]
sttp:///ww.ttarddoccc.a/o-o-.0. htmml
[4] Shoot from the stars: Startup provides early detection of wildfires from space. https
[5] Frédéric Achard, Hans-Jürgen Stibig, Hugh D Eva, Erik J Lindquist, Alexandre Bouvet, Olivier Arino, and Philippe Mayaux. Estimating tropical deforestation from earth observation data. Carbon Management, 1():271-227, 2010.
[6] Caleb Adams, Allen Spain, Jackson Parker, Matthew Hevert, James Roach, and David Cotten. Towards an integrated gpu accelerated soc as flight computer for small satellites. In 2019 IEEE aerospace conference. pages 1-7. IEEE, 2019.
[7] Ali Bahri, Sina Ghofrani Majelan, Sina Mohammadi, Mehrdad Noori, and Karim Mohammadi. Remote sensing image classification via improved cross-entropy loss and transfer learning strategy based on deep convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 1786)11087-109,, 2020.
[8] Johannes Ballé. Efficient nonlinear transforms for lossy image compression. In 2018 Picture Coding Symposium (PCS), pages 248-252. IEEE, 2018.
[9] Johannes Ballé, Valero Laparra, and Eero P Simoncelli. Density modeling of images using generalized normalization transformation. arXiv preprint urXiv:1511.0628/ 2015.
[10] Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression with scale hyperprior. arXiv preprint arXiv:1802.01436, 2018.
[11] Mare Bosch, Kevin Foster, Gordon Christie, Sean Wang, Gregory D Hager, and Myron Brown. Semantic stereo for incidental satellite images. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1524-1532. IEEE, 2019.
[12] Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pages 7939-7948, 2020.
[13] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6172-6180, 2018
[14] Laura Crocetti, Matthias Forkel, Milan Fischer, František Jurečka, Aleš Grlj, Andreas Salentinig, Miroslav Trna,, Martha Anderson, Wai-Tim Ng, Ziga Kokalj, et Earth observation for agricultural drought monitoring in the pannonian basin (southeastern europe): current state and future directions. Regional Environmental Change, 20:1-17, 2020.
[15] Rogier de Jong, Sytze de Bruin, Michael Schaepman, and David Dent. Quantitative mapping of global land degradation using earth observations. International Journal of Remote Sensing. 32 (21):6823-688) 6853, 2011.
[16] Bradley Denby and Brandon Lucia. Orbital edge computing: Nanosatellite constellations as a new class of computer system. In Proceedings of the Twenty-Fisth International Conference on Architectural 2020.
Programming Languages and Operating Systems, pages 939-954,
[17] Bradley Denby, Krishna Chintalapudi, Ranveer Chandra, Brandon Lucia, and Shadi Noghabi. Kodan: Addressing the computational bottleneck in space. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, pages 392-403, 2023. 2023.
11

---

[18] Kiruthika Devaraj, Ryan Kingsbury, Matt Ligon, Joseph Breu, Vivek Vittaldev, Bryan Klofas, Patrick Yeon, and Kyle Colton. Dove high speed downlink system. 2017.
[19] Lauren Dreyer. Latest developments spacex's 's falcon 1 and falcon 9 launch vehicles and dragon spacecraft. 2009 IEEE Aerospace conference, pages 1-15. IEEE, 2009.
[20] Thomas Esch, Soner Üreyen, Julian Zeidler, Annekatrin Metz- Metz-Marconcini, Andreas Hirner, Hubert Asamer, Markus Tum, Martin Böttcher, S Kuchar, Vaclav Svaton, et al. Exploiting big earth data from space-first experiences with the timescan processing chain. Big Earth Data, (1):66-53 55, 2018.
[21] M Esposito, BC Dominguez, M Pastena, N Vercruyssen, SS Conticello, van Dijk, PF Manzillo, and R Koeleman. Highly integration of hyperspectral, thermal and artificial intelligence for the esa phisat-1 mission. In Proceedings of the International Astronautical Congress IAC, Washington, DC, USA, pages 21-25, 2019.
[22] Warren Frick and Carlos Niederstrasser. Small launch vehicles-a 2018 state of the industry survey, 2018.
[23] Chuan Fu and Bo Du. Remote sensing image compression based the multiple prior informa- tion. Remote Sensing, 15(8):2211, 2023.
[24] Noor Fathima Khanum Mohamed Ghouse, Jens Petersen, Auke J Wiggers, Tianlin Xu, and Guillaume Sautiere. Neural image compression with a diffusion- based decoder. 2022.
[25] Gianluca Giuffrida, Luca Fanucci, Gabriele Meoni, Matej Batič, Léonie Buckley, Aubrey Dunne, Chris Van Dijk, Marco Esposito, John Hefele, Nathan Vercruyssen, et al. The ø-sat-1 mission: The first on-board deep neural network demonstrator for satellite earth observation. IEEE Transactions on Geoscience and Remote Sensing, 60: 10:1-14, 2021.
[26] Giorgia Guerrisi, Fabio Del Frate, and Giovanni Schiavon. Artificial intelligence based on-board image compression for the 6-sat-2 mission. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.
[27] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 1580-1589, 2020.
[28] Dailan He, Ziming Yang, Weikun Peng, Rui Ma. Hongwei Qin, and Yan Wang. Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 57 7718-5727, 2022.
[29] Miguel Jernández-Cabronero, Aaron B Kiely, Matthew Klimesh, Ian Blanes, Jonathan Ligo, Enrico Magli, and Joan Serra- Sarra-Sagrista. The cesds 123.0-b-2 "Jow-complexity lossless and near-lossless multispectral and hyperspectral image compression" " standard: A comprehensive review, IEEE Geoscience and Remote Sensing Magazine, 9(4): 102-119, 2021.
[30] Emiel Hoogeboom, Eirikur Agustsson, Fabian Mentzer, Luca Versari, George Toderici, and Lucas Theis. High-fidelity image compression with score-based generative models. arXiv preprint arXiv:2305.1825. 2023.
[31] Bohao Huang, Daniel Reichman, Leslie M Collins, Kyle Bradbury, and Jordan M Malof. Tiling and stitching segmentation output for remote sensing: Basic challenges and recommendations. arXiv preprint 1805 12219, 2018.
[32] Nick Johnston, Elad Eban, Ariel Gordon, and Johannes Ballé. Computationally efficient neural image compression. arXiv preprint arXiv:1912.08771 2019.
[33] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models, Advances in Neural Information Processing Systems, 35:23593-23606, 2022.
12

---

[34] Claudia Kuenzer, Marco Ottinger, Martin Wegmann, Huadong Guo, Changlin Wang. Jianzhong Zhang, Stefan Dech, and Martin Wikelski. Earth observation satellite sensors for biodiversity monitoring: potentials and bottlenecks. International Journal of Remote Sensing, 35(18): 1599-6647, 2014.
[35] Richard Lawford, Adrian Strauch, David Toll, Balazs Fekete, and Douglas Cripe. Earth observations for global water security. Current Opinion in Environmental Sustainability, 5(6): 633-643, 2013.
[36] Meng Li, Shangyin Gao, Yihui Feng, Yibo Shi, and Jing Wang. Content-orientee learned image compression. In European Conference on Computer Vision, pages 632-647. Springer, 2022.
[37] Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer- cnn architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14388 19988-14377,1 2023.
[38] Henry Martin, Conor Brown, Tristan Prejean, and Nathan Daniels. Bolstering mission success: Lessons learned for small satellite developers adhering to manned spaceflight requirements. 2018.
[39] Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity generative image compression. Advances in Neural Information Processing Systems, 33: 1913- 1 11924, 2020.
[40] Prem Chandra Pandey, Nikos Koutsias, George P Petropoulos, Prashant K Srivastava, and Eyal Ben Dor. Land use/land cover in view of earth observation: Data sources, input dimensions, and classifiers review of the state of the art. Geocarto International, 36(9))957-983) 2021.
[41] George P Petropoulos, Prashant K Srivastava, Maria Piles, and Simon Pearson. Earth observation-base operational estimation of soil moisture and evapotranppiration for agricultural crops in support of sustainable water management. Sustainability, 10(1):181, 2018.
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[43] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
[44] Ychui Tang, Kai Han, Jianyuan Guo, Chang Xu, Chao Xu, and Yunhe Wang. Ghostnetv2: enhance cheap operation with long-range attention. Advances in Neural Information Processing Systems, 35:9969 9982, 2022.
[45] Bill Tao, Om Chabra, Ishani Janveja, Indranil Gupta, and Deepak Vasisht. Known knowns and unknowns: Near-realtime earth observation via query bifurcation in serval. 21 USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 809-824, 2024.
[46] David S Taubman, Michael W Marcellin, and Majid Rabbani. Jpeg2000: Image compression fundamentals, standards and practice. Journal of Electronic Imaging, 11(2):286-287, 2002.
[47] Soner Uereyen and Claudia Kuenzer. A review of earth observation-based analyses for major river basins. Remote Sensing, 11(24):2951, 2019.
[48] Gregory K Wallace. The jpeg still picture compression standard. Communications of the ACM, 34(4):30 1991.
[49] Michael A Wulder, David P Roy, Volker C Radeloff, Thomas R Loveland, Martha C Anderson, David M Johnson, Sean Healey, Zhe Zhu, Theodore A Scambos, Nima Pahlevan, et al. Fifty years of landsat science and impacts. Remote Sensing of Environment, 180:113195,0 2022.
[50] Shao Xiang and Qiaokang Liang. Remote sensing image compression based on high-frequency and low-frequency components. IEEE Transactions Geoscience and Remote Sensing, 2024.
13

---

[51] Chen Xu, Xiaoping Du, Zhenzhen Yan, and Xiangtao Fan. Cloud-bal based parallel tiling algorithm for large scale remote sensing datasets. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pages 4030-4033. IEEE, 2022.
[52] Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. arXiv preprint arXiv:2209.0663 2090069000 2022.
[53] Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[54] Yibo Yang and Stephan Mandt. Computationalll--ffiiicnn neural image compression with shallow decoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 530-540, 2023.
[55] Xiaqiong Yu, Jinxian Zhao, Tao Zhu, Qiang Lan, Lin Gao, and Lingzhi Fan. Analysis of JPEG2000 compression quality of optical satellite images. In 2022 2nd Asia-Pacific Conference on Communications Technology and Computer Science (ACCTCS), pages 500-503. IEFE, 2022.
[56] Gokhan Yuksel, Onder Belce, and Hakan Urhan. Bilsat-1: 1: First earth observation satellite of turkey-operations and lessons learned. In Proceedings of 2nd International Conference Recent Advances in Space Technologies, 2005. RAST 2005., pages 846-851. IEEE, 2005.
[57] Lei Zhang, Xugang Hu, Tianpeng Pan, and Lili Zhang. Global priors with anchored-stripe attention and multiseale convolution for remote sensing images compression. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.
[58] Renjie Zou, Chunfeng Song, and Zhaoxiang Zhang. The devil is in the details: Window-bar attention for image compression. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pages 17492-17501, 2022.
14

---

A Details about COSMIC
A.1 Details of train and inference
We present more detailed explanation of our two-stage training and inference pipeline in algorithm T and algorithm 2
Algorithm 1: Two-stage training of COSMIC.
input :A satellite image x, referenced image x(the same as satellite image, used only for training), metadata mi E RR
output : Reconstructed image &
Parameters: Image encoder on satellite &, image encoder of diffusion E, image decoder D, noise prediction network Eg
/* Training stage 1: Train E, E, and D together.
y & (x):
2 < entropy_mmdel(())
M2 o2 & (xo);
4 E ~ N (0,I);
Zu ll + Oz Ezi
6 = D (concat (Zu, deconv (lyl)));
optimize the parameters &, E, and D following:
Lı = R AD E E log, p ([y]l<) log, p (C) + AE [d (x, 8)]
/* Training stage 2 :Frezee the parameters of E, E, and D, and only update the parameters of EØ.
repeat
sample x, m ~ dataset;
t~ 4 (1,2,.. (1.2..... T);
11
(0.T);
Giial = meetadata_encoderrie
12
optimize the parameters of eæ following:
[|e, en (z t, Ly], cfinal) ||? [1²]
Liam E.,0,))
until converge;
Algorithm 2: Compress and decompress pipeline of COSMIC.
input : satellite image metadata mn;
T
output : Reconstructed image &
Parameters: Image encoder on satellite E, image decoder 2, noise prediction network eg
/* Compress
& (x);
entropy_modelle
3 y Quantization (y, <);
Decompress
sample Zr N (0,
for t T, T...., do
en (Ze, 1, Ly], clual): _ + DDIM (z,, t,);
end
return up/:
10 x
D (concal (zol, deconv ([yl)));
15

---

-e- PEG2000
Ch20
Minnen_2018
-s-
CDC
Hific
COUIC
ELIC
H5,R
OOSMIC
LPIPS
92
2.20
14
02
42
0.
0.1
KID
DISTS 1
TOPIQ FR
0.10
3.26
0.50
2.05
0.14
0.70
0.92
0.5
3.6
9.7
0.35
0.3
3.5
0.4
0.5
0.6
0.7
03
0.6
bpp
bpp
hpp
GMSD
MAD
VSI
130
2.03
120
0.07
20
70
5.03
0.55
5.02
2.00
0.0
3.9
333
0.4
0.5
0.5
0.0
03
bpp
bpp
PSNR
MS-SSIM
CW-SSIM
32
2.95
2.800
2.444
2.022
27
2.900
3.35
0.5
2.5
3.7
0.0
3.9
3.3
0.1
0.5
0.5
0.7
0.0
0.3
0.4
0.5
bpp
bpg
bpp
Figure 9: Cate-DDistrrioon(Pereettoon) for fMoW dataset.
B Additional Rate-DissortionnPPercpiin)) Results
C Failure cases analysis
Stable diffusion was originally used for generative tasks, which can produce textures and content that do not exist in reality. In COSMIC, we ensure the consistency of content and texture by injecting discrete latent coding Ly into the Vanilla Convolution (VC) blocks of the noise prediction network provide structural information. However, in
cases, especially at extremely low bitrates, the latent discrete coding may lack structural information, leading to the diffusion generating non- existent textures. As illustrated in Figure 10 (a). when satellite image captures an ocean scene, the insufficient information in the latent coding at low bitrates requires heavy reliance on diffusion for generation, resulting textures that do not exist the reconstructed image. In the of high bitrates, as described in the main paper, the reliance on diffusion reduces, and in the same scenario, the appearance of non-existent textures can be controlled. However, in practice, remote sensing downstream tasks are more concerned with regions of interest (ROIs). For example, in remote sensing tasks for oceans, such as ship detection. only the parts where ships are present are of interest. As shown in Figure 10 (b), even at low bitrates, while the reconstruction of the sea surface may introduce non-existent textures, the reconstruction of the ship parts remains intact. Therefore, although COSMIC may have minor visual flaws, it does not affect downstream detection tasks.
16

---

Original
BPP:0.024 PSNR: 25.73 aa)
BPP:02217
PSNR:34.00
Original
BPP:0310 PSNR: 32.99
(b)
Figure 10: Failure examples.
D Why not artifact correction for JPEG2000?
The most direct way to improve the quality of satellite images is to use artifact correction on JPEG2000 compressed images. This approach does not introduce the computational burden on the satellite and can also improve image quality certain extent. We use DDRM (33, which is an image restoration method, for JPEG2000 compressed images to remove the artifact. As shown in Figure [T] DDR slightly better than JPEG2000 but the improvement is limited, and there is still big gap with COSMIC. We believe that this is mainly due to the fact that traditional compression methods based on static compression procedures cannot fit the data well like neural networks, and therefore lose large amount of information, making image restoration methods difficult to improve the image quality. As result, it is necessary to design a learned compression method to improve satellite image compression quality without introducing excessive computational burden on the satellite.
E Influence of random seed
To show the influence of different initial gaussian noise on the quality reconstructed images, we randomly select 5 different seeds for each bit rate and show the 2-Sigma Error Bars results in Table 3 The results demonstrate that our method has small variances various quantitative metries, proving the robustness of our method to initialized random Gaussian noisc.
17

---

JPEG2000
JPEG2006
COSMIC
28
3.28
0.08
0.37
2.36
0.915
26
0.034
0.05
0.25
0.04
0.54
0.07
Figure 11: Comparison results of COSMIC, JPEG2000 and JPEG2000+DDRM.
Table The results of different random seeds for IMoW dataset.
bpp
PSNR
MS-SSIM
LPIPS
FID
0.31
25.3698 + 0.0447
0.9521 + 0.0004
0.1040 + 0.0010
31.2750 0,4328
0.46
27.2411 + 0.0222
0.9682
0.0004
0.0761 + 0.0006
23.0243 + 0.2253
0.61
28.6654 + 0.0251
0.9799 + 0.0004
0.0461 + 0.0004
19.4264 + 0.1059
0.76
29.3617 + 0.0608
0.9852 + 0.0006
0.0363 = 0.0007
16.9090
= 0.1683
Additional visualization of reconstructed images
We show the full images Figure 4 and Figure 5] the main paper, and give more visualization results, as shown in Figure 12
We select visualization results for two different test sets under higher bitrates and lower bitrates from each dataset. For the tile test set, under lower bitrates, as shown in Figure in in the main paper, the reconstructed images by the baselines exhibit noticeable discontinuities at the stitching seams.
25
18

---

EHEPPIR
Original
COSMIC 10.21bpp/PSNR:28.0 70
CDC (0.28bpp/PSNR-28.55
JPEG2000|0228p/PSMM222222
HIFIC [0.19bbp/PSNR:27.62]
COLIC [0200pp//PSN::27.28]
Figure 12: Reconstructed fMoW images under low bitrates.
19

---

Original
COSMIC
30.85|
CDC [0.82bpp/PSN::30.28]
JPE22000 [O. (bpp9p//PSN:229111
HIFIC [0.6899)/PSNN-29.33]
COLIC [0.74bp//PSNR:29.00]
Figure 13: Reconstructed fMoW images under high bitrates.
20

---

Figure 14: Ground truth.
Figure 15: Low hpp: COSMIC, PSNR==4.75, bpp=0.30.
21

---

Figure 16: Low hpp: HIFIC, PSNR=24.04, bpp=0.24.
Figure 17: Low hpp: COLIC, PSNR-23.77, bpp=0.24.
22

---

Figure 18: Low hpp: CDC, PSNR=24.82, bpp=0.41.
Figure 19: Low bpp: JPEG2000, PSNR=24.41, bpp=0.38.
23

---

Figure 20: Ground truth.
Figure 21: High bpp: COSMIC, PSNR=30.73, 73, bpp=0.67.
24

---

Figure 22: High hpp: HIFIC, PSNR-29.77, bpp=0.65.
Figure 23: High hpp: COLIC, PSNR=29.82 bpp=0.70.
25

---

Figure 24: High hpp: CDC, PSNR=30.21, bpp=0.75.
Figure 25: High hpp: JPEG20000 PSNR=28.95 bpp=0.69.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have polished our abstract and introduction to accurately reflect our main contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
27

---

Justification:
The paper does not propose a theory and does not have any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experiments are described details, including the dataset we use, all the hyperparameters and the model training details.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification: The source code will be released upon acceptance with detailed instructions including data access and preparation, the exact command and the environment needed.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https Cot//ooodeeesiseeeeeeeeee wwwpobmmsssioooooooooooooooi for more details.
/nipss.cc/pulli///uuu
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All the detailed information are provided in experiment setup.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The random factor that affects our results the sampling of Gaussian noise initialized for diffusion. We stochastically sample different Gaussian noise for the decom- pression process and show quantitative error metrics in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We show the computer resources in experimental details (10x Nvidia GeForce RTX 3090 GPUs) in Section 5.1 and FLOPs of different methods in experiment results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
All the researches in this paper conform the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our goal is as simple as to improve existing compression methods of trans- mitting images from satellites to grounds. This is not a security, safety, or privacy related research direction and is not related to any potential harmful or malicious usage in future.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
30

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets we used are properly cited.
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
31

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets. The codes and model checkpoints will be released upon acceptance.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
32


Reviews:
[{"review_id": "E6ParifzM7", "reviewer": "Reviewer_gvA5", "summary": "This paper presents COSMIC, a coding scheme designed for satellite-to-ground image transmission. It addresses the disparity in computing performance between the satellite and ground station. COSMIC features a lightweight encoder on the satellite, reducing FLOPs by 2.6 to 5 times, to achieve a high image compression ratio and save bandwidth. On the ground, a diffusion-based model compensates for image detail loss during decoding. Together, these components facilitate efficient satellite-to-ground image transmission.", "questions": "See the weaknesses.", "limitations": "See the weaknesses.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1. Unlike traditional methods that rely on arithmetic coding, this paper employs a generative model to reduce the precious information bandwidth, making for an interesting and novel approach.\n\n2. The use of a diffusion model to supplement missing details is highly feasible. Despite the availability of various encoding and decoding techniques, this choice is both wise and practical.\n\n3. Experimental results demonstrate that this method achieves better rate-distortion (RD) performance compared to other approaches.", "weaknesses": "1. The application scenario restrictions are not comprehensive. In particular, the paper overlooks an important limitation of satellites: their power capability. Generally, satellites are powered by photovoltaic panels, so power consumption must be considered when calculating hardware demands. Compared to power consumption, the influence of channel width may be less critical. Therefore, I would like to see a more in-depth discussion on power challenges.\n\n2. There are significant issues in the writing of the thesis. For example, section 4.3 states: \"We first determine what details are lost in y. In the initial training stage (Figure 2(b)), we train the image compression encoder $\\mathcal{E}$, image encoder $\\widetilde{\\mathcal{E}}$, and image decoder $\\mathcal{D}$ jointly.\" However, in this figure, the image compression encoder, image encoder, and image decoder are not correctly annotated, which seriously affects the interpretation and assessment of this paper. There are many similar instances throughout the text.\n\n3. Different satellites may carry different sensors, leading to variations in the type of metadata ($m$), which may affect the method's performance. This potential variability requires more discussion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for this thoughtful review and we are glad to see their positive assessment.\n\nNote that Fig.S2 can be found in the PDF attached to the global response.\n### Q1. More in-depth discussion on power challenges.\n\nPlease see our general response 1 above.\n\n### Q2. Issues in the writing.\n\nSorry for the confusion. The image compression encoder is deployed on the satellite, which is lightweight and used to extract satellite image features. In Fig.S2, it corresponds to the image encoder on the satellite in the Compression Model part. The image encoder corresponds to the Image Encoder module in the Compensation Model part in Fig.S2. This module is only used during training to extract the information lost in the lossy compression process to provide compensation for the decompression process, and help finetune the diffusion to learn how to generate the compensation information as a target. During the inference, this module is not used, and the compensation information is generated by diffusion. Image Decoder corresponds to the Image Decoder module in the Compression Model part of Fig.S2, which is used for the image decompression process. We make detailed annotations in Fig.S2, and will further explain and modify them in revision.\n\n### Q3. Different satellites may carry different sensors, leading to variations in the type of metadata (m), which may affect the method's performance.\n\nGood question. We demonstrate that **COSMIC has SOTA results with only three common metadata (i.e. location, timestamp, and GSD) on satellites**. Different satellites carry different sensors but there are several sensors on almost all satellites. Take LANDSAT8 launched by NASA in 2013 [1] and Sentinel-2 launched by ESA in 2015 [2] as examples, they can all collect location, timestamp, and GSD. If we use these three metadata and take a bpp of 0.46 to experiment with COSMIS, PSNR and MS-SSIM decrease slightly, respectively from 27.31 to 27.20 and from 0.969 to 0.968， which can still guarantee the SOTA results. We believe that more metadata can achieve better results, and COSMIC can still achieve SOTA performance even with commonly used metadata. We’ll add them in revision.\n\n### Reference\n\n[1] Landsat 8 (L8) Data Users Handbook Version 5.0 [https://www.usgs.gov/landsat-missions/landsat-8-data-users-handbook](https://www.usgs.gov/landsat-missions/landsat-8-data-users-handbook)\n\n[2]Sentinel-2 User Handbook [sentinel.esa.int/documents/247904/685211/Sentinel-2_User_Handbook](https://sentinel.esa.int/documents/247904/685211/Sentinel-2_User_Handbook)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ey4aMPhdLU", "reviewer": "Reviewer_GEfY", "summary": "The authors propose a novel method to compress satellite images using a learned algorithm that relies on a diffusion model on the ground to decode the compressed image. The proposed method is designed for deployment on satellites.", "questions": "* Please rewrite the text and the figures to clarify the distinction between training and testing stages. \n* Update the literature review with models used for compression on-board satellites. (suggested search terms: on-board satellite compression, in-orbit satellite compression)\n*Does the proposed compression and decompression work with the complete large image? Or just the patches? When do the reconstructed patches get stitched? See the last comment on the \"Weaknesses\" section.", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "A novel method to compress satellite images by using a diffusion model to reconstruct the encoded image is presented. The diffusion and decoder models are used to compensate for the lightweight encoder used. The method is original and may have the potential to be extended to other edge devices.", "weaknesses": "* Authors claim in the literature reviews  that no algorithm for compressing data on satellite exist. A simple search will show that this is not true. Below are two examples\nArtificial Intelligence Based On-Board Image Compression for the Φ-Sat-2 Mission\nA Simple Lossless Algorithm for On-Board Satellite Hyperspectral Data Compression\n* The difference between training and testing networks is unclear in the text or the figure. \n* No comparisons with efficient compression algorithms used on satellites or edge devices \n* Authors claim that the advantages of the proposed method are particularly visible on image seams (page 7); elsewhere in the paper, it is mentioned that the proposed method deals with image patches. How can the proposed method show advantages on image seams while it is working on the individual patches as input and not the stitched image?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the very detailed review and suggestions.\nFig.S2 can be found in global response PDF.\n\n### Q1. Please rewrite the text and the figures to clarify the distinction between training and testing stages.\n\nSorry for the confusion in writing. The training is divided into two stages.\n\nIn the first stage, we train the compression model. Since the Image decoder $\\mathcal{D}$ needs two parts of information for decoding, they correspond to $y\\prime$ in Fig.S2, which represents the feature map extracted by the on-board image compression encoder, and $z_0$, which represents the compensation information. Therefore, we introduce another Image encoder, corresponding to the Image encoder in the Compensation Model part of Fig.S2, to extract compensation information $z_0$ from the original image. In the first stage, $\\mathcal{E}$, $\\tilde{\\mathcal{E}}$ and $\\mathcal{D}$ are trained together.\n\nIn the second stage of training, we freeze the parameters of $\\mathcal{E}$, $\\tilde{\\mathcal{E}}$ and $\\mathcal{D}$, and train the noise prediction network, with the goal of making the information generated by the diffusion model as close to $z_0$ as possible, denoted as $z_0\\prime$, so as to generate the compensation information required by the decoder.\n\nIn the testing stage, the trained diffusion model can generate compensation information $z_0\\prime$. Therefore, we no longer need $\\tilde{\\mathcal{E}}$. The $z_0\\prime$ generated by the diffusion model replaces the $z_0$ extracted by $\\tilde{\\mathcal{E}}$ to help the image decoder decompress the image.\n\nWe will improve the text and figures to guarantee a better reading experience.\n\n### Q2. Update the literature review with models used for compression on-board satellites.\n\nWe will add the following content to the *Background section*.\n\nThere are many methods for remote sensing image compression, but most of them don't focus on onboard deployment. There are also some works for compressing data on satellites. [1] used the CAE model to extract image features and reduce the image dimension to achieve compression. However, this method only considers the reduction of image dimension and does not consider the arithmetic coding process in the actual transmission process, resulting in the image compression rate being fixed at 8 and the bpp not being able to be flexibly adjusted. [2] proposed the complexity-reduced VAE, which reduces the amount of calculation by reducing the number of model channels and the entropy model structure. However, violently reducing the number of model channels will lead to a significant decline in model performance.\n\n### Q3. No comparisons with efficient compression algorithms used on satellites or edge devices.\n\nOn terrain, there have been some efficient compression algorithms for edge devices. However, on terrain, edge devices are usually used to decompress images at the receiving end [3][4][5], for example, receiving a picture on a smartphone. Therefore, existing efficient compression algorithms for ground-edge devices usually focus on decoder efficiency, which is not applicable to satellite scenarios. Few works focus on encoder efficiency on satellites for image compression. We choose the European Space Agency (ESA)'s paper [1] as a baseline and list comparison as follows and will add them in revision.\n[1] conducts experiments on three platforms, respectively NVIDIA GeForce GTX 1650, Intel Myriad 2 VPU, and Intel CoreTM i7-6700 Processor. However, the power consumption of GPU and CPU shown in the article is 83W and 45.5W respectively (the results are shown in Table V in [1]), which is unpractical for power-constrained satellites and no satellites deploy GeForce GTX 1650 or Intel i7-6700 as payloads. For VPU, according to Table V, taking 10.92 seconds to process 2048 $256\\times256$ patches, we can infer that the throughput onboard of [1] is 98.36Mbps, while COSMIC can achieve 507.37Mbps within satellite supported power range [6].\n\n### Q4. Authors claim that the advantages of the proposed method are particularly visible on image seams (page 7); elsewhere in the paper, it is mentioned that the proposed method deals with image patches. How can the proposed method show advantages on image seams while it is working on the individual patches as input and not the stitched image?\n\nEarth-observation satellites take large photos, for example, the swath width of WorldView-3 is 13.1km with 1m GSD [7] which will lead to several GBs raw data per photo. The constrained computing resources on the satellites cannot support to compress an entire photo. Therefore, satellite photos are typically compressed by tiling to small images and compression onboard. After receiving on the ground, they are decompressed and stitched together. COSMIC and all baselines all follow this process and do not specially process seams. Generally speaking, without any special processing on the seams, the higher the decompression image fidelity, the less noticeable the seams will be. Moreover, stitching satellite image algorithms is orthogonal to compressing each tile, and we’ll discuss them in revision.\n\n### Reference\n\n[1] Artificial intelligence based on-board image compression for the Φ-Sat-2 mission. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.\n\n[2] Reduced-complexity end-to-end variational autoencoder for on board satellite image compression. Remote Sensing, 2021.\n\n[3] Computationally efficient neural image compression. arXiv:1912.08771, 2019.\n\n[4] Computationally-efficient neural image compression with shallow decoders. ICCV 2023.\n\n[5] Complexity-guided slimmable decoder for efficient deep video compression. CVPR 2023.\n\n[6] Wildfires From Space. [https://blogs.nvidia.com/blog/ororatech-wildfires-from-space/](https://blogs.nvidia.com/blog/ororatech-wildfires-from-space/), 2021.\n\n[7]WorldView-3 [DG_WorldView3_DS_2014.pdf (spaceimagingme.com)](https://www.spaceimagingme.com/downloads/sensors/datasheets/DG_WorldView3_DS_2014.pdf)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TTT6NxhxXu", "reviewer": "Reviewer_upSj", "summary": "This paper presents a novel method to address the challenge of transmitting the increasing volume of satellite images to ground stations. The core innovation lies in designing a lightweight encoder that reduces computational complexity on satellites, coupled with a diffusion-based compensation model on the ground to enhance image quality. The experimental results demonstrate COSMIC's superior performance over existing methods in terms of both perceptual and distortion metrics.", "questions": "1) Encoder Degradation at Low Bit Rates:\na) Question: How does the encoder's degradation specifically affect the image quality at low bit rates? Are there specific types of image details that are consistently lost?\nb) Suggestion: Provide a detailed analysis of the types of image features that are most affected by the lightweight encoder at low bit rates. This can include examples or case studies highlighting these issues.\n2) Training Specificity and Pre-Trained Models:\na) Question: How does the performance of the pre-trained stable diffusion model compare with a model specifically trained on satellite images? Have any preliminary experiments been conducted in this regard?\nb) Suggestion: Discuss any preliminary results or plans for training a diffusion model specifically on satellite images. This could include potential improvements or challenges identified during these experiments.", "limitations": "1) The authors have acknowledged the following limitations:\na) Encoder Degradation at Low Bit Rates: The lightweight encoder's performance drops at very low bit rates, affecting image quality.\nb) Training Specificity: The use of a pre-trained stable diffusion model, which lacks specific prior knowledge of satellite images, may limit performance under certain conditions.", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "1) Lightweight Encoder: The design of a lightweight encoder significantly reduces the computational load on satellites, making the solution feasible for in-orbit deployment.\n2) Diffusion-Based Compensation: Utilizing a diffusion-based model to enhance image details during the decompression process effectively addresses the limitations of the lightweight encoder.\n3) Comprehensive Evaluation: The extensive experiments and comparisons with state-of-the-art baselines highlight the robustness and superiority of COSMIC in various metrics.\n4) Multi-Modal Integration: Incorporating sensor data as conditions for diffusion generation leverages the multi-modal nature of satellite images, enhancing the overall reconstruction quality.", "weaknesses": "1) Encoder Degradation: The lightweight encoder's reduced feature extraction capability may limit image quality at extremely low bit rates.\n2) Training Specificity: The reliance on a pre-trained stable diffusion model that lacks specific priors for satellite images could limit the model's performance under certain conditions.\n3) Limited Power Supply Considerations: While the lightweight encoder addresses computational constraints, the paper does not thoroughly discuss the power supply limitations on satellites and their impact on the proposed method.\n4) Real-World Application Scenarios: The paper could benefit from more detailed discussions on practical deployment scenarios and the associated challenges, such as real-time processing requirements and potential bottlenecks.\n5) Satellite images typically consider using pixel-level distortion metrics instead of perceptual metrics. Due to the sensitivity of satellite images to compression method distortion, lossless or near-lossless compression methods are usually used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Q1. Encoder Degradation and Training Specificity.\n\nThese two questions are just **repeating** our last section, i.e. **Limitations & Future work**. Moreover, **we are frustrated to find two detectors (GPT-zero and Scribbr) have 100% confidence of AI-generation on your review**. It's clearly doubtful if the review can really help this paper's decision process.\n\nAnyway, let us to explain these two questions again.\n\n1. We never claim COSMIC is perfect. In experiments, we notice the performance of COSMIC is slightly degraded at extremely low bit rate and we report this limitation honestly. The reason is simple since the less image content the image encoder provides, the more compensation from diffusion is needed at decompression. It is hard to rely only on diffusion to decompress.\n2. We believe that training a diffusion model specifically for satellite images, which has sufficient prior knowledge of satellite images, can solve this problem to some extent. However, this is not the scope of this paper.\n\n### Q2. Limited Power Supply Considerations & Real-World Application Scenarios.\n\nPlease see our general response 1 above.\n\n### Q3. Satellite images typically consider using pixel-level distortion metrics instead of perceptual metrics.\n\nIn the paper, we considered both distortion metrics and perceptual metrics, and experimental results show that COSMIC can achieve SOTA performance in both metrics.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wj6ykJc4S8", "reviewer": "Reviewer_AGki", "summary": "The authors propose COSMIC, a simplified and efficient compression method for satellite earth observation images. Due to the increasing number of satellites and the volume of image data, existing compression schemes are difficult to deploy with the limited computing power and energy available on satellites. COSMIC designs a lightweight encoder to significantly reduce computation while achieving a high compression ratio. For ground-based decoding, it uses a diffusion-based model to compensate for the detail loss caused by the simplified encoder, leveraging the multi-modal nature of satellite data (such as coordinates and timestamps) to improve image reconstruction quality. Experimental results show that COSMIC outperforms existing methods in both perceptual quality and distortion metrics.", "questions": "See Weaknesses", "limitations": "See Weaknesses", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "(1)The writing and presentation of the paper are excellent, with a clear and logical flow.\n(2)The authors introduce a substantial amount of knowledge on deep learning-based image compression and diffusion models, making it easy for readers who are not in this field to understand.\n(3)Satellite image compression is a novel topic, and the lightweight coding framework used by the authors is of practical significance.", "weaknesses": "(1)More performance comparison tests of various models need to be included, such as those introduced in Section 2.2 on deep learning-based remote sensing image compression [1-3] and some of the latest works [4,5] in deep learning-based image compression.\n(2)The process of handling metadata is not clearly explained. I only saw the Metadata Encoder mentioned in the paper. How is this part of the data processed, and how is it aligned with the satellite image data?\n(3)The authors need to further report the spatiotemporal complexity of COSMIC compared to other methods to demonstrate its excellent lightweight architecture.\n\n[1] Fu, Chuan, and Bo Du. \"Remote sensing image compression based on the multiple prior information.\" Remote Sensing 15.8 (2023): 2211.\n[2] Xiang, Shao, and Qiaokang Liang. \"Remote sensing image compression based on high-frequency and low-frequency components.\" IEEE Transactions on Geoscience and Remote Sensing (2024).\n[3] Zhang, Lei, et al. \"Global Priors with Anchored-stripe Attention and MultiScale Convolution for Remote Sensing Images Compression.\" IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (2023).\n[4] He, Dailan, et al. \"Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n[5] Liu, Jinming, Heming Sun, and Jiro Katto. \"Learned image compression with mixed transformer-cnn architectures.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review and valuable feedback.\n### Q1. More performance comparison tests of various models need to be included.\n\nPlease see our general response 2 above.\n\n### Q2. The process of handling metadata is not clearly explained.\n\nSorry for the confusion in writing. First, we normalize each metadata. And then, inspired by the way that the timestep $t$ processed by diffusion model, we use sinusoidal embedding to encode each metadata. For each metadata embedding, we use its corresponding linear function to map it to CLIP space. Finally, we use CLIP to align it with the satellite image.\n\n### Q3. Spatiotemporal complexity comparison.\n\nIn this paper, we use two lightweight modules, LCB and CAM, while other methods use ordinary convolution. We list their time and space complexity as follows. Assuming that the number of input channels is $c_{in}$, the number of output channels is $c_{out}$, the input feature map size is $n^2$ and the convolution kernel size is $k$.\n\n| Modules | LCB | CAM | ordinary convolution |\n| --- | --- | --- | --- |\n| Temporal complexity | $O(n^2\\times c_{in} \\times k^2 + \\frac{1}{2} \\times n^2 \\times c_{in} \\times c_{out})$ | $O(k\\times n^2 \\times c_{in} \\times c_{out})$ | $O(n^2 \\times k^2 \\times c_{in} \\times c_{out})$ |\n| Spatio complexity | $O(k^2 \\times c_{in} + \\frac{1}{2} \\times c_{in} \\times c+{out})$ | $O(k\\times c_{in} \\times c_{out})$ | $O(k^2 \\times c_{in} \\times c_{out})$ |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their insightful comments and acknowledging that the paper is well-written and with clear logical flow (AGki),  the method is interesting and novel (gvA5/GEfY/AGki), highly feasible (gvA5/upSj/AGki) and potential to be extended to other edge devices (GEfY), and the evaluation is comprehensive (gvA5/upSj). We have carefully considered your comments and will take them into account to further improve our work. Before we respond to each reviewer individually, we address common concerns as follows.

### Q1. Power consumption in real-world application
Thanks to Reviewer AGki for useful suggestions. We plan to add the following contents in revision. **COSMIC can be deployed on a real satellite** by deploying COSMIC's encoder $\mathcal{E}$ on an embedded GPU (i.e. Nvidia Jetson Xavier NX) which was already deployed on various satellites in-orbit (FOREST-1, 2 [1], Chaohu-1 [2], Optimus [3], etc.). After training, we convert and deploy the image compression encoder $\mathcal{E}$ compatible with the NX system via TensorRT 8.2.1 SDK (Jetpack 4.6.1, CUDA 10.2, cuDNN 8.2.1). We use the tegrastats tool to monitor NX's power consumption. During the compression, the power consumed by $\mathcal{E}$ is between 5.7W and 7.7W, which can be fully supported by satellites like FOREST-1, 2 since their payloads can support NX running at 15W [1].

### Q2. More baselines for comparison
Different from natural images, remote sensing images are actually multimodal data, which contain many sensor data in addition to images (e.g. timestamps and location). Our insight is that **multimodal information in sensor data is instructions of image contents** to a certain extent. For example, a satellite's location can roughly determine whether its photo content is a city, desert, or ocean. Thus, COSMIS is novel since we use the multimodal data nature of sensing images for better decompression which cannot be generally extended by natural images without such multimodal information as instructions. Moreover, various existing methods rely on complex encoders so they are hard to meet the computing and power constraints of satellites.

According to the suggestions of reviewers GEfY and AGki, we add the representative work HL-RS [5] in remote sensing image compression and a latest work Elic [6] as baselines. Please note that LIC_TCM [7] has FLOPs of 51.19G, which is 10 times larger than COSMIC, so we do not experiment it as a baseline for comparison. The results of the encoder efficiency analysis (FLOPs) are shown as follows (* represents the newly added baseline).
| Method | Elic*[6] | HL-RS*[5] | CDC | COLIC | Hific | mbt-2018 | cheng-2020 | COSMIC |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| FLOPs(G) | 21.78 | 11.87 | 13.1 | 26.4 | 26.4 | 8.07 | 24.45 | 4.9 |

We also show the rate-distortion (perception) results across 12 metrics in Fig.S1 in the PDF attached to the global response. In 8 of them, COSMIC achieves SOTA results at all bpp. In the other 3 metrics, COSMIC achieves SOTA results at certain bpp.

### Reference

[1] Wildfires From Space. [https://blogs.nvidia.com/blog/ororatech-wildfires-from-space/](https://blogs.nvidia.com/blog/ororatech-wildfires-from-space/), 2021.

[2] "Chaohu 1". Gunter's Space Page. Retrieved July 12, 2024, from [https://space.skyrocket.de/doc_sdat/chaohu-1.htm](https://space.skyrocket.de/doc_sdat/chaohu-1.htm).

[3]Space Machines Company, Optimus OTV, [https://space.skyrocket.de/doc_sdat/optimus-otv.htm](https://space.skyrocket.de/doc_sdat/optimus-otv.htm), 2024.

[4] Jetson Modules [https://developer.nvidia.com/embedded/jetson-modules](https://developer.nvidia.com/embedded/jetson-modules)

[5] Remote sensing image compression based on high-frequency and low-frequency components. IEEE Transactions on Geoscience and Remote Sensing, 2024.

[6] Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding CVPR 2022.

[7]Learned image compression with mixed transformer-cnn architectures. CVPR 2023.

Meta-review: This paper received 3 positive and 1 negative scores. Most reviewers believe that the rebuttal has addressed their concerns well. The negative review is not grounded on facts, and with a good chance was generated by GenAI - therefore we ignore it. After reading the rebuttal and discussions, I believe that the authors can resolve these concerns well in the revision. Of course, the authors are encouraged to refine the work based on the comments.
Decision: Accept (poster)