Controllable Blur Data Augmentation Using 3D-Aware Motion Estimation
Authors: Insoo Kim
Hana Lee
Hyong-Euk Lee
Jinwoo Shin
Venue: ICLR 2025 Poster (2025)
Keywords: Blur synthesis
Data augmentation
Blind motion deblurring
3D motion modeling
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose a controllable 3D-aware blur synthesizer to generate diverse blur images, improving deblurring performance.

Abstract: Existing realistic blur datasets provide insufficient variety in scenes and blur patterns to be trained, while expanding data diversity demands considerable time and effort due to complex dual-camera systems. To address the challenge, data augmentation can be an effective way to artificially increase data diversity. However, existing methods on this line are typically designed to estimate motions from a 2D perspective, e.g., estimating 2D non-uniform kernels disregarding 3D aspects of blur modeling, which leads to unrealistic motion patterns due to the fact that camera and object motions inherently arise in 3D space. In this paper, we propose a 3D-aware blur synthesizer capable of generating diverse and realistic blur images for blur data augmentation. Specifically, we estimate 3D camera positions within the motion blur interval, generate the corresponding scene images, and aggregate them to synthesize a realistic blur image. Since the 3D camera positions projected onto the 2D image plane inherently lie in 2D space, we can represent the 3D transformation as a combination of 2D transformation and projected 3D residual component. This allows for 3D transformation without requiring explicit depth measurements, as the 3D residual component is directly estimated via a neural network. Furthermore, our blur synthesizer allows for controllable blur data augmentation by modifying blur magnitude, direction, and scenes, resulting in diverse blur images. As a result, our method significantly improves deblurring performance, making it more practical for real-world scenarios.

Published as a conference paper at ICLR 2025
CONTROLLABLE BLUR DATA AUGMENTATION USING 3D- AWARE MOTION ESTIMATION
Insoo Kim Hana Lee' Hyong-Eute Lee' Jinwoo Shin
SAIT, Samsung Electronics "KAIST
ABSTRACT
Existing realistic blur datasets provide insufficient variety in scenes and blur pat- terns to be trained, while expanding data diversity demands considerable time and effort due to complex dual-camera systems. To address the challenge, data aug- mentation can be an effective way to artificially increase data diversity. However, existing methods on this line are typically designed estimate motions from a 2D perspective, e.g., estimating 2D non-uniform kernels disregarding 3D aspects blur modeling, which leads to unrealistic motion patterns due to the fact that camera and object motions inherently arise in 3D space. In this paper, pro- pose a 3D-aware blur synthesizer capable of generating diverse and realistic blur images for blur data augmentation. Specifically, within the motion blur interval, generate the corresponding scene images, and ag- gregate them to synthesize realistic blur image. Since the 3D camera positions projected onto the 2D image plane inherently lie in 2D space, we can represent the 3D transformation as combination of 2D transformation and projected 3D residual component. This allows for 3D transformation without requiring explicit depth measurements, as the 3D residual component is directly estimated via a neural network. Furthermore, our blur synthesizer allows for controllable blur data augmentation by modifying blur magnitude, direction, and scenes, resulting in diverse blur images. As a result, our method significantly improves deblurring performance, making it more practical for real-world scenarios.
estimate 3D camera positions
INTRODUCTION
Motion blur is a common challenge in photography, where it typically caused by camera or object movement during long exposure times. Given blur image, blind motion deblurring tackles the challenge of producing a sharp image. In recent years, a breakthrough has been made by adopting diverse neural architecture designs (Nah et al., 2017; Tao al., 2018; Kupyn al., 2019; Cho al., 2021; Zamir et al., 2021; Tu et al., 2022; Li et al., 2022; Chen et al., 2022; Nah et al., 2022; Zamir al., 2022; Wang al., 2022: Tsai al., 2022: et al., 2023b; Kong et al., 2023; Fang al., 2023; Kim et al., 2024) and utilizing the realistic blur datasets (Rim et al., 2020; 2022; Li et al., 2023a) in blind motion deblurring tasks, enabling the practical usability in woal-world scenarios.
To build more generalizable deblurring model, a realistic large-scale blur dataset containing diverse scenes and blur patterns required. Nevertheless, existing realistic blur datasets suffer from lim- ited number of scenes and blur patterns because their dual camera system is heavy and complicated to collect a large-scale blur dataset (Rim et al., 2020; 2022; Li et al., 2023a). Data augmentation an alternative to artificially increase the amount of data via a synthesis procedure. For example, one can synthesize the blur image using blur synthesizers and train a deblurring model using both real and synthetic blur images. Despite its significant successes in high-level vision tasks (De Vries & Taylor, 2017; Zhang et al., 2018; Yun et al., 2019; Cubuk et al., 2019; 2020; Hendrycks et al., 2020; Kim al., 2021; Yang et al., 2021), there has been little study particularly focused blur data augmentation (Zhang et al., 2020; Carbajal et al., 2023; Wu et al., 2024; Lee et al., 2024).
In fact, the previous literature on blur data augmentation exhibits certain limitations that merit further investigation: (1) diffusion- based data augmentation (Wu et al., 2024) requires ground-truth vidco
"Corresponding author

---

Published as a conference paper at ICLR 2025
Target Images
3D-Aware Motion Estimation
Scone
Amp, 0.5x Amp, 2.0x Phase 455" Phase +45
Saone
Contrallable
Synthesis
Image
3D-fware Vector Fielda
Synthesized Images
Figure I: Our controllable blur image synthesis. We can manipulate the motion amount (amplitude), direction (phase) of the 3D-aware vector fields, and target scene images to generate variety of blur images. They are used for data augmentation in training the deblurring model.
frame images which are not typically provided in blur datasets. Hence, its applicability to other blur datasets for blur data augmentation remains challenging. and (2) the kernel- method (Carbajal al., 2023) offers estimated blur kernels. However, simply regressing the simulated kernels without physically-driven blur modeling leads to unrealistic synthetic blur images which are not acceptable in real-world scenarios (Gong et al., 2017; Tran et al., 2021). Above all, the existing methods this line neglect to pay attention to explicit 3D geometry for blur synthesis despite the fact that the camera and object motion primarily occurs in 3D space (Whyte et al., 2010). This may lead to inaccurate or unrealistic blur patterns, constraining their practical usability for blur data augmentation.
In this paper, we propose controllable and 3D-aware blur image synthesizer to generate various realistic blur images as shown in Fig. I, leading to better deblurring performance. Our intuition is that intricate 2D non-uniform motion kernels can be parameterized by simple 3D camera position trajectory using rotation and translations. Such parametric technique helps reduce ill-posedness of the problem while it is closely grounded in realistic blur modeling. In the case of camera motion, we estimate 3D camera positions within the exposure time, use them for generating the transformed scene images with grid sampling (Jaderberg et al., 2015), and aggregate them to produce a synthe- sized blur image as shown in Fig. 2. We are motivated by the fact that the 3D camera positions projected onto the 2D image plane inherently lie in 2D space. Hence, they can be modeled by a combination of 2D rigid transformations and projected 3D residual components as shown in Fig. 2: the former is represented by parametric vector fields, and the latter is represented by non-parametric vector fields. This decomposition allows for 3D rigid transformation without depth measurements, as the 3D residual component is directly estimated via neural network. Furthermore. our paramet- ric and con-parametric motion modeling takes advantage of reducing ill ill-posedsess (by parametric) and effectively capturing 3D residual components (by non-parametric).
In the case of object motion, the 3D residual components also serve as capturing object motion be- haviors such as non-uniform and non-rigid deformation due to the flexibility and versatility of the non-parametric vector field. Therefore, the 3D residual components represent not only 3D camera residual components but also object residual components. Furthermore, while the non-parametric vector field is essential for capturing 3D camera motion as shown in Fig. 3 and object motion as shown in Fig. 4, its flexible nature can lead to undesirable results and ambiguity as shown in Fig. 6 (a). To overcome this, we introduce Laplacian and invertible geometric regularizations which en- ables produce more natural motions. Specifically, the 3D transformed scene image can be traced back to the original image using our invertible geometric regularization. As result, the non- parametric vector field remains geometrically structured. Furthermore, a key aspect of our method is to represent motion as a vector field, which provides an inherently flexible framework where the magnitude and direction of each vector can be easily adjusted. This controllability allows for the generation of millions of diverse blur patterns by randomly varying the blur intensities, directions and scenes during the deblurring training, enabling the deblurring model to handle a wide range of unseen blur scenarios. We summarize our contributions as follows:
We propose controllable 3D-aware blur synthesizer that combines 2D motion and 3D residual components to represent 3D motion. This enables 3D transformation without depth measurements and facilitates accurate 3D motion modeling for blur data augmentation.
Our blur synthesizer enables controllable blur data augmentation by modifying blur magni- tude, direction, and scenes. It does not require any extra data such as ground-truth kernels, depths or frame images during the training, which ensures compatibility to any blur dataset.
We demonstrate that our blur data augmentation scheme contributes to superior deblurring performance across various network architectures and blur datasets.
2

---

Published as a conference paper at ICLR 2025
Amplitucee-Phss Indegration
Transformed Scane Images
Sharp Image
Vecdor
kB(BB)
Coordinates
f
MLP
|||
R1
Rigid Transfomation
Sampling
Halll Compensation Network
5D Residual Fiald (Non- Parametric Vector Field)
3D AD-Awarr Vector Fields
Eistmated Blur Image
Comperatted Image
Estimation
3D-Aware Motion Estimation
Image Bynthesis
Figure 2: Training procedure for our blur synthesis model. We first predict 3D-aware vector fields (parametric + non- parammetree) from given blur image, which are used to generate corresponding scene images. These images are then aggregated construct realistic blur image. Note that the motion estimator, MLP, and compensation network are jointly trained.
2 RELATED WORKS
Data augmentation. Data augmentation has been widely explored the field of image classifica- tion, where it is proven to be effective for improving the model generalization VDeVsies & Taylor, 2017: Yun et al., 2019; Cubuk et al., 2019; Lim et al., 2019; Cubuk et al., 2020; Hendrycks al., 2020; Kim et al., 2021; Yang et al., 2021). On the other hand, there has been little study particularly focused on blur data augmentation (Zhang et al., 2020; Rim et al., 2022; Wu et al., 2024). Blur Pipeline (Rim et al., 2022) presents method to generate synthetic blur images based on GoProU dataset using saturation and noise synthesis (not data-agnostic and not controllable). Diffusion- based data augmentation (Wu al., 2024) is scheme to generate synthetic blur dataset and use for pre-training and fine-tuning the deblurring model (not cost-effective). Also, it requires the video frame images for extracting the optical flow (not data-agnostic), where the optical flow is used for the condition of the diffusion model. Also, these fail to explicitly account for the 3D geometry. despite the fact that the motion inherently occurs in 3D space (not accurate). On the other hand, our method considers the explicit 3D geometry, such that it achieves realistic blur synthesis (accurate). Furthermore, our method can be controllable, applied to any blur dataset (data-agnostic), and used during the training of the deblurring model (cost- (cost-effective).
Deblurring methods. The kernel-based methods estimate blur kernels motion information to construct latent sharp images (Gong et al., 2017; Zhang et al., 2021: Carbajal et al., 2023). We may utilize these byproducts, i.e., blur kernels, synthesize blur images. However, simply regressing the simulated kernels without physically-driven blur modeling may not hold in generating realistic blur images. In recent years, as realistic blur-sharp pair datasets (Rim et al., 2020; 2022; Li et al., 2023a) have been released, they have enabled direct prediction of latent sharp images without explicit kernel estimation. A variety of neural architecture designs (Kupyn et al., 2018; Cui et al., 2024; Mao et al., 2023; Tu et al., 2022; Cho et al., 2021; Zamir al., 2021; Chen et al., 2022; Zamir et al., 2022; Kong et al., 2023) has emerged to improve the deblurring performance. Also, some works introduce addi- tional self-generated priors further improve deblurring performance (Li al., 2022; Fang al., 2023; Kim et al., 2024). They estimate the degradation representation (l.i et al., 2022), non-uniform kernel (Fang et al., 2023), and blur segmentation map (Kim et al., 2024) as prior information and exploit them for improving the deblurring performance. These methods require additional compu- tational costs to generate their own priors. On the other hand, our method used for blur data augmentation during the training, such that it does not require additional inference cost.
3 CONTROLLABLE 3D-AWARE BLUR SYNTHESIS
3.1 2D BLUR SYNTHESIS MODEL
In this section, present new blur synthesis framework that considers 3D perspective on blur modeling, rather than simply regressing non-uniform motion kemnels from a 2D perspective. Simple regression- based motion estimation may not be generalizable and controllable. Our primary goal to estimate camera positions in 3D space within the exposure time, generate the corresponding scene images, and aggregate them to achieve realistic blur image synthesis. To this end,
3

---

Published as a conference paper at ICLR 2025
(a)
(b)
(c)
(d)
(c)
(f)
Figure 3: Blur synthesis results on 3D motion blur such as in-plane rotation blur, e.g., z-axis rotation: (a) Blur image, (b) Blur synthesis result, and (c) Estimated blur trajectory, and forward motion blur, e.g., z-axis translation: (d) Blur image, (e) Blur synthesis result, and (f) Estimated blur trajectory.
consider a dataset D = ()), which contains a pair of blur image B and sharp image S. Let T, = (T, ) be vector field, i.e., set of transformed coordinates, where u, TR2 denotes 2D image coordinate at pixel index i, (u,) E I2² indicates i th transformed pixel coordinate at time т. and N is the number of pixels. Note that initially consider 2D motion and extend it to 3D motion subsequently. The blur image is expressed by using the camera exposure model over the exposure time T:
B [ S(T,) dr,
where S(T,) is scene image at a time т. As scenes may change due to object movements and cam- era shake over the exposure time, we generate the corresponding scene images (T)) ... (TMM) using transformed vector field (Ti, ... ,TT)) with grid sampling (Jaderberg et al., 2015). Note that the exposure time is approximately divided into M discrete camera positions for implementations. Here, we begin by considering a certain case, i.e., camera motion, which is easily encoded by rigid transformation. The 2D rigid transformation is simply parameterized by [R-|ty]. e.g., rotation and translation. The transformed coordinate (u) is denoted by
rf
(22)
(1)
x
rF + +r y (1)) (11)
(11)
(12)
T (u) [R,
(2)
(21)
(22)
r x + + (
Since the unknown parameters are now [R|t] in (1), one can view the blur synthesis problem as a
camera pose estimation by neural network, fe B + [71.-.., YM, t., tt] where "r g is the axis-angle representation (Murray et al., 2017) for a rotation matrix and t, E R² indicates a translation component. Once we estimate 7, we can generate 3D rotation matrix using Rodrigues' formula (Murray et al., 2017), whose 2D components are used to construct 2D transformation matrix R., Given R, and f,, we can make a 2D rigid transformation field T, using (2) and optimize the following loss train the blur synthesis model, i.e., Taaur = || 1 Σ, S(T=)lli
M
3.2 3D 3D-AWARE BLUR SYNTHESIS MODEL
camera motion blur synthesis. We further develop our blur synthesis model, elaborating on its extension to 3D space. In fact, the camera motion appears in 3D space, but it is primarily modeled from a 2D perspective. ie., 2D non-uniform kernels (Carbajal et al., 2023; Fang et al., 2023), which insufficient ensure the geometric coherence of the camera motions. Therefore, we begin by investigating the 3D rigid transformation below
r
(11)
r
(12)
r-
(13)
I
r
(11) x + ( (12)
+ (13) + (1)
XX(X = X
(21)
(22)
(23)
V
=
(21) (22)
y (33)
(2)
(3)
(31)
(32)
(33)
(31) x rt (32)
y tr (())
(33)
where X is a 3D canonical coordinate. We discover that the 3D rigid transformation vector T, (X) E
R can be decomposed into 2D rigid transformation vector T (u) = [T, E R& and 3D residual vector (X) E R3, written by
FT x + r- y + ! (21)
(11)
(13)
T, (X)
r7 x + r7 (22)
+ (2)
(23)
(4)
0
x rr y + r +
(32)
(1)
e, (X)
4

---

Published as a conference paper at ICLR 2025
(a)
(b)
(c)
(d)
(b)
(c)
(d)
Figure 4: Visual results on (a) Blur image, (b)
Figure 5: Visual results on (a) Synthesized blur image, (b-d) transformed scene images.
Synthesized blur image, (c) 2D parametric blur trajectory, and (d) 2D parametric parametrie blur trajectory. Person movements are captured by (d), but not (c).
3D non-
Our blur synthesizer generates 3D-aware trans- formed scene images without requiring ground- truth video frame images.
Using a projection operation π R3 7 R² with camera intrinsics K, we compute the projected coordinate vector T, (u) (X) K) = x (T) (X); K) to sample 3D-aware transformed scene images over the exposure time. Here, we are motivated by the fact that (+( lies in IR? Therefore, following the rationale of the decomposition principle in (4), we express the projected coordinate vector (u) as combination of 2D rigid transformation coordinate Tz(u) E IR² and projected 3D residual vector (u) E RR² derived from & (X(), i.e.,
(u) C(T, (u), (u)),
(5)
where C serves as a composition function for integrating the two components. Since Tu(u) contains 3D residual components, it is referred to as the 3D-awar coordinate vector. Then, we simply induce the 3D-aware vector field, i.e., T, C(T,, 67) where T is the 2D rigid transformation field as discussed in Section 3.1 and 6,- denotes the 3D residual field modeled by neural network, 9p : B -> {e1, 62, ... <M }. This decomposition scheme allows for 3D rigid transformation without requiring explicit depth mcasurements", as the 3D residual field directly estimated via the neural network. To investigate the composition function C for better motion controllability, we first reformulate the 3D-aware vector field using a known canonical vector field U, i.e., 5 =SU)), which
T = C(TT, = -C e,) = U &( = U 6
(6)
T.
where 8, = C(TT, E,) a displacement field which consists of the 2D rigid transformation resid- ual field AT, (parametric field) and 3D residual field e, Inon-parametric field). Note that the canonical vector field U is the constant field, such that it is factored out from the composition func- tion. Then, adopt implitude-phasee integration in polar coordinates for the composition function, i.e., 6, = C(ATT, f ) = AT |c> (() + ((<))), )). where indicates the magnitude of the vector and Φ a function calculate the vector angle. This integration scheme provides straight- forward way to manipulate the magnitude and direction of the motion, facilitating a controllable blur synthesis, as opposed to simple vector field integration, i.e., 8, C(ATr, E,) AT, + Er. Given target scene image S and the 3D-aware vector field T,, we can render 3D-aware transformed scene images and synthesize a blur image as exemplified in Fig. 5 by
B= Σ S(T.).
Finally, we suggest minimizing the following loss to train the 3D-aware blur synthesis model:
Lotur.30 30 B B|I | (B)]i
where hg is an auxiliary network that compensates for photometric variations between blur and sharp images, arising from different image sensors, lenses, and color drifts. This compensation is required encourage the 3D-aware vector fields focus only on blur components (not disrupted by the photometric issues). This will be more discussed in Section C of Appendix. Furthermore, we emphasize that our compositional training framework leverages structured parametric modeling
provide underlying derivation and insights of projected 3D residual vector in Section A of Appendix. In our experiments, we observe that inaccurate absolute depth measurements can lead to performance degradation, discussed in detail in Section 4.4.
5

---

Published as a conference paper at ICLR 2025
to alleviate ill ill-posedness and flexible non-parametric modeling effectively capture 3D residual components, thereby achieving accurate 3D motion modeling.
3D object motion blur synthesis. As discussed carlier, estimate the 3D residual components using the non-parametric vector field E,, which is initially designed to represent the 3D camera residual fields. Additionally, the 3D residual components capable of representing object residual fields. fact, non- non-paramnn con-paamettiic motion modeling, i.e., directly estimating the vector field via a neural network, is particularly effective in capturing the object motion behaviors (i.e., non-uniform and non-rigid motion) due to its flexibility and versatility. Hence, our 3D-aware blur synthesis model (7) inherently compatible with scenarios containing object motion without further modification. As result, our method can handle 3D camera motions as shown in Fig. 3 and object motions as shown in Fig. 4, using (6). Additional visualization results can be found in Fig. 14 and 15 of Appendix.
3.3 AMBIGUITY REGULARIZATION AND TOTAL LOSS
As discussed in Section 3.2, the non-parametric vector field provides substantial flexibility, but its arbitrary and unconstrained natures can lead to ambiguities. This makes the optimization more challenging. As result, it yields implausible synthesis images and artifacts as shown in Fig. 6 (a). To address this, we present Laplacian regularization and invertible geometric regularization. These regularizations help mitigate the ambiguities and ensure the geometric coherence regarding the non-parametric vector field.
Laplacian regularization. The Laplacian regularization helps smooth irregular vector fields. This holds under the prior knowledge that the motion vector field is not irregularly changed in the spatial space. To promote the smoothness of the vector fields, we derive the Laplacian regularization with respect the 3wD-aware vector field T,:
Tmmothh = =E
E.. + + 4 (-,,-1 (9)
Invertible geometric regularization. To adapt the geometric consistency for the 3D-aware vector field T,, we propose invertible geometric regularization that allows us to trace back to the original scene image. Specifically, the 3D transformed scene image S- S(T=) should be geometrically reverted to the original one S via an inverse vector field. To this end, we recall the 3D-aware vector field (6), i.e., T, U + ô,, where 6, denotes the displacement field. The inverse vector field T is computed by using the reversed direction of the displacement field, i.c., T" = U 8, Finally, the geometric-consistent vector field is achieved by optimizing the following loss:
(a)
(b)
Figure 6: Synthesized blur im-
ages (a) without regularizations
and (b) with regularizations.
I Iggeometric
M Σ EIS- -
(10)
where (T()) the geometrically inverted scene image. This geometric consistency loss is the key component of our method ensure that the 3D-aware vector field remains geometrically structured. Therefore, ensures geometric-ave aware blur synthesis as shown in Fig. (b).
Total loss. We suggest minimizing the following loss to train our geometric-aware blur synthesizer:
Littl Lotur 3D Lymoohh + Xg Lecometiic
(11)
where d1 22 > 0 are hyper-parameters to balance the loss terms, which will discussed Sec- tion 4.5 and Section F.I of Appendix.
3.4 CONTROLLABLE BLUR IMAGE SYNTHESTS
Our blur synthesis model is designed to enable blur data augmentation during the deblurring model training, thereby improving final deblurring performance and model generalization. Specifically, if

---

Published as a conference paper at ICLR 2025
MARA ReyarA Heres Meeweasss Afhi MAR Caposse
NarA KheerKyy KA HEER was WiFes Georee
NéTE Reex V Here APsfiiii
Next they wi Here was frie what George au GeoSyn (ours)
WAR Grosse.
Blur
NAFNet-64
FFTYormer
ID-Blau
Figure 7: Qualitative comparison results on real-world blur images. NAFNet is used for all methods. Our blur synthesizer produces sharper and artifact- free results compared to wate-offthe--a methods.
a blur dataset contains 3, 000 blur images and the deblurring model trained over 1,000 epochs, it generates approximately 3,000,000 different synthetic blur images. This extensive diversity en- hances the generalization ability of the deblurring model, enabling it to handle a wide range of unseen blur scenarios. To this end, first extract 3D-aware displacement vector 5 (II6, y/5 from (6). Note that we use the notation ö for simplicity here. It can be converted into the amplitude-
phase representations in the polar coordinates, i.e., 8 = [ where | V + y/ in- dicates the amplitude of the vector, i.e., motion intensity and (65) = tan 25)) ( represents the us
phase of the vector, i.e., motion direction. Then, every displacement vector is controlled by the amplitude control parameter and phase control parameter B3, i.e., Es ag8 cos( + (3) and Vs = a 6) sin( + (3, whose parameters are explored in Section B of Appendix. We remark that this amplitude-phi phase motion adjustment preserves the geometric structure of the vector fields since the control parameters are identically applied to all vector fields generated from single blur image adjust overall blur intensity and direction. Meanwhile, the control parameters can be determined for each blur image. Finally, we employ this modified displacement motion vector, ò (#5 96) instead of the original one 5 for the subsequent scene image rendering and blur image synthesis as in (7). We present some blur synthesis results for different scenes, amplitudes, and phases as shown in Fig. 1, Fig. 16 and 17 of Appendix.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and evaluation metrics. For datasets, we use RealBlur (Rim et al., 2020). RSBlur (Rim et al., 2022), and GoPro (Nah et al., 2017) datasets for training and evaluation. RealBlur is split into two types: RealBlur-J (sRGB domain) and RealBlur-R (RAW domain). Each RealBlur type consists of 3,758 and 980 image pairs for training and test sets, respectively. RSBlur contains 8,878 and 3,360 blur-sharp image pairs for training and test sets, respectively. GoPro contains 2,103 and 1,111 image pairs for training and test sets, respectively. We use the evaluation metrics such as Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity (SSIM) (Wang et al., 2004).
Implementation details. We present the implementation details for training our blur synthesis model. Our blur synthesis model consists of motion estimator (NAFNet-8) and compensation net- work (NAFNet-166. NAFNet-16 16 means NAFNet (Chen et al., 2022) with 16 base channel widths. Note that the compensation network is only used for the photometric compensation during the train- ing, as discussed in Section C of Appendix. According to Fig. 2, the blur input image is fed into the encoder of the motion estimator. Then, the intermediate features are fed to MLP. including global average pooling, two fully connected layers, and one ReLU activation, in order to estimate 2D motion parameters ("1,"' " Yas, ti, ... t.s.. Using the parameters, we obtain parametric vector
7

---

Published as a conference paper at ICLR 2025
Table 1: Comparison results across diverse neural networks and blur datasets.
GoPro
HIDE
RealBlur-J PSNR
RealBlur-R
Model
PSNR
SSIM
PSNR
SSIM
SSIM
PSNR
SSIM
MIMO-UNet+
32.44
0.957
30.00
0.930
31.92
0.919
39.10
0.969
+ GeoSyn (ours)
33.01
0.962
30.86
0.940
32.55
0.925
39.68
0.972
Restormer
32.92
0.961
31.22
0.942
32.32
0.924
39.47
0.972
GeoSyn (ours)
33.37
0.964
31.61
0.946
33.05
0.937
40.31
0.974
NAFNet
33.69
0.966
31.32
0.943
32.50
0.928
39.89
0.973
GcoSyn (ours)
34.09
0.969
31.64
0.947
32.99
0.936
40.49
0.976
FFTformer 34.21
0.969
31.62
0.946
32.62
0.933
40.11
0.973
GeoSyn (ours)
34.39
0.970
31.98
0.949
33.68
0.938
40.89
0.977
fields (71, T,,.... Tas} }or as discussed in Section 3.1. Also, the intermediate features are fed into the decoder of the motion estimator to predict non- con-aarmeetric vector fields {E1, Cg. .wy Then, we aggregate the parametric and non parametric vector fields using the amplitude- -hhase integration and finally obtain the 3D-aware vector fields as discussed in Section 3.2. To produce diverse blur images for blur data augmentation, we modify amplitudes, phases of the vector field, and scene contents as discussed in Section 3.4. We will discuss how to synthesize and where to synthesize in detail in Section B of Appendix. Note that our blur synthesizer is only used for training. Therefore, it does not increase the computation cost the evaluation stage. We use blur datasets randomly cropped by 256 256 during the training. We train our blur synthesizer up to 1, 000 epochs for RealBlur, 3, 000 epochs for GoPro, and 500 epochs for RSBlur. Also, our blur synthesizer is optimized by the AdamW (Loshchilov & Hutter, 2019) algorithm (3) 0.9, 82 0.9 and weight decay 1e-8) with the cosine annealing schedule (1e-3 to 1e-3) (Loshchilov & Hutter, 2016) gradually reduced for total iterations of each dataset. Unless otherwise specified, we use \ 0.1, X2 1.0 and the number of camera positions as 16, which is discussed in Section 4.5 and F.I of Appendix. We use RealBlur-J for all ablation studies. The pseudo-codes can be found in Section G and H of Appendix.
4.2 MAIN RESULTS
To demonstrate the effectiveness of our blur synthesizer, we conduct experiments across various datasets such as GoPro (Nah et al., 2017). HIDE (Shen et al., 2019), RealBlur-J and R (Rim et al., 2020), and network architectures such MIMO-UNet+ (Cho et al., 2021), Restormer (Zamir et al., 2022), NAFNet (Chen et al., 2022) and FFTformer (Kong et al., 2023). We train with GoPro and evaluate GoPro and HIDE. Also, train with RealBlur-J and R, and evaluate the corresponding trained models. As shown in Table 1, the results clearly demonstrate that our blur synthesizer, i.e., GeoSyn boosts deblurring performance across only different network architectures but also various datasets. For example, FFTformer equipped with our GeoSyn significantly improves PSNR from 32.62 33.68 dB on RealBlur-J, Furthermore, we present a more comprehensive analysis of the generalization ability of our GeoSyn in Section E. Also, we present the visual comparison results on GoPro as shown in Fig. 12 of Appendix, RealBlur as shown in Fig. 13 of Appendix, and real-world blur images as shown in Fig. 7 and Fig. 11 of Appendix.
4.3 COMPARISON TO OTHER METHODS
Comparison to Kernel-based methods. J-MKPD (Carbajal et al., 2023) and MotionETR (Zhang et al., 1)21) generate motion information for deblurring purposes. However, motion information can be used for blur data augmentation. Since their motion estimators are trained with GoPro, we train our blur synthesizer with GoPro for fair comparison. Then, the deblurring model, NAFNet-64 is trained with GoPro, using data augmentation with their motion estimators and ours. As shown in the upper side of Table MTRionETR exhibits a performance improvement from 33.69 to 33.92 dB, but it lags behind our GeoSyn (34.09 dB). We believe their motion information does not account for explicit 3D geometry, causing performance limitations.
Comparison to blur synthesis methods. Blur Pipeline (Rim al., 2022) and ID-Blau (Wu et al., 2024) are the blur data augmentation methods and are trained with a certain dataset (GoPro). For example, ID-Blau requires the ground-truth video frame images extract optical flows, such that it
8

---

Published as a conference paper at ICLR 2025
Table 2: Comparison results against related works. We compare our method with kernel- based methods (above) and blur data aug-
Table 3: Comparison results RSBlur. The
best results are indicated in bold.
RSBlur
Methods
GMACss
mentation methods (below).
PSNK 1
SSIM
SRN-Deblur
1434.82
32.53
0.840
Augmenlation Types
Datasels
PSNR
SSIM
MIMO-UNets
154.41
33.37
0.856
None
33.69
0.9866
MPRNet
777.01
33.61
0.861
J-MKPD
GoPro
33.59
Restoreer
141.00
33.69
0.863
MTR
33.92
0.968
GooSyn (ours)
34,09
0,969
Uformer-B
89.50
33.98
0.866
None
32.50
0.928
ConvIR-L
71.22
34.06
0.868
Blur Pipeline
32.57
0.931
NAFNet-64
63.64
33.97
0,866
ID- ID-Blau
RealBlur-J
32.70
0.932
+ Geo8yn (ours)
63.64
34.23
0.870
GeoSyn (ours)
32,99
0.936
SegDeblur-L
62.68
34.21
0.870
ID-Biau GeoSyn (ours)
33.09
0,938
+ -GooSyn (ours)
62.68
34.31
0.872
Table 4: Ablation study on various types of vector fields. "p" indicates the paramet- ric vector field while "NP" exhibits the non- parametric vector field. The best results are indicated in bold.
Table 5: Ablation study on model sizes. Our blur augmentation enables us to build an ef- ficient deblurring model. It reduces the com-
putational cost by up to 4x.
Methods
GMACss
PSNR f
SSIM
Methods
P
NP
PSNR
SSIM
NAFNet-16
31.58
0.912
No Augmentation
32,50
0.928
4.0
+ GeoSyn (ours)
31.97
0,921
2D Parametric
32.65
0.932
3D Non-Parametre Parametric
4
32.61
0.930
NAFNet-32
16.0
31.99
0.920
GeoSyn (ours)
32.59
0,931
3D Flat Depth
32.66
0.932
3D Monocular Depth
32.47
0.929
NAFNet-64
63.5
32.30
0.928
GeoSyn (ours)
32,99
0,936
+ .GeoSyn (ours)
32,99
0,936
not be trained with other datasets containing only blur and sharp images such RealBlur (Rim et al., 2020) and RSBlur (Rim et al., 2022). As shown in the lower side of Table 2, ID-Blau, trained with GoPro, leads to limited deblurring performance improvement 32.70 dB. On the other hand, our GeoSyn can be trained with RealBlur-J, to capture dataset-specific motion patterns. Hence, our GeoSyn yields better deblurring performance (32.99 dB), thanks to our data-agnostic motion modeling. Furthermore, we remark that our GeoSyn is compatible with ID- Blau. Specifically, we train the deblurring model initialized by ID-Blau pre-trained model, with our GeoSyn. We observe that it further improves the deblurring performance (33.09 dB), achieving the best performance. More comparison results with ID-Blau can be found in Section of Appendix.
RealBlur-J, from 32.50 to
Results RSBlur. We experiment with additional dataset, RSBlur (Rim al., 2022) which contains a variety of camera and object motions with high-resolution images. We train and evaluate NAFNet-64 (Chen et al., 2022) and SegDeblur-L (Kim al., 2024) with RSBlur, using our data augmentation scheme. As shown in Table. 3, our GeoSyn improves the performance on NAFNet-64, from 33.97 to 34.23 dB. Interestingly, our GeoSyn can be compatible with the prior-bases method such as SegDeblur-L, improving the deblurring performance from 34.21 to 34.31 dB.
4.4 VARIOUS FORMS OF VECTOR FIELDS
The goal of this experiment is to identify an optimal configuration of the vector fields for blur data augmentation, by evaluating various combinations of parametric and non-parametric vector fields including 2D motion, 3D motion, and depth information, as reported in Table 4.
2D parametric vector field. The parametric vector field approach is beneficial for reducing the ill- posedness of the problem, rather than naïvely estimating non parametric vector fields. However, the 2D parametric vector field that relies solely on transformations fails incorporate 3D motion information, leading to sub-optimal performance of 32.65 dB.
3D nonpparamttric vector field. The non-parametric vector field is specialized to capture 3D mo- tions. However, when the non non-parametri method is used only, it struggles to effectively learn mo- tion behaviors due to huge ill-posedness, c.g., the presence of numerous feasible solutions, achieving insufficient performance (32.61 dB).
3D parametric vector field using depth information. The parametric vector fields using 3D trans- formation with depth measurements yield sub-optimal results. The deblurring result (32.47 dB)

---

Published as a conference paper at ICLR 2025
33.0-
32.99
33.0-
32.99
33.0-
32.49
32.95
32.92
32.9
52.9
32,92
32.85
52.9
52.05
32.8
52.8
12.76
32.8
82.7
92.7
$2.7
32.6
32.6
32.6
52.52
22.5
22.
52.5
22.4
hone Lapledn Geumetic
524
To Aug ATP
Scene
52.
16
(b)
(c)
Figure 8: Ablation studies on (a) Regularizations, (b) Controllability, and (c) of camera positions.
using the monocular depth estimates generated by Ke et al. (2024) is even worse than that of flat depth (32.66 dB). We believe that monocular depth values inherently inaceurate because they are typically relative depth information rather than accurate absolute depth measurements required for 3D transformations, leading to unfavorable blur synthesis.
2D parametric + 3D non-parametric vector field (ours), Our method fuses 2D parametric and 3D non parametraaec vector fields, which circumvents the need for absolute depth values. Therefore, we take advantage of both reducing ill- ill-posedness (by parametric) and effectively modeling 3D residual components (by non- na--parmmtrrc), achieving the best performance (32.99 dB).
4.5 ABLATION STUDY
Efficient deblurring models. In this section, discuss the effectiveness of our blur data augmen- tation scheme in constructing an efficient deblurring model. We train and evaluate NAFNet with different model sizes, incorporating sults show that our GeoSyn reduces the computational cost, i.e., GMACs, by up to 4x. Specifically, NAFNet-32 equipped with our data augmentation (16 GMACs, PSNR 32.59 dB) produces even better performance than that of NAFNet-64 64 (63.5 GMACs. PSNR 32.50 dB). This highlights the potential benefits of our blur synthesis model in building an efficient deblurring model.
data augmentation scheme. As shown in Table. 5, the
Effects regularization. To verify that the ambiguity regularization losses necessary, train deblurring models using our blur synthesizers trained without regularization, with Laplacian only (A1 0.1), with geometric only (A2 = 1.0), and with both regularizations. As shown in Fig. (a), we observe that each regularization technique contributes to reducing ambiguities, leading to better subsequent deblurring performance (32.50 + 32.87 or 32.92 dB). Furthermore, the results demonstrate that both regularizations are crucial for achieving further improvements, reaching 32.99 dB. We explore the hyperparameters {^r, }2} for more details in Section F.I of Appendix.
Controllability. We explore the controllability of GeoSyn. As discussed in Section 3.4, manipulate amplitude, phase of the vector fields, and scene contents to produce blur images with diverse blur patterns and scene contents. To confirm which augmentation type is more effective, we train deblurring models with amplitude, phase, scene, and all augmentations. As shown in Fig. 8 (b), each augmentation type demonstrates its effectiveness in improving deblurring performance. More- over, the results indicate that all augmentation types are necessary to achieve the best performance.
The number of camera positions. We investigate the effect of the number of camera positions when synthesizing blur image. We train our blur synthesizer with different numbers of camera positions: (4, 8, 16, 32, 64). and the deblurring model using individual synthesizers. As shown in Fig. (c), sufficient number of camera positions, e.g., more than 8, can capture intrinsic motion behaviors, such that it results in meaningful subsequent deblurring performance.
More ablation studies. For further insights, we perform an in-depth analysis of our blur synthesizer in Section F of Appendix.
5 CONCLUSIONS
In this paper, we propose a new 3D-aware blur synthesizer designed to generate diverse blur im- ages for data augmentation, improving deblurring performance. We integrate parametric and non- parametric vector fields to take advantage of reducing the ill-posedness and modeling 3D camera and object residual components. We demonstrate the effectiveness of our blur synthesizer ous network architectures and datasets. Our 3D-aware vector fields can be applied to optical flow and depth prediction from blur images while facilitating motion estimation for video generation.
vari-
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
This work was supported by Institute for Information & communications Technology Promo- tion(IITP) grant funded by the Korea goverament(MSITT) NNo.RS-2019-11190075. Artificial Intelli- gence Graduate School Program(KAISTT); 12002 Artificial Intelligence Innovation Hub).
REPRODUCIBILITY STATEMENT
To facilitate reproducibility, we present comprehensive implementation details for training our blur synthesizer in Section 4.1 and Fig. 2. We also describe the details of the data augmentation strategy. i.., where to synthesize, how to synthesize, and how augment during the training of the deblur- ring model, in Section B of Appendix. The hyperparameter settings are empirically optimized as described in Section 4.5 and Section F.1 of Appendix, and are summarized in Section 4.1. We also provide the pytorch pseudo-codes of our blur synthesis and blur data augmentation in Section G and H of Appendix.
REFERENCES
Guillermo Carbajal, Patricia Vitoria, José Lezama, and Pablo Musé. Blind motion deblurring with pixel-wise kernel estimation via kernel prediction networks. IEEE Transactions Computational Imaging, 2023.
Liangyu Chen, Xiaojic Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. The European Conference Computer Vision (ECCV), 2022.
Sung-Jin Cho, Seo-Won Ji, Jun-Pye Pyo Hong, Scung- Won Jung, and Sung-Jea Ko. Rethinking coarse- to-fine approach in single image deblurring. The IEEE International Conference on Computer Vision (ICCV), pp. 4641 1641-4650, 2021.
Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. Autoaugment: Learning augmentation strategies from data. The IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), 2019.
Ekin D. Cubuk, Barret Zoph. Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with a reduced search space. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Yuning Cui, Wenqi Ren, Xiaochun Cao, and Alois Knoll. Revitalizing convolutional network for image restoration. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024. doi: 10.1109/TPAMM.2224441199999
Terrance Vries and Graham W Taylor. Improved regularization convolutional neural networks with cutout. arXiv preprint 1rXiv:1708.04552 2017.
Zhenxuan Fang, Fangfang Wu, Weisheng Dong, Xin Li, Jinjian Wu, and Guangming Shi. Self- supervised non-uniform kernel estimation with flow-based motion prior for blind image deblur- ring. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18105- 181 2023.
Dong Gong. Jie Yang, Lingqiao Liu, Yanning Zhang, Ian Reid, Chunhua Shen, Anton Van Den Hen- gel, and Qinfeng Shi. From motion blur to motion flow: deep learning solution for removing heterogeneous motion blur. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2319-2328, 2017.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph. Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. International Conference on Learning Representations (ICLR), 2020.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and koray kavukcuoglu. Spatial transformer networks. Advances in Neural Information Processing Systems (NIPS), 2015.
11

---

Published as a conference paper at ICLR 2025
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion- image generators for monocular depth estimation. Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
Insoo Kim, Seungju Han, Ji-Won Baek, Seong-Jin Park, Jae-Joon Han, and Jinwoo Shin. Quality- agnostic image recognition via invertible decoder. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Insoo Kim, Jae Seok Choi, Geonseok Seo, Kinam Kwon, Jinwoo Shin, and Hyong-Euk Lee. Real- world efficient blind motion deblurring via blur pixel discretization. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 25879-25888, 2024.
Lingshun Kong, Jiangxin Dong, Mingqiang Li, Jianjun Ge, and Jinshan Pan. Efficient frequency domain-based transformers for high-quality image deblurring. The IEEE Conference Com- puter Vision and Pattern Recognition (CVPR), 2023.
Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jifí Matas. Deblur- gan: Blind motion deblurring using conditional adversarial networks. The IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 8183-8192, 2018.
Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. Deblurgan-v2: Deblurring (orders-of-magnntudd) faster and better. The IEEE International Conference on Computer Vision (ICCV), pp. 1578-8887, 2019.
Dongwoo Lee, JoonKyu Park, and Kyoung Mu Lee. GS-blur: A scene-based dataset for real- istic image deblurring. The eight Conference Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2024.
Dasong Li, Yi Zhang, Ka Chun Cheung, Xiaogang Wang, Hongwei Qin, and Hongsheng Li. Learn- ing degradation representations for. The European Conference on Computer Vision (ECCV), pp- 736-753, 2022.
Haoying Li, Ziran Zhang, Tingting Jiang, Peng Luo, Huajun Feng, and Zhihai Xu. Real-world deep local motion deblurring. Association for the Advancement of Artificial Intelligence (AAAI), 2023a.
Yawei Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx. Rakesh Ranjan, Radu Timofte, and Lue Van Gool. Efficient and explicit modelling of image hierarchies for image restoration. The IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2023b.
Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim. Fast autoaugment.
Advances in Neural Information Processing Systems (NIPS), pp. (665-6675, 2019.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:6608 /60.0088,, 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Conference on Learning Representations (ICLR), 2019.
Xintian Mao, Yiming Liu, Fengze Liu, Qingli Li, Wei Shen, and Yan Wang. Intriguing findings of frequency selection for image deblurring. Association for the Advancement of Artificial Intelli- gence (AAAI). 2023.
Richard M Murray, Zexiang Li, and S Shankar Sastry. A mathematical introduction to robotic manipulation. CRC press, 2017.
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3883-3891, 2017.
Seungjun Nah. Sanghyun Son, Jaerin Lee, and Kyoung Mu Lee. Clean images hard to reblur: Exploiting the ill pos-dd inverse task for dynamic scene deblurring. International Conference on Learning Representations (ICLR), 2022.
12

---

Published as a conference paper at ICLR 2025
Jaesung Rim, Haeyun Lee, Jucheol Won, and Sunghyun Cho. Real-worlh blur dataset for learning and benchmarking deblurring algorithms. The European Conference on Computer Vision (ECCV), pp. 184-201, 2020.
Jaesung Rim, Geonung Kim, Jungeon Kim, Junyong Lee, Seungyong Lee, and Sunghyun Cho. Realistic blur synthesis for learning image deblurring. The European Conference on Computer Vision (ECCV), 2022.
Ziyi Shen, Wenguan Wang, Xiankai Lu, Jianbing Shen, Haibin Ling, Tingfa Xu, and Ling Shao. Human-aware motion deblurring. The IEEE International Conference on Computer Vision (ICCV), pp. 5572-5581, 2019.
Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. Scale-recurrent network for deep image deblurring. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8174 8182, 2018.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. The European Conference on Computer Vision (ECCV), 2020.
Phong Tran, Anh Tuan Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11956-11965, 2021.
Fu-Jen Tsai, Yan-Tsung Peng, Yen- Yu Lin, Chung-Chi Tsai, and Chia-Wen Lin. Stripformer: Strip transformer for fast image deblurring. The European Conference on Computer Vision (ECCV), 2022.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxim: Multi-axis mlp for image processing. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5769-5780, 2022.
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. The IEEE Conference Com- puter Vision and Pattern Recognition (CVPR), pp. 17683-17693, 2022.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4))600- 612, 2004.
Oliver Whyte, Josef Sivic, Andrew Zisserman, and Jean Ponce. Non-uniform deblurring for shaken images. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
Hao Wu, Fu-Jen Tsai, Yan- -aunn Peng, Chung- Chi Tsai, Chia-War Lin, and Yen- Yu Lin. Id- blau: Image deblurring by implicit diffusion-based reblurring augmentation. The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2024.
Zhou Yang, Weisheng Dong, Xin Li, Mengluan Huang, Yulin Sun, and Guangming Shi. Vector quantization with self-attention for quality-independent representation learning. The IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jaejun Yoo, Namhyuk Ahn, and Kyung-Ah Sohn. Rethinking data augmentation for image super- resolution: A comprehensive analysis and new strategy. The IEÉE Conference Computer Vision and Pattern Recognition (CVPR), 2020.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. The IEEE International Conference on Computer Vision (ICCV), pp. 6023-6032, 2019.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14821-14831, 2021.
13

---

Published as a conference paper at ICLR 2025
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transforerr for high-resolution image restoration. The IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5-28-5738 2022.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations (ICLR), pp. 6438- 6447, 2018.
Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Bjom Stenger, Wei Liu, and Hongdong Li. De- blurring by realistic blurring. The IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 2737-2746, 2020.
Youjian Zhang, Chaoyue Wang, Stephen J Maybank, and Dacheng Tao. Exposure trajectory recov- ery from motion blur. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1 1): 7490-7504, 2021.
Zhihang Zhong, Ye Gao, Yingiang Zheng, and Bo Zheng. Efficient spatio-temporal recurrent neural network for video deblurring. The European Conference on Computer Vision (ECCV), pp. 191- 207, 2020.
14

---

Published as a conference paper at ICLR 2025
A
MORE DETAILS ON PROJECTED 3D RESIDUAL VECTOR
To derive the projected 3D residual vector, we recall the 3D transformation vector (4) as follows:
X
17 I + 77 y 1
(11)
(12)
(13)
T²(X)= =
Y
17
(21)
I + r7 (22)
1++ 17
(23)
(12)
Z
0
x 17 + r.
(3)
where R = RRz, Ry. 0] is the 2D transformation vector and & [E,, Ey. &g] is the 3D residual vector. Note that we use the notations It and instead T (u) and & (X) in (4) for simplicity here. Using a projection operation π R I with camera intrinsics K, we can compute the projected coordinate vector Tu( = [ar, v] = т (T++(X) K) = x + E; K). By using the pinhole
camera principle, we express a a component of the projected coordinate vector as
X P
Rs & P E
(13)
where R2 is a 2D transformation component, & and E are 3D residual components, fa focal length, and Pa is principal point. Here, we emphasize that x is characterized by the 2D transforma- tion component R2 and 3D residual components {Ez, } Since we defin x as a combination of 21) transformation component Rz and projected 3D residual component cr, i.e., x C(Rz, Gx) where C a composition function for integrating the two components, the projected 3D residual component ea contains the 3D residual components {Ez, }. focal length fz, and principal point Pa. These components are combined by a single value ez which is estimated by our motion estimator.
B DATA AUGMENTATION STRATEGY
Where to synthesize? The camera motion regarded as a global motion while the object motion treated a local motion. To accommodate the synthesized local motion, whose methodology is similar to other data augmentation schemes such as CutBlur (Yoo et al., 2020), CutMix (Yun et al., 2019) and CutOut (DeVries & Taylor, 2017). Specifically, randomly choose region for augmentation and apply controllable blur synthesis, i.e., Baug M o B + (1 M) @ S where M E {0, 1} is the mask image indicating where to synthesize, B means the synthesized blur image, S is the sharp image, and @ denotes the element-wise multiplication. Note that we will discuss our amplitude and phase augmentation policy, i.e., how to construct the synthesized blur image B, in the following paragraph. On the other hand, RealBlur has only camera motion blur images (global motion only). In this case, we apply the controllable blur synthesis in the whole region, e.g., Bag B. As shown in Table 6, the local data augmentation is beneficial for GoPro which contains camera and object motions. Meanwhile, the global data augmentation shows better performance on RealBlur-J since it contains camera motion only.
CutSyn strategy,
Table Comparison results on regions of blur augmentation. The best results are indicated in bold.
GoPro
RealBlur-J PSNR*
Region of augmentation
PSNRT
SSIM
SSIM
None
33.69
0.966
32.50
0.928
Global
34.02
0.969
32.99
0.936
Local (CutSyn)
34.09
0.969
32.94
0.936
How to synthesize? We discuss our amplitude and phase adjustment policy for constructing syn- thesized blur image B during the deblurring training- As discussed in Section 3.4, we introduce the amplitude control parameter a and phase control parameter to adjust the amplitude and phase of the displacement field, i.c., = a 6 ((Ø(8) + 3). Since many data augmentation approaches adopt random augmentation policy and show remarkable performance, we also use the random choice of both amplitude and phase control parameters. To determine the dynamic range of amplitude and conduct experiments
phase control parameters for random augmentations, presented in Table 7. For phase augmentation, although the best performance observed across multiple phase ranges, we select the range of -90 90" as it allows for phase augmentation over
various ranges, as
15

---

Published as a conference paper at ICLR 2025
wider range of phase variations. For amplitude augmentation, the bounded amplitude augmentation gives the best performance. Specifically, the dynamic range of the phase values is bounded between -180% and 180 even though adjusting the phase values by B. Mcanwhile, the dynamic range of amplitude values may not be bounded. For example, a large amplitude control parameter can sig- nificantly increase the blur amount in large-motion scenes, resulting in unnatural large-motion blur images. Such unbounded amplitude values may not be optimal for amplitude augmentation. To address this, we bound the absolute amplitude values adjusted by a to the values between 1.0x and 1.5x, and a is randomly chosen to keep the bounded range. As result, the bounded amplitude augmentation policy achieves the best performance. In summary, we randomly select a keep the absolute amplitude values between 1.0x and 1.5x for amplitude augmentation, and we use random |3 values between -900 and 90º for phase augmentation unless otherwise specified.
Table 7: Effects on amplitude / phase augmentation policies. The best results are indicated in bold.
Method
Adjustment
Range
PSNR 1
SSIM 1
0.5x 1.0x
32.68
0.932
1.0x ~ 1.5x
32.80
0.933
Amplituee a
1.0x 2.0x
32.78
0.933
1,0x ~ 4.00x
32.76
0.933
Bounded Amplitudes
32.82
0.933
GeoSyn (ours)
-15 ~ 15
32.73
0.932
-30% 30
32.78
0.933
Phase /
-45" ~ 45"
32.78
0.933
-90 90"
32.78
0.933
-1 1800 ~ 180
32.65
0.932
Data augmentation during the deblurring training. When training the deblurring model, we use both real blur image B and augmented blur image Bang based on the accumulated gradient (AG) strategy. Specifically, it relies on accumulating gradients in both real data and synthesized data (1:1) and then backpropagation. As it seems similar to 2 increases of mini-batch size, experiment 2x increases of mini-batch size using AG technique, resulting in slight performance improvement, 32.50 - 32.56 dB, shown in "Accumulated gradient" of Table 8. We clarify that the performance improvement is not due to gradient accumulation itself, but rather our data augmentation scheme 32.50 - 32.99 dB as shown Table
Table 8: Effects on accumulated gradients. The best results are indicated in bold.
Methods
PSNR 1
SSIM 1
None
32.50
0.928
Accumulated gradient
32.56
0.929
GeoSyn (ours)
32.99
0.936
C CONSIDERATION ON THE COMPENSATION NETWORK
As shown in Fig. 2, we use the compensation network to address the photometric issues. Basi- cally, RealBlur (Rim al., 2020) and RSBlur (Rim et al., 2022) datasets are acquired using dual- camera systems. These systems consist of two cameras with different lenses or sensors, resulting in color drifts. Furthermore, since blur and sharp images require different exposure time, this leads to variations in brightness and contrast. Although such photometric inconsistencies are corrected by post-proc,ssing they are not fully compensated. This means that the synthesized blur image using a sharp image may be different from ground-trute blur image in terms of color, brightness, or contrast. This difference may disrupt motion modeling. To confirm that the compensation net- work is necessary, we conduct experiments with the configurations: (1) training our blur synthesizer without the compensation network and using the estimated blur image B for blur data augmenta- tion, (2) training our blur synthesizer with the compensation network and using the estimated blur image B for blur data augmentation (ours), (3) training our blur synthesizer with the compensation network and using the compensated blur image he (B) for blur data augmentation, and (4) training our blur synthesizer with the compensation network and using the compensated blur image he * (B) with finetuning. For simplicity, refer to configuration (1) as C1, configuration (2) as C2, config-
16

---

Published as a conference paper at ICLR 2025
uration (3) C3, and configuration (4) C4. As shown in Table 9, the results show that the C2 gives the best deblurring performance (32.99 dB). We believe that the usage of the compensation network during the synthesizer training enables the 3D-aware vector fields to focus exclusively on learning blur components, thereby free from photometric variations such as color drifts and sensor differences. On the other hand, the C1 shows sub-optimal result (32.86 dB) since it may suffer from the photometric issues for motion modeling. Furthermore, when training deblurring models with our blur data augmentation scheme, various blur patterns can be generated by manipulating the amplitude and phase of the vector field. Such modified blur patterns may be unseen to the compensa- tion network, which may cause unexpected artifacts or blur effects after the compensation network. Therefore, the C3 leads to sub-optimal performance (32.75 78 dB), compared to that of the C2 (32.99 dB). To address this, we finetune the compensation network during the deblurring training, which is indicated as the C4. We observe that it somewhat handles unseen blur patterns by finetuning the compensation model (32.78 + 32.89 dB). but it lags behind our configuration, C2. As a result, we empirically prove that the C2 gives the best performance, and use it unless otherwise specified.
Table Effects on the compensation network. hg* is a finetuned version of hg. The best results are indicated in bold.
Contigurations
Compensation Net
Synthetic Blur
PSNR
SSIM
CI
B
32.86
0.935
C2 (ours)
B
32.99
0.936
C3
hg (B)
32.78
0.934
C4
hg*
32.89
0.934
D COMPARISON RESULTS WITH ID-BLAU
We compare our method with ID-Blau (Wu et al., 2024) to demonstrate its effectiveness. We conduct experiments across various network architectures such as MIMO-UNet+ (Cho et al., 2021) and FFT- former (Kong et al., 2023), and datasets such as GoPro (Nah et al., 2017) and RealBlur-J (Rim et al., 2020). As shown in Table. 10, our GeoSyn gives better performance than ID-Blau. In particular, our GeoSyn achieves a PSNR of 33.01 dB on GoPro, outperforming 32.93 dB obtained by ID-Blau MIMO-UNet+. We believe that our blur synthesizer effectively accounts for explicit 3D motion modeling, leading to better performance. Furthermore, our method shows remarkable performance improvement on RealBlur-J, compared with that of ID-Blau. Notably, while ID-Blau shows the per- formance improvement in FFTformer from 32.62 to 32.88 dB, our GeoSyn achieves significant performance improvement, reaching 33.68 dB. Unlike ID-Blau which requires video frame images, GeoSyn is compatible with training RealBlur-J which only contains blur-sharp image pairs. Hence, our method can generate more dataset- motion patterns for data augmentation, such that it yields better subsequent deblurring performance on RealBlur-J.
Table 10: Comparison results against D-Blau. The best results are indicated bold.
GoPro
RealBlur-J PSNR
Methods
PSNR1
SSIM
SSIMT
MIMO-UNet+
32.44
0.957
31.92
0.916
ID-Blau
32.93
0.961
31.96
0.921
+ GeoSyn
33.01
0.962
32.55
0.925
FF Tformer
34.21
0.969
32.62
0.932
+ ID-Blau
34.36
0.970
32.88
0.934
+ GeoSyn
34.39
0.970
33.68
0.938
E GENERALIZATION ABILITY
To demonstrate the generalization ability of our method, we conduct cross-dataset evaluations. First, we train both our blur synthesis and deblurring models on RealBlur (i.e., GeoSyn- R) and test on Re- alBlur (Rim et al., 2020), RSBlur (Rim et al., 2022) and BSD (Zhong al., 2020). Additionally, we train our blur synthesis model on GoPro and deblurring model on RealBlur (i.e., GeoSyn-Gi and test again on RealBlur, RSBlur and BSD. As shown in Table 1, GeoSyn-R shows remarkable
17

---

Published as a conference paper at ICLR 2025
performance RealBlur, and our GeoSyn-G demonstrates promising generalization performance on RSBlur and BSD. This relies on what dataset is used for training our blur synthesizer. Specifi- cally, the GeoSyn-R is trained on RealBlur, enabling it to generate diverse and datasct-specifie blur patterns that contribute to performance improvement on RealBfur: Even though the GeoSyn-R uses only RealBlur in both trainings, it also shows good generalization results RSBlur and BSD be- cause it can generate numerous and diverse blur patterns during the deblurring training, On the other hand, our GeoSyn-G leverages separate datasets for the blur synthesizer (GoPro) and the deblurring model (RealBlur), enabling it to benefit from multiple blur datasets. As result, it achieves superior generalization performance (see the results on BSD Test set).
Table 11; Ablation study on the cross-data validation. The best results are indicated in bold.
Train
Train (Synthesizer)
NAFNet
MIMO-UNet+
Test
Methods
(Deblur)
PSNR
SSIM
PSNR
SSIM
No Aug
32.50
0.928
31.92
0.919
GoPro RealBlur
ID-BIau
32.70
0.932
31.96
0.921
RealBlur
RealBlur
GeoSyn-R
32.99
0.936
32.55
0.925
GoPro
GeoSyn-G
32.94
0.935
32.47
0.924
No Aug 30.61
0.809
29.72
0.790
GoPro
ID-Blau 30.90
0,814
29.43
0.786
RealBlur
RSBlur
RealBlur
GeoSyn-R
30.91
0.815
29.95
0.794
GoPro
GeoSyn-G
30.98
0.815
29.81
0.794
No Aug
29.67
0.893
29.25
0.891
GoPro
ID-Blau
30.42
0,907
28.93
0.882
RealBlur
BSD
RealBlur
GeoSyn-R
30.83
0.911
29.73
0.899
GoPro
GeoSyn-G
30.91
0.912
29.91
0.904
F
COMPREHENSIVE ANALYSIS ON GEOSYN
F.1 EFFECTS ON
We examine the hyperparameters \ and A2 to confirm performance sensitivity across the hyperpa- rameters. We train our blur synthesizers using combinations of X (0.1, 1.0). A2 1.0). resulting in the pairs (Λ₃ A2) such as (0.1, 0.1), (0.1, (1.0, 0.1). and (1.0, 1.0). The results are given Table. 12. We observe that the higher impact (A2 1.0) of invertible geometric loss gives better performance. This means that it accounts for the importance geometric consistency for blur synthesis. In contrast, we observe that the greater impact (A1 1.0) of Laplacian smoothing loss leads to decreased deblurring performance. This is because a stronger smoothing constraint impedes the learning of diverse motion patterns, reducing data diversity for data augmentation. Subsequently, it limits performance improvements.
Table 12: Ablation study on hyperparameters A and A2. The best results are indicated in bold.
Methods
A1
A2
PSNR +
SSIM 1
0.1
0.1
32.90
0.935
0.1
1.0
32.99
0.936
GeoSyn (ours)
1.0
0.1
32.76
0.933
1.0
1.0
32.84
0.934
F.2 BLUR DIVERSITY AND ITS RELATIONSHIP M
When the number of camera positions M is small (e.g., M = 4), the expressive power of complex motion using only four camera exposures becomes inherently constrained. Namely, complex mo- tion trajectory is represented by simplified form, leading to lack of blur diversity. In contrast, increasing M to 16 provides the capability to capture intricate motion patterns, leading to better blur diversity. As shown in Fig. 9, observe that the larger number of M, i.e., M 16 yields more accurate motion results, ultimately promoting a wider range blur patterns. Therefore, leads to better final deblurring results discussed in Section 4.5.
18

---

Published as a conference paper at ICLR 2025
000
000
DO0
Kddn
Blur Image
Synthetic Blur (M = 4)
Synthetic Blur (M = 16)
Blur Trajectory (M = 4)
Blur Trajectory (M = 16)
Figure Blur diversity and its relationship with the number of camera positions M.
F.3 PERFORMANCE SENSITIVITY AND ITS RELATIONSHIP TO GEOMETRIC CONSISTENCY
To investigate the performance sensitivity to the accuracy of the vector field, we experiment by introducing random perturbations the vector fields. We believe that the robustness of the final deblurring performance is closely tied to the geometric consistency of the vector field. As shown in Table 13, without geometric consistency (A2 0.0), the vector field lacks geometric coherence, and thus even no perturbation leads to performance degradation (32.52 dB) compared to the baseline performance, i.e., no data augmentation (32.56 dB). In contrast, with strong geometric consistency (A2 = 1.0), the deblurring model remains robust the perturbations of the vector field, consistently outperforming the baseline (indicated in bold). For mid-level geometric consistency (A2 = 0.5), the vector field is generally robust small perturbations (indicated in bold), but large perturbations disrupt its geometric coherence and adversely affect performance (32.51 dB). These observations highlight the critical role of our geometric consistency regularization in mitigating the performance sensitivity to the accuracy of the vector field. Namely, blur data augmentation with inaccurate vector fields reduces performance gain but does not degrade baseline deblurring performance, long as our motion estimator is trained under strong geometric consistency.
Table 13: Ablation study on performance sensitivity under perturbations the vector fields. The performance exceeding the baseline is indicated in bold.
Perturbations
Methods
A2
0
0.001
0.005
0.01
Baseline
32.56
(No Augmentation)
1.0
32.92
32.90
32.83
32.70
With GeoSyn
0.5
32.77
32.77
32.75
32.51
0.0
32.52
32.51
32.49
32.49
F.4 VISUAL COMPARISON ON BLUR TRAJECTORIES
Our primal goal is to build a controllable blur synthesizer that estimates motions from a single blur image. This enables our blur synthesizer to be directly applicable various blur datasets (blur- sharp image pairs) such as GoPro (Nah et al., 2017), RealBlur (Rim et al., 2020), RSBlur (Rim al., 2022), BSD (Zhong et al., 2020), and ReLoBlur (Li et al., 2023a). However,
compare
19

---

Published as a conference paper at ICLR 2025
blur trajectories estimated by blur synthesizer with those obtained from video frames using an off-the-shelf optical flow model, e.g., RAFT (Teed Deng, 2020). To this end, we utilize GoPro, which provides video frame images that allow us extract optical flow maps. Then, we convert these optical flow maps into vector fields to represent a real-like blur trajectory, which provides a straightforward way verify that our blur trajectory aligns well with real ones. The results visu- alized in Fig. 10. We found that the blur trajectories are nearly identical, confirming the alignment between our blur trajectories and the real one. Despite estimating blur trajectories from single blur images, our method is comparable to those derived from video frames.
Blur Image
Synthetic Blur Image (Blur Image, Ours)
Blur Trajectory (Video Frame Images)
Blur Trajectory (Blur Image, Ours)
Figure 10: Comparison results blur trajectories (video frame images vs. single blur image).
20

---

Published as a conference paper at ICLR 2025
G VECTOR FIELD GENERATION IN PYTORCH PSEUDOCODE
def vec2skew(v)) (v)
torch. zeros (1, atype=torch. 30 device=v. device)
skew v0 torch. cat zero,
-v|2:31, v|1:211) # 13, 2)
skew vl torch. (I v|2:3], zero, -v[0:1]1)
skew_v2 - march.cate ( [-v[[:2]]
v[0:1], serol)
skew_v torch.stane ([skew_vO, skew_vl, skew_v2l, dim=0) (3, 3)
return skew_v (3,
def Exp (r)
skew_r vec2skew (r) # (3, 3)
norm_r () 1e-15 eye torch. eye dtype=torch. f2,at32, device=r. device=r.device. device)
R eye + (torch.sin (norm_r) / norm_r) + skew_r
[(1 torch, cos (norm_r)) norm_r++2) (skew_r @ skew_r)
return
def make (E, tl:
R = Exp (r) (3, 3) c2w torch. cat ( [R, t. u.unquueeze()) (1)), dim=1) # (3, return c2w
def complex_to to to_polar(2): (2): 2_complex = 2(..., 0] 1j z(..., 1]
return torch, abs (z_complex), forch.angles (2_moo__ll)
def polar_te to tt__oool_ecccceeeteeeeeeeeeeeeet (amplitude, phase)
real = amplitude + torch.cos cos (phase) imag = amplitude + torch.sin (phase) return torch. stack (real, imag), , axis=-1)
def vec sec_field_gen (motion_results,
control_paramt img_size): Σ, t [b, num pamesssss__ * 31
2D Rigid
--> -->
3D-Aware Residual Field --> _oc_loca,, phase_locat local [b, h, w]
r, t, amp_local, phase_local motion_ressin results
Control paramoters are randomly chosen for blur data augmentation amp_control, phase_contros panarol_params
(1, 0) for training
r_split, t_split r.split dim=l), spspitt dim-1) total_grid, cotal_grid_invert for i in range (len (r__stt))lt)))
[], []
rigid_20 2d
for j in range (len (r split(il))
append rigid_2d. rigid_2d.appendd (make (ttpaee_ccci___ss_t__i__e___ rigid_2d torch, stack (rigid_d2ddd
(i]
grid _ano = = P.affine_griddtoochh.es
3). cuda unsqueeze (0).
repeat ( 1, I:, :2, :3],
grid_2d_rigid F.fffine_griddtriiiddd... :2, :3], img_size)
res_grid_2dd grid_2d Idinn :2]
amp_global. phase_global complex to polar (res seee_rriidddeeee 2d rdggid
res_grid_3d_awan polar_to tt__ttt___eeee__ee tattaa__tt_ccopeeee____eee (amp_gi obal amp_ local
amp_control, phase_global obaal + phase_local + phase_control)
total
(res grid 3d
grid 2d ca__)
total -ital_grid_ii invert apw.ndrr.ppend-reer grid 3d aware grid 2d 2d cano) total_grit torch. stack (total (tatall_ttttriiiia grid) total__giddinvert corch.star (totatgrridddddieee
return total_grid, iotd
21

---

Published as a conference paper at ICLR 2025
H BLUR SYNTHESIS AND DATA AUGMENTATION IN PYTORCH PSEUDOCODE
model motion est, compensation net models $ neural networks def blur synthesis (blur img, sharp ing,
motion results model motion est (blur (img) b, c, h, = bchw sharp img. sizel) grids, grids_inv = vec_field_gen reottonn control bchw)
num_poses, out, out_invert, lap_reg_loss 16, [l, [], for range (num (auu_pooees]__
grid, grid_inv grlds grids_inv[1]
lap_reg_loss = lap_reg_loss + compute lap loss (grid)
gaussian_noise torch. rench.randn_iii like (sharp_ . cuda () 0. 0112
view view_ing__st = torch.clang (torch.nn. wuncciionaniiii. www.ioooaa.iiideee.e sample sharp_img, grid, mode= billinar') + gauss gaussian_note aaciiannnniisss 0, 1)
out out. (vapennniiie___n___sy))
out aapperrt w...nnniiionaa.. wwwiiionnn.iiii....o sample (view img syn, grid inv, mode=' bilinear')
out torch, stack (out) avt_avg out oat.mean())
out out_invert torch. stack (out_inverts
$ Sram_poses, by Wj
out_invert out_invert reshape (num poun__pses+b,s.. w,
sharp_imggreehhpped sharp_img.unspppeeeooo unsqueeze (0) repeat (num_poses, 1, .reshape (num pawammmpssssbbeocm 2, geo geo_reg_loss = nn. inlsls () (out_inverrs @earp_ing_att rmg_res
compen_pred_blur_imme compensation_net (out_avg)
blur_syn_results torch. cat
out_avg)
return blur_syn__ett___eeeeeee syn_results, lap_reg loss, geo_reg loss
def blur data augmentation (blur_img, target_img blur synthesis): blur_ing, size [0]
amp_control = corch.empty () (nn.f/r).....__11111. 2.0)
phase_come coase_cootoll corch.cmpty 1nni., (-1.000_ mmath.p1/2.00
control_params (amp_control, phase_control)
perm_ide idx torch. randperm (B)
target_shiee shuf = target imgiinglso perm, idx]
prea imgs, blur synthesis (blur img, target_ shuf, control params)
return pred_ings I: 3;, =1 (etach
def compute_lap llsee_llpp_llssssseeesss (grid)
grid = (torch, clamp (grid, -1.0, 1.0) 1.0) / 2.0 center gridl: 1:-1, 1:-1,:]
up, down gridl: :-2, 1:-1,;], gridl: 2:, 1:-1,
left, right gridl: 1:-1, :-2,;l, grid[: 1:-1, 2;y:] laplacian = 4 center (up down + left + right)
return (Haplacian 2) sum 2, 3)))
def (blur__mm img, sharp_ing, blur blur_synthesis)
pred_imgs, laps_lap, loss_gec blur_synthie (blur_ing, sharp_ing)
pred blur avg pred ing pred lous_blurl blurl nn. Llloss () (pred blur, blur blur_img) loss_blur2 nn. Llloss () (pred_blur_avg, blur_ing)
loss
return
loss_blur2 1.0+loss_goo 0.14loss_lap
22

---

Published as a conference paper at ICLR 2025
I GENERALIZATION TO REAL-WORLD BLUR IMAGES
ANUTS
thes badae Goonge thongle londias wert uol Hotdly the are calionss She warted Gesegy ort fitlng onr the flndd on diame.
West shep risiced then kenllas George thoughh the londar weet cam Her www.sgg----eeeeeee Recaddy the was curious, Sie wanned
Ais lasage Hegi cating, held ourihie band diane.
Blur Input
FFTformer
ANOrSS
ANOTS
Mlees Iheaal bisties Geoorge dlouuhh the lonadlas
they visized the knnlas. George thought the koulss were Har friendy one- was curious, too. Sbe wanted our his hand to shace.
aaa iiteettl ane Ses cutvonss soc Sher wanred
-Nlr-Sinnng dating dfefe feeld faam us
what George was exting,
FFfformer + ID-Blau
FFTformer + GeoSyn (ours)
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer GeoSyn (ours)
23

---

Published as a conference paper at ICLR 2025
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer + GeoSyn (ours)
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer GeoSyn (ours)
24

---

Published as a conference paper at ICLR 2025
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer + GeoSyn (ours)
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer + GeoSyn (ours)
25

---

Published as a conference paper at ICLR 2025
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer + GeoSyn (ours)
Blur Input
FFTformer
FFTformer + ID-Blau
FFTformer + GeoSyn (ours)
Figure 11: Visual results on real-world blur images.
26

---

Published as a conference paper at ICLR 2025
J
QUALITATIVE RESULTS ON GOPRO
63B 4823
63R 4823
63X 4823
4823
Blur
ID-Blau
GeoSyn (ours)
Sharp
Figure 12: Qualitative results on GoPro (Nah al., 2017). All models are trained with GoPro and are based on MIMO-UNet (Cho et al., 2021).
27

---

Published as a conference paper at ICLR 2025
K
QUALITATIVE RESULTS ON REALBLUR-J
U48
U048
U048
OP
E OH
OH
DMAAO CONBIAAN
OMAAT HOMEA
OWA/O HOUSE
HARDN
4 BR
ARRER
0/3
REV
2IE4 B
4244
+344
10
A 16:0 e
8ICH
WCE!!
WIC!!
MIC!!
MIC!! 053)
0531
611-9154
611-9154
611- 911-9154
611 611-9154
AIOL VOO
APOI S:
H: 2010
12.000
PETUBI
112 5000
5400
PETIED
H11
9/1
500ml
PETRO
11.500
010
H1
ENERCING
H2 1..
Sctre
+2.000
BA9 BIR 73RA
CIEED 053-616-7366
953-616-7364
053-616-7364
ELPYOTO
PREUNN
EUPPATI
Viratub
512
Blur Input
ID-Blau
GeoSyn (ours)
Ground -ruth Sharp
Figure 13: Qualitative deblurring results on RealBlur-J (Rim et al., 2020), All models are trained
with RealBlur-J and are based on NAFNet-64 (Chen et al., 2022).
28

---

Published as a conference paper at ICLR 2025
L
BLUR SYNTHESIS AND TRAJECTORY RESULTS
SMGIAN
NesHEs
SOP
Ii J
Blur Image
Synthetic Image
2D Blur Trajectory
3D Blur Trajectory
Figure 14: Synthesized blur images and blur trajectories on 2D camera motion (1st row), 3D camera
motion (2 3rd rows), and 3D object camera motion examples (4 - 6th rows).
29

---

Published as a conference paper at ICLR 2025
M BLUR SYNTHESIS AND TRAJECTORY RESULTS ON OBJECT MOTION
21 O a O 21
T
21 O DD 21
Blur Image
Synthetic Blur Image
Sharp Image
Blur Trajectory
Figure 15: Blur synthesis results and their blur trajectories for object motion using RSBlur dataset (Rim et al., 2022).
30

---

Published as a conference paper at ICLR 2025
N
CONTROLLABLE BLUR IMAGE SYNTHESIS ON GOPRO
Blur Image
Synthesized Image
Phase 90
Amplitude 2
Figure 16: Controllable blur image synthesis on GoPro (Nah et 2017).
31

---

Published as a conference paper at ICLR 2025
O
CONTROLLABLE BLUR IMAGE SYNTHESIS ON REALBLUR-J
616
Blur Image
Synthesized Image
Phase 90
Amplitude 2
Figure 17: Controllable blur image synthesis on RealBlur-J (Rim et al., 2020).
32


Reviews:
[{"review_id": "p1UIAIbLNC", "reviewer": "Reviewer_BF8H", "summary": "The manuscript presents a 3D-aware blur synthesizer designed to generate diverse blur images for data augmentation, which further improves the deblurring performance of existing approaches.  The blur synthesizer enables controllable blur data augmentation not requiring any extra data during the training by combining 2D motion and 3D residual component.", "questions": "See the Weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The strengths of this paper could be summarized as follows:\n\n1. Comprehensive experiments have been conducted to demonstrate the \neffectiveness of the proposed method.\n\n\n2. The proposed blur image augmentation method could be beneficial to relative research.", "weaknesses": "There are some concerns and problems that should be further clarified and addressed:\n1. The amplitude-phase integration directly offers controllability to motion patterns behind the blur images. However, it seems unable to explicitly adjust the overall blur intensity. Have the authors ever considered providing a way to control it? Such as adjusting exposure time or introducing a blur intensity parameter.\n\n\n2. Please give more details about how to derive the projected 3D residual vector.\n\n\n3. Authors have explored the effects of different values of M on the final deblurring performance. But what concerns me most is the correlation between the value of M and blur diversity, which could possibly provide more controllability to the blur synthesizer. For example, the visualizations or metrics showing how blur patterns change as M varies, or specific experiments to show blur diversity and its relationship to M.\n\n\n4. As observed by the authors, inaccurate depth measurements can lead to performance degradation. On the other hand, Motion estimation itself is a very complex problem, even under 2D space. So, how can the authors guarantee that the inaccuracy of the estimated displacement field will not corrupt the final deblurring performance? Or authors can prove that deblurring effects are not sensitive to the accuracy of the motion field. For example, when training the blur synthesizer, we can randomly apply perturbations to the generated displacement fields to see whether it would make a difference.\n\n\n5. In the experiments, the blur synthesis model is separately trained for different datasets. Nevertheless, this reduces the credit of the proposed model having better generalization ability compared with existing approaches. I would like to see more experiments involving a blur model trained on a single dataset and tested on different datasets.\n\n6. Please provide more comparisons of real-world blurred images captured by the authors and clarify whether the compared models utilize existing blur augmentation methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PNffDMc4tO", "reviewer": "Reviewer_RsRS", "summary": "This paper introduces a method for controllable blur image synthesis to address the challenges in capturing datasets for deblurring (GeoSyn). Specifically, it estimates M-many 3D vector fields for each pixel of a blur image and reconstructs the original blur image through projections of these estimated vector fields. During evaluation, blur with various trajectories can be generated by adjusting these 3D-aware vector fields. The method combines both parametric and non-parametric trajectory fields to capture both object motion and 3D camera motion. Training deblurring models with this augmentation leads to improved performance.", "questions": "1. Could the authors provide the distribution of 2D-based motion trajectories (obtained via optical flow between the sharp and original blur images) and the distribution of the proposed 3D-aware vector fields? The reviewer would like to see if the augmented blur trajectories align well with real ones. This could be visualized using t-SNE projection of the motion vectors.\n\n2. How do the authors obtain the camera intrinsics, as this could be ill-posed from a single image? Do the authors fix the intrinsic parameters and then optimize only the extrinsic scaling factor?\n\n3. For the synthetic dataset, the reverted sharp image mean({\\sum^{M}_{*=1}}\\tilde{S}_{τ}_{*}) should be identical to the sharp image, based on Equation 10. Could the authors provide quantitative results on this?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper effectively identifies key challenges in blur data augmentation and addresses them logically and directly by leveraging 3D spaces.\n\n2. The writing and figures clearly illustrate the motivation and methodology of the proposed approach.\n\n3. The ablation studies and experiments provide strong evidence of the proposed augmentation's effectiveness.", "weaknesses": "1. Limitation of motion trajectories: Since motion trajectories are determined independently of content, unrealistic trajectories can occur. For instance, part of a car might move left while another part moves right. Additionally, the method cannot represent motion caused by shape-changing objects. While real-world blur includes cases (e.g., folding fingers or blinking eyes), the proposed method only addresses part of this blur complexity.\n\n2. The proposed method increases computational costs. The reviewer acknowledges that the augmentation scheme could help build an efficient deblurring mode, as mentioned in lines 507-513 of the main paper. However, could the authors provide quantitative metrics (latency, GMACs, and memory usage) when adopting GeoSyn?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DxybDaN0Cc", "reviewer": "Reviewer_iVrH", "summary": "This paper presents a novel, controllable blur data augmentation method aimed at enhancing the performance of learning-based deblurring models. The authors propose using a neural network to learn both a parametric vector field (2D rigid transformation) and a non-parametric vector field (3D residual field) to simulate 3D transformations. By adjusting these learned vector fields, the method can generate diverse and realistic blur patterns for data augmentation.", "questions": "Please address the concerns raised in the weaknesses section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The motivation is well-founded: improving deblurring performance through data augmentation is highly cost-effective.  \nThe paper is clearly written and easy to follow.  \nBased on the experimental results, the proposed method achieves state-of-the-art performance in deblurring tasks.", "weaknesses": "The upper performance limit for video deblurring is typically higher than for image deblurring, as videos provide more temporal information. It remains unclear whether the proposed data augmentation method can handle video sequences while maintaining temporal consistency, which is crucial for real-world applications like Apple Live Photos.\n\nThe paper repeatedly emphasizes the term \"physical,\" which this reviewer finds somewhat exaggerated. From a single image, a neural network theoretically cannot disentangle object motion from camera motion due to the numerous possible motion combinations. At best, the method can adapt to the motion patterns of the dataset it is trained on. Additionally, even if the transformation accurately models 3D motion, the transformed image is still 2D and inherently lacks occluded information, making it less \"physical\" than synthesis methods using nearby video frames.\n\nThe paper does not sufficiently address blur caused by object motion (This reviewer did not see any blurred images dominated by object motion). This raises doubts about the method's \"physical\" nature and its effectiveness in real-world scenarios involving complex object motion.\n\n> \"ID-Blau requires ground-truth video frames to extract optical flows, so it cannot be trained on datasets containing only blurred and sharp images, such as RealBlur and RSBlur.\"\n\nGiven the advantage of synthesis methods using nearby frames, for a fair comparison, ID-Blau could utilize datasets like BSD[1] and RBI[2], which contain blur-sharp video pairs. Specifically, the BSD[1] dataset offers subsets with varying blur intensities. A cross-dataset evaluation between RealBlur and BSD is necessary to demonstrate the generalization capability of the proposed data augmentation method.\n\n[1] Efficient spatio-temporal recurrent neural network for video deblurring. In *Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16* (pp. 191-207). Springer International Publishing.\n\n[2] Blur interpolation transformer for real-world motion from blur. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition* (pp. 5713-5723).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)