An Intelligent Agentic System for Complex Image Restoration Problems
Authors: Kaiwen Zhu
Jinjin Gu
Zhiyuan You
Yu Qiao
Chao Dong
Venue: ICLR 2025 Poster (2025)
Keywords: image restoration
low-level vision
agent
large language model
vision language model
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper proposes an LLM-based agentic system that utilize various tools for complex image restoration problems.

Abstract: Real-world image restoration (IR) is inherently complex and often requires combining multiple specialized models to address diverse degradations. Inspired by human problem-solving, we propose AgenticIR, an agentic system that mimics the human approach to image processing by following five key stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. AgenticIR leverages large language models (LLMs) and vision-language models (VLMs) that interact via text generation to dynamically operate a toolbox of IR models. We fine-tune VLMs for image quality analysis and employ LLMs for reasoning, guiding the system step by step. To compensate for LLMs' lack of specific IR knowledge and experience, we introduce a self-exploration method, allowing the LLM to observe and summarize restoration results into referenceable documents. Experiments demonstrate AgenticIR's potential in handling complex IR tasks, representing a promising path toward achieving general intelligence in visual processing.

Published as a conference paper at ICLR 2025
AN INTELLIGENT AGENTIC SYSTEM FOR COMPLEX IMAGE RESTORATION PROBLEMS
Kaiwen Zhu Jinjin Gu Zhiyuan You's
Yu Qiao²
Chao Dong 1
Shanghai Jiao Tong University SShaggaai Artificial Intelligence Laboratory
'hhe University of Sydney *The Chinese University of Hong Kong
SShnnzhen Institutes of Advanced Technology, Chinese Academy of Sciences
zhukai lab org. en, jinjin. guésydney edu.au, Thiyyanyou@foomale qi acyuuppllll ab. org. cn, chao. danges singeesiiee ac. aw..essatti...o.....mmmoom
ABSTRACT
Real-world image restoration (IR) is inherently complex and often requires com- bining multiple specialized models to address diverse degradations. Inspired by human problem-solving we propose AgenticIR, an agentic system that mimics the human approach to image processing by following five key stages: Percep- tion, Scheduling, Execution, Reflection, and Rescheduling. AgenticIR leverages large language models (LLMs) and vision-language models (VLMs) that interact via text generation to dynamically operate toolbox of IR models. We fine-tune VLMs for image quality analysis and employ LLMs for reasoning, guiding the system step by step. To compensate for LLMs' lack of specific IR knowledge and experience, we introduce self-exploration method, allowing the LLM to observe and summarize restoration results into referenceable documents. Experiments demonstrate AgenticIR's potential in handling complex tasks, representing a promising path toward achieving general intelligence in visual processing. The code is available at https:: : /tte:://ggttttt github. com/ Kaiwen- wchu/Aggnticll
INTRODUCTION
Image restoration (IR) problems in real-world scenarios inher- ently complex. Over the past decades, researchers have abstracted various degradation phenomena into independent IR tasks, propos- ing advanced models tailored to address these problems. However, in practice, these models are seldom used isolation. Instead, they serve as optional tools that collaborate with other methods to solve complex problems. For example, today's most successful image processing softwares usually integrate multiple models, allowing users to analyze the image, select models, and interact with the pro- cessing. This approach enables users to accomplish tasks far more complex than what any single model could achieve, even when us- ing the most basic processing operations. To advance beyond exist- ing models designed for passive and structured IR tasks, we propose systematic method capable of assuming dynamic and agentic role in diverse and complex IR scenarios. We aim for this system function like a human, assessing the image, selecting, planning, and executing various existing IR models to address challenges that individual models alone struggle solve. We believe this could be one of the promising paths towards general intelligence in IR.
Iizur
Perception
Tuelity
Scheduling
Evecution
Aduated plar
Rescheduling
Intrmedate
reaah
Rolbuck
Reffection
Durpst
Figure 1: The five stages of human process of IR (some details are hidden).
To achieve this goal, we first need abstract and generalize how human users use tools to handle IR tasks in real-world scenarios. Fig. illustrates an example. When faced with such an input image, human users may first analyze its quality and degradations (the Perception stage), realizing that contains two types of degradation: noise and rain streaks. Based on the analysis and professional
"This work was done during his internship at Shanghai Artificial Intelligence Laboratory.
1 Equal contribution. *Corresponding author.

---

Published as a conference paper at ICLR 2025
experience, they may devise a plan for using tools (the Scheduling stage) for instance, applying a denoising model before a deraining model to prevent noise from affecting the deraining effect. Following the plan, they sequentially apply these models to the image step by step (the Execution stage). However, the effects of the models can be dynamic and unpredictable; even models with the same function may behave differently, making it difficult determine the optimal plan directly. Typically, after an operation, human users may re-analyze the image to assess whether the tool was effective (the Reflection stage). If the tool was ineffective or even worsened the image, they undo the operation and formulate a new plan to achieve better results (the Rescheduling stage). After repeating this process multiple times as shown in Fig. I, they eventually achieve satisfactory result and complete the task. Thus, we abstract the human process of using tools for IR into five stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. In this work, we propose a system that operates in this manner to tackle tasks in complex IR scenarios.
Constructing such system is challenging because each of the aforementioned stages requires non- trivial capabilities. We summarize these capabilities follows:
Ability to Analyze Image Quality: The system needs to understand and assess the quality of images generated at each stage of the processing pipeline. This involves not just outputting quality score but also understanding the condition of the image quality and identifying the causes of degradation and quality issues. This capability forms the foundation for the perception and reflection stages and provides the necessary information for the (re (re)scuuuuling stage.
Ability to Reason Based on Context: The system involves many conditional judgments with complex and ambiguous contexts. Rule-based systems struggle to perform such complex and dynamic reasoning. This ability is crucial for connecting the various stages within the system.
Knowledge and Experience in IR: The system needs to propose a model usage plan based on the condition of the image, which requires the model to have prior knowledge of the behavior of IR models and experience in using them. The system even needs to know the impact of various IR models on each other when they are used in combination.
Fortunately, large language models (LLMs) (OpenAI, 2023a; Touvron et al., 2023) and vision- language models (VL.M) (OpenAI, 2023b; Yin et al., 2023; Liu et al., 2023) offer solutions to these challenges. VLM-based image quality analysis methods provide excellent tools for recog- nizing complex image degradations, quality assessment, quality comparison, and reasoning about image quality (Wu et al., 2024a;b;e; You et al., 2024b;a). LLMs can perform reasoning and judg- ment involving complex contexts through text generation. Moreover, advanced pre-trained language models inherently contain foundational knowledge about image restoration, and it is also possible to inject additional prior information into them. The combined and interactive use of LLMs and VLMs can build the IR intelligent agent system we envision.
In this work, we first construct a "toolbox" comprising several IR models and synthetic complex mixed degradation scenarios to emulate real-world problems, serving as the foundational platform for our research. Building upon this, we propose AgenticIR, system composed of LLMs and VLMs that interact and reason through text generation to dynamically operate the tools within the "toolbox" to solve complex IR tasks. To enable AgentielR to analyze image quality we extend a VL.M-based image quality assessment method named DepictQA (You et al., 2024a) by fine-tuning it to meet our complex requirement. We also utilize advanced LLMSs for reasoning, guiding the system to solve complex problems following the process described in Fig. 1. Although advanced LLMs typically contain some foundational knowledge of IR, they lack understanding of the complex behaviors of IR models and the intricate interactions between models. Therefore, we design self-exploration and experience summarization method, allowing the LLM to observe and summarize large number of restoration results, organizing the necessary experiential knowledge into referenceable documents. During the decision-making process, AgenticIR retrieves relevant information and knowledge as concrete ground to make informed decisions.
demand,
Our experiments demonstrate the potential of AgenticIR solving real-world problems. Although research was primarily conducted in laboratory environment, we anticipate that this paradigm holds significant promise for applications in automated and intelligent image processing. In practical scenarios, the agents' operations not limited to individual models; they may involve manipulating different parameters of complex large-scale models (Yu et al., 2024), as well as handling highly dynamic tasks that require judgment and action. We hope that our work can serve stepping stone to inspire research toward truly general and intelligent visual processing Al systems.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Image Restoration (IR) aims reconstruct high-quality images from their degraded counterparts. Over the years, this field has witnessed lots of successful models for single-degradation restoration, e.g. denoising (Chen et al.. 2023a), deblurring (Nah et al.. 2017; Chen et al., 2023c), deraining (Fu et al., 2017; Gu et al., 2023) and super-resooe repeerresolltion (SR) (Dong et al., 2016; Ledig et al., 2017; Tao al., 2024). However, these models' effectiveness limited a narrow range as they just focus on specific degradation. There are also efforts devoted to unifying multiple IR tasks (Liu et al., 2024; Chen et al., 2024d). Most of them either mix different degradations into training data (Zhang al., 2021; Valanarasu et al., 2022; Yu et al., 2024), or predict the degradation type assist restoration (Li et al., 2022; Luo et al., 2024; Zhang et al., 2023a; Gu et al., 2019; Zhang et al., 2023b; Chen et al., 2024b). Despite remarkable achievements, they may suffer from limitations including optimization difficulty. parameter inefficiency, and poor extensibility. Besides, Yu et al. (2018) and Chen et al. (2024a) invoke multiple legeedderraattion restoration tools to address multiple degradations and thus are somewhat similar to our work. But Yu et al. (2018) employ reinforcement learning (RL) while Chen et al. (2024a) fine-tune VLM to directly give execution plan, not agentic as ours. Besides, their preoccupation is not combining existing tools as our work.
LLMS and VLMs have been the forefront of advancing general intelligence. Despite being trained on extensive text datasets, their remarkable problem-solving capabilities extend far beyond typical language processing tasks (OpenAI, 2023a;b; Touvron al., 2023; Liu et al., 2023). LLMs are now capable of handling complex challenges once thought to require human expertise or spe- cialized algorithms, such as mathematical reasoning (Ahn et al., 2024), code generation (Jiang et al., 2024a), and addressing complex legal queries (Cui et al., 2024). Recent research also suggests that LLMs can generate intricate plans for robotics (Driess et al., 2023) and game AI (Zhu et al., 2023), representing significant step toward their role as general intelligent agents.
Agents can be described as systems interacting with environments to solve complicated problems under their agendas (Franklin & Graesser, 1997; Li et al., 2024; Xi et al., 2023; Sumers al., 2024). Early exploration mainly focuses on symbolic architectures (Franklin & Graesser, 1997) reminis- cent of mooleddee-bssed expert systems, rich in expressiveness but poor in flexibility. In the past decade, RL agents have gained success in many tasks (Silver et al., 2018 8; Hwangbo et al., 2019), but they lack explicitly maintained long-term plans, just acting step by step. L.MMs bring new opportu- nities to this field. With extensive general knowledge. LLMs can answer simple questions logically and flexibly (OpenAI, 2023a). Such responses can function as intermediate steps in a compound problem-solving system. Yao et al. (2023) compare LLMs' response to humans' automatic thinking mode "Sysstem 1", while the compound system corresponds the deliberate thinking mode "Sys- tem 2" (Kahneman, 1). Along this trend, many works contextualize LLMs within compound systems to construct language agents. To adapt them to specific tasks and fully harness LLMs' po- tential, these works draw inspiration from how humans solve problems and intuitively design various mechanisms ( (Wang et al., 2024). For instance, Tree of Thoughts (Yao et al., 2023) and Graph of Thoughts (Besta et al., 2024) prompt the LLM to provide heuristics for search; HuggingGPT (Shen et al., 2023) and Visual ChatGPT (Wu et al., 2023a) invokes various external tools for diverse tasks; Ghost in the Minecraft (Zhu al., 2023) and Generative Agents (Park et al., 2023) retrieves from memory to help decision-making Reflexion (Shinn et al., 2023) reflects on errors in exploration to accumulate experience. As well foundation models, such compound agent systems integrated with LLMs are widely deemed promising for artificial general intelligence (Zaharia et al., 2024).
3 METHOD
We first outline the abstraction of complex IR problems and the design of our research platform (Sec. 3.) We establish a simulated playground where our AgenticIR can experiment and demon- strate its capabilities. Next, we provide overview of the workflow design for our proposed method (Sec. 3.2). Finally, we introduce targeted enhancements and designs to equip our system with the specific abilities and knowledge required for effective performance (Sec. 3.3).
3.1 RESEARCH PLATFORM DESIGN
We abstract real-world IR scenarios into set of mixed scenes with well-defined degradations (Kong et al., 2024b; Wang et al., 2021; Kong et al., 2022). While this approach may not cover all possible scenarios, it provides a simple yet general and feasible playground for our research. Specifically,
3

---

Published as a conference paper at ICLR 2025
(a) Entree workflow
(b) Workflow in the view of tree search
Perception
Schecuuing
Plan: Denairing
Degradations: swwresslution
Dersining
5R
Denobing
Denoising
SR
Denolding Denaning Denalding Deraiinng
Rescheciuling
Denotiing
Danohing Dersiting
Deraining
Execution X
Reflection
Adjusted plant Danclsing
Execution detalls
Stepp 1: Deroining
Derairing
Rallback
c-1 Falled
Reflection
feib!
Daraining
SR
Toat
Tool 1
Reflection
3
Tool
Execution
K
Execution X
Stop
Denaising sucocedsl
Step Deraining
Devaiting
Reflection
Denatsing
sucoseds!
Tool Ni
c-2 Successful
Reflection
S Execution K
Toot
Reflection
Step
SSR
Toe
TTool
SR
succeedb
Tool
Figure 2: An example illustrating the framework of our AgenticIR. (a) presents the entire workflow, where bubble frames beside robots represent responses from LLMs and VLMs, and the numbers in circles correspond to those in (b). (b) points the tree search nature of the system. (c) expounds how to execute single-degradation restoration operation with toolbox.
Input image with baze Result of denoising noise
Result of first denois- ing then dehazing
Input image with rain Resalt of deraining and now-resolution
Result of first derain. ing and then SR
High-quality image
Result first dehaz- ing and then denoising Dehazing after denoising works dehazing first fails.
Resalt of dehazing
High-quality image
of then deraining SR fails.
(b) Deraining first works deraining
Figure 3: The importance of operation order in image restoration.
select eight types of degradation: low resolution, noise, motion blur, defocus blur, rain, haze, JPEG compression artifacts, and low light. There are many specialized models for each type of degradation that be used tools. For each degradation, the "toolbox" that the intelligent agent can use. Although these models are designed for single- degradation tasks, our goal for the intelligent agent leverage them to tackle more complex IR problems. We further create complex restoration tasks of varying difficulty by combining different degradations (Kong et al., 2024b), allowing us to evaluate the agent's capability in solving complex problems and its generalization ability to unseen scenarios. The degraded data used for testing are introduced in the experimental section. More details can be found in the Appendix A.3 and A.4.
collect three six advanced models build
3.2 WORKFLOW DESIGN
Overall Workflow. As illustrated in Fig. 2(a), our agent restores images with complex degrada- tions through multi-stage process. The agent begins with the Perception stage, dynamically ana- lyzing the content and degradations of the input image using a multi- vision language model. This information is then utilized the subsequent Scheduling stage by the language model. In the Scheduling stage, the language model leverages the identified degradation information, its inherent commonsense knowledge about degradation restoration, and additional domain-specifie knowledge provided by us to formulate judgments and develop possible overall plan to restore the image. This plan consists of a sequence operations. Following this, in the Execution stage, the agent executes the first operation in the plan, shown in Fig. 2(c). The agent incrementally tries available tools in an attempt to achieve the goal of the plan. During this process, the Reflection mechanism evaluates whether tool has been successful and whether operation has achieved its intended purpose. In
4

---

Published as a conference paper at ICLR 2025
our approach, reflection is also implemented by a multi- model with degradation and quality awareness. If the goal is met, the agent proceeds to the next operation, repeating this cycle until all operations completed and the restoration is finalized; otherwise, rollback is triggered (introduced later). For more details, please refer to Appendix A.
The Importance of Scheduling. In our workflow, correct scheduling is crucial because the com- plexity of IR model behaviors means that the selection of models and their execution order can greatly impact the final outcome. In real-world scenarios, complex degradations are mixtures of multiple degradations, and the mutual influence between different models leads to fundamental dif- ferences in results based on model selection and execution sequence. Fig. illustrates two examples. In the first example, due to the simultaneous presence of haze and noise, if we directly use dehaz- ing model, the noise interferes with the dehazing model's s ability, resulting in failure. the second example, the original rain streaks can be removed by a deraining model, but after super-resolution processing. the rain streaks exceed the deraining model's processing capabilities. The main purpose of the Scheduling stage is to develop better IR strategies for the current task and all the phenomena described above need to be taken into account.
Rollback Mechanism. However, the plan may encounter issues, as the scheduling stage is not guaranteed to give the optimal strategy. If execution at any stage fails to meet its objective, we have the reflection mechanism to identify the failure. Inspired by human interaction with image processing software, we design a rollback mechanism where the agent returns to the previous stage, learns from the failure, and creates new plan (Rescheduling). The agent then re-enters the execution stage with the updated plan. This process essentially performs a depth-first tree search among all possible degradation plans, as shown in Fig. 2(b). Given the vast number of degradation possibilities and the complexity of the situations, it is impractical to exhaustively traverse all options in real-world applications. However, our method provides an efficient mechanism to find feasible restoration paths. The LLM's reasoning capabilities, its knowledge of IR. and the experiential information we supply all contribute to improving the efficiency of this search and recision-making process.
3.3 CAPABILITY ACQUISITION
Sec. 3.2 outlined the workflow of the proposed AgenticIR. To make this complex process both feasible and effective, the system requires several non-trivial capabilities. In the following, we describe these essential capabilities and how we equip the system to acquire them.
Ability to Analyze Image Quality. For an agentic system that processes images based on varying degradation conditions, is crucial to provide it with the ability to recognize, analyze, and assess im- age quality. While numerous models exist for image quality assessment, most simply output scores or similarities. These approaches are insufficient for two key reasons: (1) they fail to provide the necessary information for the subsequent scheduling stage; and (2) they cannot establish a threshold to judge the success of an execution, dation has been effectively removed. To address this limitation, we focus on a new form of image quality perception and assessment one based on VLMs (You et al., 2024b;a; Wu et al., 2024a;b;d). These models can describe both the content and quality issues of an image using natural language, offering critical information for the scheduling stage to develop restoration plan. Some VLMs can even perform reasoning and answer questions, enabling accurate judgment of the success or failure of specific executions. Our system incorporates VLM-based image quality assessment model to acquire these advanced capabilities.
mere score does not indicate whether specific degra-
Specifically, we extend DepiciQA (You al., 2024a) a successful multi-modal image quality assessment model. DepictQA, trained on a large dataset, can analyze image content and categorize types of quality issues, evaluate the quality of single images, and compare the quality of multiple images. It already possesses substantial knowledge and capabilities in image quality assessment. We further adapt DepiciQA to the specific requirements of our system through LoRA fine-tuning (Hu et al., 2022). We additionally train the model to evaluate the severity of various degradations in images. With this extension, the DepicIQA method can evaluate each plan exccution effectively. Implementation details are described in Appendix A.2.
Ability to Reason Based on Context. To effectively connect the various stages within the agent system especially during the scheduling and rescheduling stages the agent must have thor- ough understanding of the tasks and workflow. In such a dynamic and complex system, making judgments based on predefined rules is nearly impractical. Moreover, the increasing number of pos-
5

---

Published as a conference paper at ICLR 2025
How schedule deniiiing and deheding?
Iooue
Debkur teel
VLN evaluste
LLM summancc
from the guidelinng ez isues reiate image enhancement should astressed after
tod
Successful rates:
Blls oftan belter move entiacts frem compressinn before improv the images content
Retrived
Oubter
deblur than de P 72% Filst-oe PEC debiur 90%
noise Can be considered artiict and de de-hacing image co contant by removing
DepPFi
Ball
Exploration
ttatistics (Experience)
Documents (Knowledge)
Referring to document inference
Figure 4: LLMS alone fail to grasp the intricate interactions among operations and thus cannot plan reliably. To address it, we let the agent self- self-explore beforehand and then summarize the accumulated experience distill knowledge. The knowledge will be a concrete ground for planning inference.
Table 1: Prompting GPT-4 (OpenAI, 2023a) to plan.
ID Prompt
Response
we will conduct these tasks: (*mo= tion dablurring", , "Jpag compression artifact removal'). Please provide some insights into the correct order these unordered tasks.
Not deblurring should ideally be performed first because it works beat on images that aave not been subjected to additienel processing ...
we will conduct these taska: ('ipeg compression artifact removal', 'motion deblurring' 1. Flease provide seme in- sights into the correct order of these unordered Laske.
deilurring itself may exaceriaste the appearance compreasion art facta, as it attempts to sharpen the image and restore detail ...
we will conduct these Laekat ['mo- tion deaturring', "ipeg compreastin artifact removal') ... de have the tol- lowing experience ... please give the correct order of the Leaks.
the experience suggests that it is often better to remove art facta before improving the image's content or quality we should renove JPRG art racts before addressing the motion blur.
sible states and heightened complexity make rule-based approaches exceedingly convoluted, and many of the judgment conditions are ambiguous. This necessitates that the agent system possesses considerable degree of intelligent "thinking" and "reasoning" capabilities. We utilize advanced LLMs to perform this reasoning, serving as bridge that links the different stages of the system and facilitates thought processes. Trained on vast amounts of human language data, LLMs can mimic human reasoning through text generation. The reasoning abilities of LLMs have been recognized across various tasks (OpenAI, 2023a: Wei et al.. 2022; Zhou et al., 2023). In our work, we employ GPT-4 (OpenAI, 2023a) for this reasoning because it possesses excellent comprehensive abilities and knowledge that can be conveniently accessed. is important note that other language models with sufficiently strong capabilities can also be used to construct this agent system.
Knowledge and Experience in IR. As previously discussed, the execution order of IR models significantly impacts the final results. We rely on the capabilities of LLMs to formulate model usage plans based on image conditions. While LLMs are extensively trained and possess some knowledge of IR, this foundational knowledge alone is insufficient to master the usage experience of the diverse tools in our toolbox. For example, in Cases 1 and 2 of Tab. 1. we prompt GPT-4 (OpenAI, 2023a) to arrange the sequence of motion deblurring and JPEG artifact removal. When we change the presentation order, which is meaningless, of these two degradations, GPT-4's answers also change. This illustrates that GPT-4 cannot reliably resolve these issues because it is merely speculating. This echoes the viewpoint of Kambhampati et al. (2024): LLMs possess only general planning knowledge but cannot truly handle subtask interactions, thus failing provide executable plans.
To address this limitation, we need to inform the LLM of experiential information about the usage of tools in the toolbox during reasoning, allowing it to base its reasoning on the prior knowledge we provide. To obtain comprehensive prior knowledge and minimize human subjective judgment, we propose method of self-exploration and summarization, as illustrated in Fig. 4. The agent first actively explores actual scenarios to accumulate experience and summarize this experience into referenceable documents for future reasoning. Specifically, for some images with complex degradations, we apply all possible sequences of the corresponding tools and use the fine-tuned DepictQA to evaluate the restored images, determining whether the restoration is successful. We calculate the success rate of each operation sequence. After collecting these statisties. GPT-4 to summarize them and store the results in a knowledge base, which will be retrieved during reasoning to assist in operation scheduling. For example, when we prompt GPT-4 again arrange
prompt

---

Published as a conference paper at ICLR 2025
Table 2: Degradation evaluation performance of fine-tuned DepiciQA.
A-
Degradation
Precision
Recall Fl score
wel exper lefice w/ experience
wib exper lericee experience
Noise
0.99
0,92
0,95
Motion blur
0.88
0.52
0.65
0.0
0.2
0.4
0.5
0.8
0%
10%
20%
30%
Defocus blur
0.82
0.65
0.72
Entropy
Variation
JPEG compression artifact
0.98
1,00
0.99
Figure Comparison between dispersion of scheduling results with and without experience. Lower metric indicates higher consistency.
Rain
0.97
0,98
0.98
Haze
0.88
0.91
0.89
Low light
0.87
0.65
0.74
Table 3: Quantitative comparison between act- Table 4: Quantitative comparison with the ran-
ing as the agent's plan and the opposite. "Not dom tool invocation. The better performances as planned" means randomly shuffling the plan are marked in bold. means the lower the bet- ter, and for others, the higher the better.
(guaranteed to be different).
Degradatios As planned PSNR SSIM EPIPS] MANIQA
CLIP-IQA MUSIQ
Degradatoon Method PSNR
MANIOA CLIP 40A SIQ
Group A
20.70 0.6652
21,14 0.6836 0.2753
0,3469
0.3585
0,5091
60.77
Group
AgeaticlR 21.04 0.8188
0368
0.3071
0.2963
56.3
Group
20.32 0.8111 0.2976
0.2683
0.3623
0.5257
Group
AgenticlR 20.55
0.3204
0.3120
Group
18.78
0.5277 0.4345 0.3058
0.3118
0.4719
51.32
Group
AgenticlR 18882
Random 18.87
0.4796
0.2698
0.2354
0.3543
the order of motion deblurring and JPEG artifact removal using this experiential knowledge, it can find references in the experience and consistently provide answers like Response 3 in Tab. 1. In this way, GPT-4 can offer reliable heuristics for the execution order of operations.
EXPERIMENTS
As described in Sec. 3.1, we construct our research data using mixed degradations. In the experi- mental section, we first build the test dataset. We designed 16 combinations of mixed degradations involving 2 or types of degradation and divided them into three groups: A, B, and C. Group A contains 8 combinations, while groups B and C each contain 4 combinations. The degradation combinations in groups A and B consist 2 degradations, whereas those in group C consist of degradations to simulate more complex situations. During the exploration phase, the agent is ex- posed only to group A that is, the agent is familiar with the degradations present in group A but unaware of those in groups B and C. This setup helps us investigate the system's generalization ability. We applied each of the 16 degradation combinations to every one of the 100 images in the Mi0100 (Kong al., 2024a:b) dataset. For each combination in group A, we allocated 20 images for exploration, totaling 160 images. The remaining 1,440 images are used for testing. More detailed information can be found in Appendix A.3.
4.1 EFFECTIVENESS OF INDIVIDUAL DESIGNS
Fine-Tuning Tuning DepictQA for Image Quality Analysis. For each image in the test set, we prompted the fine-tuned DepictQA to assess the severity of each degradation. By treating degradation assess- ment as binary classification problem-thee -hat is, determining whether an image suffers from a partic- ular degradation-ww we computed the precision, recall, and Fl scores, as shown in Tab. 2. Considering that we did not invest effort in designing special strategies and that the fine-tuning consumed only moderate computational resources (three hours on four V100 GPUs), the learning process proved to be quite efficient. The fine-tuned DepictQA effectively recognized almost all types of degradation, except that the recall rates for defocus blur and motion blur were somewhat limited. This limitation is understandable, as these two types of blur are very similar in many cases; even humans find it difficult fully distinguish between them.
Self- Self-Exploratee Exploration and Experience Summarization. Informed of successful rates of each opera- tion order for each degradation combination in group A, GPT-4 tries to summarize them and distill insights. Fig, 9 shows snippets of the distilled insights and how they help operation scheduling. It can be seen that GPT-4 does conclude general rules that seem reasonable and apply them to infer the order of operations logically. The scheduling of "deraining, super-resolution' and "denoising, dehazing" also echo our discussion about Fig. 3 in Sec. 3.3. The example in Fig. 6 shows the correct- ness of the plans derived from experience: without experience, the agent decides to deblur first, only

---

Published as a conference paper at ICLR 2025
Input
Defocus debluming JPEG artifact rumoval
Ieput
JPEG artifect removil
Defocus debluriing
(e) without experience
(b) with experience
Figure An example showing the correctness scheduling with experience.
DenoLaang
Beroizing
Braghtening
teol
Denobing
Brightering
Denoising
Brightendin
tool 3
tnal
Brightoning
(a) w/o reflection
(b) w/ reflection
Figure Exemplary comparison between with and without reflection.
Deblur
Bestar
Dalaaus
cetruming
Se
beblur
Dethocs
ceeS
daskuring
cast
(a) w/o rollback
(b) w/ rollback
Figure 8: Exemplary comparison between with and without rollback.
distort the JPEG artifact and render it hard remove; scheduling with experience can avoid this. To quantitatively evaluate, we let the agent act as the plans to restore images in the test set; as the control group, we conduct another experiment wherein the agent does not as the plans. Six image quality assessment metrics are used for evaluation: three full-reference metrics PSNR, SSIM (Wang et al., 2004), LPIPS (Zhang et al., 2018), and three non-reference metrics MANIQA (Yang et al., 2022), CLIP-IQA (Wang et al., 2023), MUSIQ (Ke al., 2021). Tab. lists the results, which indicate the efficacy of the proposed method. For more details, refer to Appendix B. and B.2.
We also investigate the consistency improvement to justify the motivation introduced in Sec. 3.3: for a set of operations, GPT-4 fails to consistently give a scheduling result, especially sensitive to the presentation order of operations; providing experience should alleviate this problem. We prompt GPT-4 to schedule each set 60 times with random presentation order, and measure the dispersion of the results. Two metrics are adopted: (I) treating the results discrete distribution over all per- mutations of operations, entropy reflects the overall dispersion; (2) variation ratio is the proportion of samples that are not mode, straightforwardly reflecting confidence in the dominant result. Fig. compares the dispersion of scheduling results of the 16 operation sets with and without experience averaged in groups. fact, for some operation sets, the experience improves the consistency from almost random to deterministic (refer to Appendix B.3 for details). These results support our claim.
mechanisms in the inference workflow ablate: reflection, ie., whether to check the tool results, and rollback, i.e., whether to roll back failed operations. We compare the performances with and without the two mechanisms respectively! and the quantitative results² are listed in Tab. 5. We can see the absence of any mechanism leads to a performance drop most metries. Qualitative comparisons are shown in Fig. and Fig. 8. In Fig. 7, without reflection, although the plans are the same, the agent fails to select the appropriate tool, resulting in remnant noise. In Fig. 8, without rollback, the agent fails correct the suboptimal operation order
Inference Workflow. There
Reflection prerequisite rollhack, so rollhack is disabled when ablating reflection.
When ablating rollback, to highlight the impact, the statisties are only calculated on cases wherein rollback triggered (20% cases).
8

---

Published as a conference paper at ICLR 2025
Insights distilled from experience
From these observations, we can infer that generally It is often better to remove artifacts (rain, J PEG compression) before improving the image's content or quality...
Guide: How to schedule ...
Deraining, SR (in group A) According to the collected experience is more ef- fective address the arti- fact caused by rain before enhancing the image's solution...
Denoising, dehazing (in group B)
B Brightening, defocus deblurring, PEG artifact removal (in group C)
from the guidelines ... issues related to image content enhancement should be addressed after dea ling with artifacts noise can be considered an artifact and dehazing related to enhancing image content by removing haze
more effective to address blurring issues before image content enhance- ment removing artifacts like PEG compression should be done before addressing blurring issues
Figure Examples of GPT-4's responses for operation scheduling.
Table 5: Ablation studies of the inference workflow, including reflection (Ref.) and rollback (Rb.). (b) Ablation study of rollback.
(a) Ablation study of reflection.
Degradations Ref. Rb.
Method PSNR SSIM LPIPS) MANIQA CLIP-IQA MUSIQ
Degradations Ref. Ra.
Method PSNR SSIM LPIPS, MANIQA CLIP LLIP-QQAII MUSIQ
Group 20:47 0.6659 0.3282
21.12 0.6809 0.3079
0.3179
0.2906
0,4617
57.52
55.56
Group A
20.23 0.6626 0.3249
19.77
0.3067
0.3197
0.3042
0.4484
0.4158
59.87
58.70
Group B
/ X 20.74 0.6986 0,3084
X x 20. 0.6798 0,3412
0.2966
0.3126
0.4567
0,4359
56.66
54.81
Group B
/ V
18,76 0.6642 0.3348
18. 0.6348
0,3591
0.3251
0,3082
0.4528
0.4525
$7.43
55.94
Group /
x x 18.93 0.5447 0.1764 0,23
0.5510
0,3595
43.77
47.38
Group C
18.99 0,4604
18.64 0.5446 0.4634
0.2643
0.2348
0.3974
0.3669
46.73
suggested by the experience: although defocus deblurring before SR makes sense, the defocus blur in this case is slight enough to be addressed by SR tools", but first conducting defocus deblurring over-smoothens the image. These examples show that the mechanisms are necessary in various scenarios, working together to enable the agent to find the proper operation order and tools.
4.2 COMPARISON WITH OTHER METHODS
We test AgenticIR on our test set for the complex-degradation restoration task. To knowledge, there is no open-source work for IR tool ensemble yet, so we de- sign a simple method compare: with the degradations predicted by our fine- tuned DepictQA, randomly selecting one tool for each degradation and invoking them random order. Tab. 4 lists the quantitative results. AgenticIR outper- forms this method in almost all metrics.
Table 6: Quantitative comparison with all-in-one mod- els. The best and second best performances are marked in bold and underline respectively.
Degradations Method PSNR SSIM LPIPS) MANIQA CLIP-IQA MUSIQ
AirNet 19.13 0.6019 0.4283
0.2581
0.3930
42.46
PromptlR
20.06
0.6088 0.4127
0.2633
0.4013
42.62
MiOR
20.84 0.6558 0.3715
0.2451
0.3933
47,82
Group
DA
19.58
0.60322 0.4266
0.2418
0.4139
42.51
InstrucUR
18.03
0.5751
0,2660
0,3528
45.77
AutoDIR
19.64
0.6286 0.3967
0,2500
0.3767
47.01
AgenticIR
21.04 0,6818 0,3148
0.3071
0.4474
56.88
AirNet 19.31 0.6567 0.3670
0.2882
0,4214
47,55 88
PrompilR
20.47 0.6704 0.3370
0,2893
0.4289
48.
MiOIR
20.56 0.6905 0.3245
0.2638
0,4330
$1.87
As for the IR performance, we com- pare AgenticIR with several all-in-one models: AirNet (Li et al., 2022), PromptIR (Potlapalli et al., 2023), MiOIR (Kong
Group
DA-CLIP
18.56 0.5946 0.4405
0.2435
0,154
43.20
InstructR
18.34 0.6235 0.4072
0,3022
0,3790
50.94
AutoDRR
19.90 0.6643
0,2534
0,3986
69.64
AgcuticIR
20.55
0.7009
0,3072
0.3204
0.4648
57.57
AirNet
17.95
0.5145 0,5782
0,1854
0,3113
30.12
al., 2024a). DA- CLIP (Luo et al., 2024), Instruc- IR (Conde et al., 2024), and Au- toDIR (Jiang et al., 2024b). Tab. 6 lists the quantitative comparison. AgenticIR wins in almost all metrics. Note that by this comparison we do not intend to argue the superiority of AgentielR over all-in-one models. Instead, we hope to verify that combining single-degradation IR tools can achieve comparable performance with all-in-one models, so this approach is feasible and meaningful. After all, the intrinsic purpose of this paper is not IR but IR tool ensemble.
PrompiIR
18.51 0.5166 0.5756
0.1906
0.3104
29.71
MiOIR
15.63
0.4896 0.5376
0.1717
0,2891
37.95
Group
DA-C
18.53
0.1916
0.3476
33.87
InstructR
17.09 0.5135 0.5582
0.1732
0.2537
33.69
AutoDIR
18.61 0.5443 0.5019
102055
0.2939
37.86
AgenticlR 18.82 0.5474 0.4493
0.2698
0.3948
48.68
Qualitative comparison with the baseline and all-in-one models is shown in Fig. 10. We can see AgenticlR effectively addresses all degradations and yields visually pleasing results. In contrast, random invocations may fail to find the appropriate operation order and tools, and all-in-one models may struggle to address all degradations due to their diverse and even conflicting characteristics.
'The training data of many SR models also include blur, as proposed by Wang et al. (2021).

---

Published as a conference paper at ICLR 2025
AirNer (20.27)
PrompdR (20.39)
MiOIR (22.27)
DA-CLIP (25.22)
Input image with min and low resolution (PSNR in dB)
InstruetRR (20.84)
AutoDIR (22.31)
Random (20.90)
AgenticR (26.45)
AirNet (2165)
PromptR (19.60)
MIOIR (19.60)
DA-CLIP (16.78)
D
Input image low Tight, defocus blur, and JPEG compression artifact (PSNR dB)
Instuei (20.54)
AutoDIR (17.14)
Random (19.90)
AgenticRR (24.53)
Figure 10: Qualitative comparison with other methods.
Figure 11:
Examples of AgenticIR restoring real-world complexly degraded images. First two images: taken by under-display camera, restored by motion deblurring, defocus deblurring, and brightening; middle images: taken underwater, restored by defocus deblurring, dehazing, and motion deblurring; last two images: taken in heavy rain, restored by deraining and dehazing.
Tnput
Restormer (Demining) DehnzeFormer (Dehazing)
AntoDIR
AgenticIR
Figure 12: Comparison with individual models on real-world image.
Real-Worke Applications. We also verify AgenticlR's s ability on real-world complexly degraded images. Fig. 11 shows some examples: the first one is from under-display camera dataset (Zhou et al., 2021), ), and AgentieIR restores it by motion deblurring, defocus deblurring, and brightening; the second one is from an underwater dataset (Islam al., 2020), and AgenticIR restores by de- focus deblurring, dehazing, and motion deblurring (similar decomposition is also observed by Jiang al. (2024b)); the last one is an image with heavy rain downloaded from the Internet, and Agen- ticIR restores it by deraining and dehazing. These examples illustrated that AgenticIR can really restore some real-world low-quality images by decomposing the complex-degradation IR task into several tractable single-degradation operations, verifying the application value of AgenticIR. Fig. 12 compares AgenticIR with individual models. In this example, the deraining model (Restormer (Za- mir et al., 2022)), dehazing model (DehazeFormer (Song et al., 2023)). and all-in-one model (Au- toDIR (Jiang et al., 2024b)) all behave poorly, while AgenticIR, sequentially conducting deraining by Restormer and dehazing by DehazeFormer, manages yield clean image. This illustrates that AgenticIR as compound system does have an advantage over individual models on some occasions.
5 CONCLUSION
In conclusion, we introduce AgenticIR, system that combines LLMSs and VLMs to emulate human- like strategies in complex IR tasks by dynamically utilizing a toolbox of IR models through the stages of perception, scheduling, execution, reflection, and rescheduling. Our experiments demon- strate that AgenticIR effectively tackles complex IR problems beyond the capability of individual models, showing significant potential for applications in automated and intelligent image process- ing. We hope this work can lay a promising foundation for future research toward truly general and intelligent visual processing AI systems.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work is supported by the National Key R&D Program of China (No. 20222D016010), the National Natural Science Foundation of China (Grant No. 62276251), and the Joint Lab of CAS- HK.
REFERENCES
Eirikur Agustsson and Radu Timofte. NTIRE 2017 challenge on single image super-resolution: Dataset and study. In CVPRW, 2017. 17
Janice Ahn, Rishu Verma, Renze Lou, Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157, 2024.
Oded Berger-Tal, Jonathan Nathan, Ehud Meron, and David Saltz. The exploration-explottation dilemma: multidisciplinary framework. PloS one, 9()):955633, 2014. 25
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of Thoughts: Solving Elaborate Problems with Large Language Models. AAAI, 2024. 3
Haoyu Chen, Jinjin Gu, Yihao Liu, Salma Abdel Magid, Chao Dong, Qiong Wang, Hanspeter Plister, and Lei Zhu. Masked image training for generalizable deep image denoising. In CVPR, June 2023a.
Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Sixiang Chen, Tian Ye, Renjing Pei, Kaiwen Zhou, Fenglong Song, and Lei Zhu. RestoreAgent: Autonomous image restoration agent via multimodal large language models. arXiv preprint arXiv:240..18035, 2024a.
Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Haoze Sun, Xueyi Zou, Zhensong Zhang, Youliang Yan, and Lei Zhu. Low-res leads the way: Improving generalization for super-resolution by self- supervised learning. In CVPR, pp. 25857-25867, 2024b. 3
Xiangyu Chen, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong. Activating more pixels in image super-resolution transformer. In CVPR, pp. 22367-22377 June 2023b. 20
Xiangyu Chen, Zheyuan Li, Yuandong Pu, Yihao Liu, Jiantao Zhou, Yu Qiao, and Chao Dong. A comparative study of image restoration networks for general backbone network design. In ECCV, 2024c. 20
Xiangyu Chen, Yihao Liu, Yuandong Pu, Wenlong Zhang, Jiantao Zhou, Yu Qiao, and Chao Dong- Learning low-level vision generalist via visual task prompt. In ACM MM, 2024d. 3
Zheng Chen, Yulun Zhang, Liu Ding, Xia Bin, Jinjin Gu, Linghe Kong, and Xin Yuan. Hicrarchical integration diffusion model for realistic image deblurring. In NeurIPS, 2023c. 3
Marcos V Conde, Gregor Geigle, and Radu Timofte. InstructIR: High-quality image restoration
following human instructions. In ECCV, 2024. 9
Jiaxi Cui, Munan Ning, Zongjian Li, Bohua Chen. Yang Yan, Hao Li, Bin Ling, Yonghong Tian, and Li Yuan. Chatlaw: A multi-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experrs large language model. arXiv preprini arXiv:2306. 16092, 2024. 3
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE TPAMI, 2016. 3, 18
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Mare Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. PaLM-E: An embodied multimodal language model. arXiv preprint XxXiv:2303.03378. 2023.

---

Published as a conference paper at ICLR 2025
Stan Franklin and Art Graesser. Is it an agent, or just program?: A taxonomy for autonomous agents. In Intelligent Agents III Agent Theories, Architectures, and Languages, 1997. 3
Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. Clearing the skies: A
deep network architecture for single-image rain removal. IEEE TIP, 26(6):244-22566, 2017.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurIPS, 2014. 20
Jinjin Gu, Hanman Lu, Wangmeng Zuo, and Chao Dong, Blind super-resolution with iterative kernel correction. In CVPR, pp.
2019.
Jinjin Gu, Xianzheng Ma. Xiangtao Kong, Yu Qiao, and Chao Dong. Networks are slacking off: Understanding generalization problem in image deraining. In NeurIPS, 2023,
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. In CVPR, 2009. 18
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 5
Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics, 2019.
Md Jahidul Islam, Youya Xia, and Junaed Sattar. Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters, 2020.
Jiaxi Jiang, Kai Zhang, and Radu Timofte. Towards flexible blind JPEG artifacts removal. In ICCV, October 2021. 20
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint (XXi:2446.000155 2024a.
Yitong Jiang. Zhaoyang Zhang, Tianfan Xue, and Jinwei Gu. AutoDIR: Automatic all-in-one image restoration with latent diffusion. In ECCV, 2024b. 10
Daniel Kahneman. Thinking, fast and slow. Farrar, Straus, and Giroux, 2011. 3
Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. Position: LLMs can't 't plan, but can help planning in LLM-modulo frameworks. In ICML, 2024. 6
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. MUSIQ: Multi-scale image quality transformer. In ICCV, Octoher 2021. 8
Xiangtao Kong, Xina Liu, Jinjin Gu Yu Qiao, and Chao Dong. Reflash dropout in image super- resolution. In CVPR, pp. 6002-6012. 2022. 3
Xiangtao Kong, Chao Dong, and Lei Zhang. Towards effective multiple-in-one image restoration:
A sequential and prompt learning strategy. arXiv preprint arXiv:2401.03377, 2024a. 7, 9,
Xiangtao Kong, Jinjin Gu, Yihao Liu, Wenlong Zhang, Xiangyu Chen, Yu Qiao, and Chao Dong. A preliminary exploration towards general image restoration. arXiv preprint arXiv:2408. 15143, 2024b. 4,7
Christian Ledig, Lucas Theis, Ferene Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo- realistic single image super-resolution using a generative adversarial network. In CVPR, July 2017. 3
Junyong Lee, Hyeongseok Son, Jaesung Rim, Sunghyun Cho, and Seungyong Lec. Iterative filter adaptive network for single image defocus deblurring. In CVPR, pp. 2042, June 2021. 20
12

---

Published as a conference paper at ICLR 2025
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. Benchmarking imagle-mmae dehazing and beyond. IEEE TIP, 2019. 18
Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restora- tion for unknown corruption. CVPR, June 2022.
Chunyuan Li, Zhe Gan, Zhengyuan Yang. Jianwei Yang, Linjie Li, Lijuan Wang. and Jianfeng Gao. Multimodal foundation models: From specialists to general-purposs assistants. Foundations and Trends in Computer Graphics and Vision, May 2024. 3
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. SwinIR: Image restoration using swin transformer. In ICCVW, October 2021. 18, 20
Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Wanli Ouyang, Yu Qiao, and Chao Dong. DitIBIR: Towards blind image restoration with generative diffusion prior. In ECCV, 2024. 20
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023. 2,
Yihao Liu, Xiangyu Chen, Xianzheng Ma, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong. Unifying image processing visual prompting question answering. In ICML, 2024. 3
Llama Team, Al @ Meta. The Llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024. 25
Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B. Schön. Controlling vision-language models for multi-task image restoration. In ICLR, 2024. 3,9
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexan- der S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustmess object detec- tion: Autonomous driving when winter is coming. arXiv preprint arXiv: pet.i/907707888 2019. 18
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, July 2017. 3
OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023a 3, 6, 17
OpenAL. (is-oV( system card, 2023b. URL https::/cdn. GPTV_System_ Cat. 2,
wwm cow/papers/
Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. Generative agents: Interactive simulacra of human behavior. In Annual ACM Symposium on User Interface Software and Technology. 2023.
Vaishnav Potlapalli, Syed Waqas Zamir, Salman H Khan, and Fahad Shahbaz Khan. PromptIR: Prompting for all-in-one image restoration. In NeurlPS, 2023. 9
Lingyan Ruan, Bin Chen, Jizhou Li, and Miuling Lam. Learning deblur using light field generated and real defocus images. In CVPR, June 2022. 20
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hug- gingGPT: Solving Al tasks with ChatGPT and its friends in Hugging Face. In NewrIPS, 2023.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In NeurlPS, 2023. 3
David Silver. Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap. Karen Si- monyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 2018. 3
Yuda Song, Zhuqing He, Hui Qian, and Xin Du. Vision transformers for single image dehazing. IEEE TIP, 2023. 10, 20
13

---

Published as a conference paper at ICLR 2025
Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas Griffiths. Cognitive architectures for language agents. Transactions on Machine Learning Research, 2024.
Keda Tao, Jinjin Gu, Yulun Zhang, Xiucheng Wang, and Nan Cheng. Overcoming false illu- sions in real-world face restoration with multi-modal guided diffusion model. arXiv preprint arXiv:2410.04 2024. 3
Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming-Hsuan Yang, Lei Zhang, Bee Lim, Sanghyun
Son, Heewon Kim, Seungjun Nah, Kyoung Mu Lee, Xintao Wang. Yapeng Tian, Ke Yu, Yulun Zhang. Shixiang Wu, Chao Dong, Liang Lin, Yu Qiao, Chen Change Loy, Woong Bae, Jaejun Yoo, Yoseob Han, Jong Chul Ye, Jae- see-Seok Choi, Munchurl Kim, Yuchen Fan, Jiahui Yu, Wei Han,
Ding Liu, Haichao Yu, Zhangyang Wang, Honghui Shi, Xinchao Wang. Thomas S. Huang. Yunjin Chen, Kai Zhang, Wangmeng Zuo, Zhimin Tang. Linkai Luo, Shaohui Li, Min Fu, Lei Cao. Wen Heng, Giang Bui, Truc Le, Ye Duan, Dacheng Tao, Ruxin Wang. Xu Lin, Jianxin Pang. Jinchang Xu, Yu Zhao, Xiangyu Xu, Jinshan Pan, Deqing Sun, Yujin Zhang, Xibin Song, Yuchao Dai, Xueying Qin, Xuan-Phung Huynh, Tiantong Guo, Hojjat Seyed Mousavi, Tiep Huu Vu, Vishal Monga, Cristovao Cruz, Karen Egiazarian, Vladimir Katkovnik, Rakesh Mehta, Arnav Kumar Jain, Abhinav Agarwalla, Ch V. Sai Praveen, Ruofan Zhou, Hongdiao Wen, Che Zhu, Zhiqiang Xia, Zhengtao Wang, and Qi Guo. NTIRE 2017 challenge on single image super-resolution: Methods and results. In CVPRW, July 2017. 17
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023. 2,
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. MAXIM: Multi-axis mlp for image processing. In CVPR, June 2022. 20
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M. Patel. Weather: Transformer-based restoration of images degraded by adverse weather conditions. CVPR, June 2022. 3
Jianyi Wang, Kelvin C.K. Chan, and Chen Change Loy. Exploring CLIP for assessing the look and feel of images. AAAI, June 2023. 8
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science. March 2024.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-ESRGAN: Training real-world blind
super-resolution with pure synthetic data. In ICCVW, October 2021. 3,
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 2004. 8
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. In British Machine Vision Conference, 2018. 18
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Fd Chi, Quoe V Le, and Denny Zhou. Chain-of-thought prompting clicits reasoning in large language models. In NeurIPS, 2022.
Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual ChatGPT: Talking, drawing and editing with visual foundation models, 2023a.
Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, and Weisi Lin. Q-Bench: A benchmark for general- purpose foundation models on low-level vision. ICLR, 2024a. 2,5
Haoning WU, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-Intruuc: Improving low-level visual abilities for multi malti-modality foundation models. In CVPR, June 2024b. 2,
14

---

Published as a conference paper at ICLR 2025
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtao Zhai, and Weisi Lin. Q-Align: Teaching LMMs for visual scoring via discrete text-defined levels. In ICML.
PMLR, 2024c.
Haoning Wu, Hanwei Zhu, Zicheng Zhang. Erli Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Annan Wang, Wenxiu Sun, Qiong Yan, Xiaohong Liu, Guangtao Zhai, Shiqi Wang, and Weisi Lin. Towards open-ended visual quality comparison. In ECCV, 2024d. 5
Rui-Qi Wu, Zheng-Peng Duan, Chun-Le Guo, Zhi Chai, and Chongyi Li. RIDCP: Revitalizing real
image dehazing via high-quality codebook priors. In CVPR, June 2023b. 20
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong. Ming Zhang, Jun- zhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng. Xiaoran Fan, Xiao Wang. Limao Xiong. Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuan- jing Huang, and Tao Gui. The rise and potential of large language model based agents: survey. arXiv preprint wrXiv:2309.07864 2023. 3
Sidi Yang, Tianhe Wu, Shuwei Shi. Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. MANIQA: Multi-dimension attention network for no-reference image quality as- sessment. In CVPRW, June 2022. 8
Shunyu Yao. Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In NeurlPS, 2023. 3
Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Lu Sheng, Lei Bai, Xiaoshui Huang, Zhiyong Wang. Jing Shao, and Wanli Ouyang. LAMM: Language-assisted multi-modal instruction-tuning dataset, framework, and benchmark. In NeurlPS, 2023. 2
Zhiyuan You, Jinjin Gu, Zheyuan Li, Xin Cai, Kaiwen Zhu, Chao Dong, and Tianfan Xue. De- scriptive image quality assessment in the wild. arXiv preprint arXiv:2405. 18842, 2024a. 2, 5, 17
Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting beyond scores: Advancing image quality assessment through multi-modal language models. In ECCV, 2024b. 5.5
Fanghua Yu, Jinjin Gu, Zheyuan Li, Jinfan Hu, Xiangtao Kong, Xintao Wang, Jingwen He, Yu Qiao, and Chao Dong. Scaling up to excellence: Practicing model scaling for photo-realistic image restoration in the wild. In CVPR, June 2024. 2,
Ke Yu. Chao Dong, Liang Lin, and Chen Change Loy. Crafting toolchain for image restoration by deep reinforcement learning. In CVPR, June 2018. 3
Matei Zaharia, Omar Khattab, Lingjiao Chen, Jared Quincy Davis, Heather Miller, Chris Potts, James Zou, Michael Carbin, Jonathan Frankle, Naveen Rao, and Ali Ghodsi. The shift from mod- els to compound AI systems, 2024. URL https://bbat wtw...beeeelleeee/llllooo.. 02/1 / compound- wws.tssssssss.............
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. In CVPR, June 2021. 20
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, June 2022. 10, 20
Jinghao Zhang, Jie Huang, Mingde Yao, Zizheng Yang. Hu Yu, Man Zhou, and Feng Zhao. Ingredient-oriented multi-degradation learning for image restoration. In CVPR, June 2023a. 3
Kai Zhang, Jingyun Liang, Lue Van Gool, and Radu Timofte. Designing a practical degradation model for deep blind image super-resolution. In ICCV, October 2021. 3
15

---

Published as a conference paper at ICLR 2025
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, June 2018.
Ruofan Zhang, Jinjin Gu, Haoyu Chen, Chao Dong, Yulun Zhang, and Wenming Yang. Crafting training degradation distribution for the accuracy-gennralization trade-off in real-world super- resolution. In ICML. PMLR, 2023b.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. -esrructionfoolloiing evaluation for large language models, 2023.
Yuqian Zhou, David Ren, Neil Emerton, Schoon Lim, and Timothy Large. Image restoration for under-display camera. In CVPR, June 2021. 10
Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang. Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang. Yu Qiao, Zhaoxiang Zhang, and Jifeng Dai. Ghost in the Minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. arXiv preprint arXiv:2305.17144 2023. 3
Karel Zuiderveld. Contrast limited adaptive histogram equalization. In Gruphics Gems, pp. 474 485. Academic Press, 1994. 20
16

---

Published as a conference paper at ICLR 2025
A MORE IMPLEMENTATION DETAILS
A.1 INFERENCE WORKFLOW
Imp-lmmlemeettd Functions. Algorithm
Algorithm 1: Inference workflow
and 2 describe the entire inference workflow of AgenticIR, where the functions EVALU-
Input: Low-quality image I
Output: Restored high-quality image
ATE, SCHEDULE, REFLECT, RESCHEDULE and PICKBEST are implemented by LLMs.
agenda <- EVALUATE(T):
plan SCHEDULE( agenda);
In the function EVALUATE, fine-tuned Depic- IQA (You et al., 2024a) evaluates the sever- ities of all types of degradation to recognize degradations present in the image; if DepiciQA thinks the severity is medium, high, or very high, then the degradation is deemed existent. In REFLECT, DepictQA evaluates the degra- dation corresponding to the single-degradation restoration subtask at hand judge whether the execution is successful. In SCHEDULE and RESCHEDULE, GPT-4 (OpenAI, 2023a) (in this paper we use the gpt-4-1106ppzess version) references accumulated experience to infer the order of given subtasks; specially, in RESCHEDULFF, GPT-4 is informed the previous failed attempts. In PICKBEST, the candidates are linearly scanned and compared in pairs by DepictQA, resulting in the highest quality can- didate. Prompts are listed in Tab. 7.
while plan not empty do
1, success +- DFS(L, plan);
success then
output T;
else
plan + the remaining plan for
output T;
A Algorithm 2: DFS
Input: Image list of subtasks plan
Output: Restored image, successful or not
if plan empty then
output true; attempts t 0:
inferiors + 0:
repeat
subtask the first subtask of plan;
i i- --rsuutt of subtask on I; pass REFLECTTI,, subtask); pass then
10
Remove sublask from plan: I, success + DFS plan); success then
Subtask Execution. To select the appropriate 11 tool for the subtask, we iteratively invoke and reflect. After invoking tool, if the fine-tuned DepictQA thinks the severity of the degradation very low, then we accept the tool result im- mediately; otherwise, we continue to try other tools. no tool gives very low degradation severity but some give low severity, then we run PICKBEST to select the best one of them as successful subtask result. If no tool gives low or very low severity, then the subtask is reported as failed; in this case, still run PICKB BEST to find the best one, which is useful all subtask orders from the input image fail.
12
13
output true;
14 Add subtask to attempts;
15 Add I to inferiors;
if size of allempls & size of plan then plan RESCHEDULE(plan, attempts);
is
else
+ (icxxEsT((nferiors);
output 1, false:
A.2 FINE-TUNING VIM
DIV2K (Agustsson & Timofte, 2017) and Flickr2K (Timofte et al., 2017) datasets are used for fine-tuning DepictQA (You et al., 2024a) on degradation evaluation. We randomly add two to four degradations on each image to obtain 15,000 complexly degraded images. For each of them, we iterate the seven degradations* to synthesize seven question-answer pairs: question: "What's the severity of degradation i in this image?"; answer: "very low / low / medium / high / very high". We fine-tune DepictQA for one epoch with batch size 64 on NVIDIA Tesla V100 GPUs, using learning rate 0.0005, weight decay 0.001, and Adam optimizer (8) 0.9, B2 0.95).
*The degradation "low resolution" is excluded since we can directly recognize it by checking the resolution.
17

---

Published as a conference paper at ICLR 2025
A.3 DATA
This section details how we synthesize degraded images. We first introduce how the eight degrada- tions implemented respectively and then list the combinations.
Single Degradation. For low resolution, following most SR works (Dong et al., 2016; Liang et al., 2021), we downsample the images by factor of 4 using bicubic interpolation. For noise, we add Gaussian or Poisson noise with random scale. For motion blur, following Michaelis et al. (2019), we filter images with linear kernels with random direction and radius. For defocus blur, following Michaelis et al. (2019), filter images with circular kernels with random radius. For rain, following Kong et al. (2024a), we first add noise and then filter the noise with linear kernels with random direction. For haze, following most dehazing works (He et al., 2009; Li et al., 2019), we adopt the atmospheric scattering model with random global atmospheric light and scattering coefficient. For JPEG compression artifact, we compress images with random quality factor. For low light, randomly reduce the V channel value of HSV color space with one of the following approaches: subtracting constant, Gamma correction, and linear mapping.
Degradation Combination. Tab. 8 lists the 16 degradation combinations for exploration and test- ing. Note that we consider real-world scenarios deliberately when designing the combinations. The combinations should be common in reality, e.g., haze and rain, dark and noise. Besides, the degra- dation order should conform to physical limitations, e.g., blur should added before noise since they follow chronological order in imaging.
A.4 TOOLS
Tab. 9 lists the single-degradation restoration tools adopted in our implementation. Except for bright- ening, all tools are cutting-edge edge deep models. For brightening, deep models are not employed since the low-light conditions considered here are not so severe as those considered by deep models (Wei al., 2018) for practical purposes.
Table 7: Prompts for VLM and LLM. {+} represents slot to fill according to the context.
Prompt for DepictQA to evaluate degradation
What's the severity of fdegradation) this image? Answer the question using a single word or phrase in the followings: very low, low, medium, high, very high.
Prompt for GPT-4 (re)schedule subtasks
There's image suffering from degradations (degradations]. We will invoke dedicated tools to address these degradations, i.c., we will conduct these tasks: {agenda). Now we need to determine the order of these unordered tasks. For your information, based on past trials, we have the following experience:
(experience)
Based on this experience, please give the correct order of the tasks. Your output must be JSON object with two fields: "thought" and "order", where "order" must be a permutation of {agenda) in the orcer you determine.
(Besides, in attempts just now, found the result is unsatis- conducted first. Remember not arrange (failed tries) in the first place.)
factory if ffailed.tries)
Prompt for DepictQA to compare two images
Waich of the two images, Image A or Image B, do you consider to be of better quality? Anawer the question using a single word or phrase.
18

---

Published as a conference paper at ICLR 2025
Table 8: Degradation combinations for exploration and testing.
Group # Degradations
Scen in exploration?
Degradation combination
rain, haze
motion blur, low resolution
low light, noise
defocus blur, JPEG compression artifact noise, JPEG compression artifact rain, low resolution
A
2
Yes
motion blur, low light
defocus blur, haze
motion blur, JPEG compression artifact haze, noise
B
No
defocus blur, low resolution
rain, low light
haze, motion blur, low resolution rain, noise, low resolution
C
3
No
low light, defocus blur, JPEG compression artifact
motion blur, defocus blur, noise
B
MORE RESULTS
B.1 LEARNING FROM EXPLORATION
After collecting the statistics in exploration, we prompt GPT-4 by:
We are studying image restoration with multiple degradations. The degradation types we are focusing on are: low-resolution noise, metion blur, defocus blur, rain, haze, dark, and ipeg compression artifact. We have tools to address these degradations, that is, we can conduct these tasks: super-resolution, denoising, motion deblurring, defocus deblurring, deraining, dehazing, brightening, and jpeg compression artifact removal. The problem is, given the tasks to conduct, very complicated because different tasks may have special re- quirements and side-effects, and the correct order of tasks can significantly affect the final result. We have conducted some trials and collected the following experience:
need to determine the order of them. This
To address darktnoise in the image, when conducting first denois- ing and then brightening, the fail rates of addressing ['dark', "noise'] are ['228', '438'] respectively, and the total fail rate is 328; when conducting first brightening and then denoising, the fail rates of addressing d'dar'', 'noise'] are ['28%', (226] re- spectively, and the total fail rate is 35%.
To address defocus blurthaze in the image, when conducting first defocus deblurring and then dehazing, the fail rates of addressing [' ['defocus blur', 'haze'] are ['08', '363'] respectively, and the total fail rate is 18%; when conducting first dehazing and then defocus deblurring, the fail rates of addressing [' ['defocus blur', "haze'] are ['08', (408'] respectively, and the total fail rate is 20%.
To address defocus blurtipeg compression artifact in the image, when conducting first ipeg compression artifact removal and then defocus deblurring, the fail rates of addressing d'defocus blur', "ieg compression artifact'] are ['108', 3318] respectively, and the total fail rate is 20%; when conducting first defocus deblur- ring and then jpeg compreasion artifact removal, the fail rates
19

---

Published as a conference paper at ICLR 2025
Table Single-degradation restoration tools.
Task
Tools
DifIBIR (Lin et al., 2024)
X-Restormer (Chen al., 2024c)
Super-resolution
HAT (Chen et al., 2023b)
SwinIR (Liang et al., 2021) trained under GAN (Goodfellow et al., 2014) paradigm
SwinIR (Liang et al., 2021) trained for optimizing PSNR
SwinIR (Liang et al., 2021) trained on noise level
SwinIR (Liang et al., 2021) trained on noise level 50
MAXIM (Tu et al., 2022)
Denoising
MPRNet (Zamir al., 2021)
Restormer (7amir al., 2022)
X-Restormer (Chen ct al., 2024c)
SwinIR (Liang et al., 2021) trained on quality factor
JPEG compression
FBCNN (Jiang ct al., 2021) trained on quality factor 90
artifact removal
FBCNN (Jiang et al., 2021) trained on quality factor 5
FBCNN (Jiang al., 2021) blind quality factor
MAXIM (Tu et al., 2022)
MPRNet (Zamir al., 2021)
Motion deblurring
Restormer (Zamir al., 2022)
X-Restormer (Chen et al., 2024c)
DRBNet (Ruan et al., 2022)
Defocus deblurring
IFAN (I.ce et al., 2021)
Restormer (7amir al., 2022)
MAXIM (Tu et al., 2022)
MPRNet (Zamir et al., 2021)
Deraining
Restormer (Zamir al., 2022)
X-Restormer (Chen et al., 2024c)
MAXIM (Tu al., 2022)
X-Restormer (Chen et al., 2024c)
Dehazing
RIDCP (Wu et al., 2023b)
DehazeFormer (Song al., 2023)
Constant shift (Adding a constant 40) Gamma correction (r = 2/3)
Brightening'
CLAHH (Zuiderveld, 1994)
All tools operated the V channel of the HSV color space.
of addressing ng ['defocus blur', 'jpeg compression artifact'] are ['88', r48*]] respectively, and the total fail rate is 288.
address motion blurtdark the image, when conducting first metion deblurring and then brightening, the fail rates of ad- dressing l'motion blur', 'dark') are ['22$', '25%'] respectively, and the total fail rate is 23%; when conducting first brightening and then motion deblurring, the fail rates addressing ('motion blur', 'dark'] are ['288', (258 respectively, and the total fail rate is 26%.
To address motion blurtlow resolution in the image, when conduct- ing first motion deblurring and then super-resolution
the fail rates of addressing ('motion blur', 'low resolution') are ['238', ]9 respectively, and the total fail rate is 163; when conduct- ing first super-resolution and then motion deblurring, the fail rates of addressing l'motion blur', 'low resolution') are ['315', '68' respectively, and the total fail rate is 198.
20

---

Published as a conference paper at ICLR 2025
To address poise+ipeg compression artifact in the image, when con- ducting first denoising and then ipeq compression artifact re- moval, the fail rates of addressing ('noise', ' 'ipeg compression aartfacct]] are ('388', (138'] respectively, and the total fail rate is 26%; when conducting first jpeg compression artifact re- moval and then denoising, the fail rates of addressing l'noise', "peg compression artifact'] are ['388', (148'] respectively, and the total fail rate is 268.
address rainthaze in the image, when conducting first deraining and then dehazing, the fail rates of addressing ['rain', (haze'] are ['58', '378"] respectively, and the total fail rate is 218; when conducting first dehazing and then deraining, the fail rates addressing ['rain', 'haze'] are ['258', '248'] respectively, and the total fail rate is 248.
To address raintlow resolution in the image, when conducting first deraining and then super-resolution, the fail rates of addressing ('rain', 'low resolution' are ['268', '23'] respectively, and the total fail rate is 148; when conducting first ruper-resolution and then deraining, the fail rates of addressing [' l'rain', 'low resolu- tion'] are ('638', '08'] respectively, and the tetal fail rate is 328.
The response, which directly serves experience in Tab. 7,
Based on the cellectdd experience, we can distill the following knowledge to guide the order of tasks for image restoration:
1. **Dark Noise**: It is slightly better to denoise before brightening (328 total fail rate) than to brighten before denois- ing (358 total fail rate).
2. **Defocus Blur + Hazex*: Defocus deblurring should be done before dehazing, as it results in a lower total fail rate (18% vs. 20%).
3. D*Defocus Blur + JPEG Compression Artifact**: Removing JPEG compression artifacts before defocus deblurring leads lower total fail rate (208 vs. 28%)
4. **Motion Blur Dark**: Motion deblurring should done be- fore brightening, as it results in a lower total fail rate (23% vs. 268).
5. **Motion Blur + Low Resolution=:: Motion deblurring should be done before super-resolution, as it results a lower total fail rate (168 vs. 198).
6. **Noise JPEG Compression Artifact*:: The order of denois- ing and JPEG compression arti removal does not affect the total fail rate (both 26%).
7. **Rain Hazex*: Deraining should be done before dehazing, as it results in a lower total fail rate (218 vs. 248).
8. **Rai + Low Resolutions:: Deraining should be done before super-resolution, as it results in a significantly lower total fail rate (148 vs. 32%)
From these observations, we can infer that generally, it is more effective to address blurring issues (motion blur, defocus blur) before addressing issues related to image content enhancement (super-resolution, brightening, dehazing). Additionally, it is often better to remove artifacts (rain, JPEG compression) before
21

---

Published as a conference paper at ICLR 2025
improving the image's s content quality. However, the impact of the order may vary, and in some cases, such as noise and JPEG compression artifact, the order does not significantly affect the outcome.
When dealing with more complex degradations, these guidelines can serve as a starting point, but it may be necessary to conduct ad- ditional trials determine the most effective order of tasks.
B.2 SCHEDULING RESULT
Tab. 10 comprehensively lists the reasoning and results of scheduling the 16 degradation combina- tions, and Tab. 11 lists the performance the resultant plans over each combination. We can see that the scheduling is logical and profitable.
B.3 SCHEDULING CONSISTENCY
Fig. 13 compares the dispersion of scheduling results with and without experience for each degra- dation combination. It can be seen that some extreme cases, scheduling with experience can al- ways yield the same result, while scheduling without experience yields nearly random results. This strongly supports our motivation that providing experience is must for the sake of consistency.
Besides. as introduced in Sec. 3.3, we find that the scheduling result may be sensitive to the presen- tation order subtasks. Therefore, the dispersion of scheduling results may arise from two aspects: one is the intrinsic uncertainty of the LLM, ie., the response may differ even though the prompt remains the same; another is the bias to presentation order, ie., the order significantly affects the result. To investigate the bias, the first aspect should be ruled out. We design an intuitive approach: for a metric that measures dispersion, compute the metric values for all presentation orders respec- tively and then subtract their average from the metric value for the overall results. Formally, denote the metric by M and the distribution of scheduling results by p. Also, when the presentation order fixed, denote the distributions of scheduling results by P1, P2, ... ,NN, respectively, where N is the number of subtask permutations. The difference
1 N 1=1
M())
M(p.)
can reflect the sensitivity to presentation order. For entropy and variation ratio, Fig. 14 compares this difference for scheduling results with and without experience. Notably, for the degradation combination in group A and B. the differences are all zero for scheduling with experience, which means the bias presentation order is totally eliminated in sense. These results again justify our design of exploration.
B.4 COMPARISON WITH OTHER METHODS
Fig. 15 shows more visual results of AgenticIR, compared with all-in-one models and random tool invocation.
Table 10: Scheduling of degradation combinations.
Group A
Degradations: rain, haze, Scheduling result: deraining, dehazing.
Reasoning:
According to the collected experience, speci point 7, more effective to perform deraining before dehazing as it results in a lower total fail rate. Therefore, the correct order of tasks should be to first apply the deraining tool and then the dehazing tool.
22

---

Published as a conference paper at ICLR 2025
Table 10 Continued
motion blur, low resolution. Scheduling result: motion deblurring, SR.
Degradations:
Reasoning:
According to the collected experience,
point 5, it is more effective to address motion blur before dealing with low resolutien. Therefore, should perform motion deblurring before super-resolution to achieve lower total fail rate.
Degradations:
low light, noise.
Scheduling result: denoising. brightening.
Reasoning:
According to the collected experience, it is slightly better to denoise before brightening when dealing with an image that is both dark and noisy. This is supported by the Lower total fail rate observed in past trials for dencising before brightening (328) compared to brightening before denoising (358).
Degradations: defocus blur, JPEG compression artifact.
Scheduling result: JPEG compression artifact removal, defocus deblurring.
Reasoning:
According ng to the collected experience, it is more effective to ad- dress image artifacts like JPEG compression before dealing with blurring issues. specifically. for the combination of defocus blur and JPEG compression artifact, the experience indicates that removing JPEG compression artifacts before defocus deblurring leads to a lower total fail rate (208 vs. 288). Therefore, the correct order of tasks should start with JPEG comprossion artifact removal followed by defocus deblurring.
Degradations: noise, JPEG compression artifact.
Scheduling result: JPEG compression artifact removal, denoising.
Reasoning:
Since the experience indicates that the order of denoising and JPEG compression artifact removal does not signiticantly affect the total fail rate, we can choose either order for the tasks. However, to maintain consistency with the general guideline that suggests removing art facts before improving the image's 8 content or quality, we will opt to remove JPEG compression artifacts be- fore dencising.
Degradations: rain, low resolution. Scheduling result: deraining, SR.
Reasoning:
Accordi ng the collected experience, speci fically point 8, more effective to address the artifact caused by rain before enhancing the image's s resolution. Therefore, we should perform deraining before super-resolution to achieve Lower total fail rate.
Degradations: motion blur, low light. Scheduling result: motion deblurring, brightening.
Reasoning:
According ng to the collected experience, speci fically point 4, it is effective to address motion blur before brightening, This results in Lower total fail rate.
Degradations: defocus blur, haze.
Scheduling result: defocus deblurring, dehazing. Reasoning:
According the collected experience, it advised address defocus blur before haze. This is because defocus deblurring should be done before dehazing, as it results in lower total fail rate.
23

---

Published as a conference paper at ICLR 2025
Table 10 Continued
Group B
Degradations: motion blur, JPEG compression artifact.
Scheduling result: JPEG compression artifact removal, motion deblurring.
Reasoning:
Since the experience suggesta that it is often better to remove artifacts before improving the image's content oz quality, and specifically that removing JPEG compression artifacts before de- focus deblurring leads to a lower total fail zate, it would be logical to apply the same principle to motion blur. Therefore, should remove JPEG compression artifacts before addressing the
motion blur.
Degradations: haze, noise.
Scheduling result: denoising, dehazing.
Reasoning:
While the provided experience does not directly address the bination of haze and noise, we can infer from the guidelines that issues related to image content enhancement should be addressed after dealing with artifacts. Since noise can be considered an artifact and dehazing is related to enhancing image content by removing haze, it would be logical denoise the image before dehazing it.
Degradations: defocus blur, low resolution. Scheduling result: defocus deblurring, SR.
Reasoning:
Followin the guidelines it is generally more effective to address blurring issues before image content enhancement. Therefore, should perform defocus deblurring before super- resolutien.
Degradations: rain, low light. Scheduling result: deraining, brightening.
Reasoning:
rect experience related to the combination of rain and dark, we can draw parallels from similar aituations. The guidelines suggest addressing artifacts like zain before enharcing the image's content. This is supported by point 7, which rec- onmends deraining before dehazing, and point 8, which advises deraining before super-resolution. resolution are content enhancement tasks, similar to brightening. Therefere, it would be logical to apply the deraining tool before the brightening tool.
Since there
Both dehazing and super-
Group C
Degradations: low light, defocus blur, JPEG compression artifact. Scheduling result: JPEG compression artifact removal, defocus deblurring, brightening.
Reasoning:
According ng to the collected experience, it its generally more effec- tive to address blurring issues before image content enhancement. Additionally, removing artifacts like JPEG compression should done before addressing blurring issues. Therefore, the order should start with JPEG compression artifact removal, followed by defocus deblurring, and finally brightening the image.
24

---

Published as a conference paper at ICLR 2025
Table 10 Continued
Degradations: haze, motion blur, low resolution. motion deblurring, dehazing, SR.
Scheduling result:
Reasoning:
According to the guidelines, it is more effective to address blur- ring issues before image content enhancement. Therefore, we should start with motion deblurring. There is no direct guide- line for the order of dehazing and super-resolation in relation to motion blur, but since dehazing is an image content enhancement and super-resolution is also an enhancement but more towards image quality, we should perform dehazing before super-resolution. This aligns with the general principle improving the image's content or quality.
removing artifacts before
Degradations: motion blur, defocus blur, noise.
Scheduling result: motion deblurring, defocus deblurring, denoising. Reasoning:
Since both rotion blur and defocus blur are types of blurring is- sues, and based on the experience that blurring issues should be addressed before image content enhancement, which in this case is denoising, we should first deal with the blurring, Between motion blur and defocus blur, there is no direct experience suggesting which shculd be addressed first. However, considering that mo- tion blur might affect the entire image while defocus blur could more localized,
might be more effective to correct the mo- tion blur first to ensure that any subsequent defocus deblurring operates on more stable image base. Therefore, the order should start with motion deblurring, followed by defocus deblurring, and finally denoising.
Degradations: rain, noise, low resolution. Scheduling result: deraining, denoising, SR.
Reasoning:
Accordi ng to the guidel guidelines, it is generally ly more effective to re- move artifacts like rain before addressing image content enhance- ment like super-resolution.
Additionally, there is no specific mention of the interaction between noise and 1ow resolution, but since noise an artifact, it would be logical to address it be- fore enhancing the image content. Therefore, the order should start with deraining, followed by denoising, and finally, super- resoluticn.
C ALTERNATIVE LLM
We check whether our framework can also achieve competitive performance with other LLMs. Tab. 12 lists the performance outcome of replacing GPT-4 with the popular open-source Llama 3.1 405B (Llama Team, AI @ Meta, 2024). It can be seen that the shift from GPT-4 Llama does not influence the performance severely, and AgenticIR with Llama surpasses random invocation by a large margin too. This is because provided with the experience, enough to be handled by different LLMs, and Llama can give results similar to GPT-4. In fact, the performance difference is more likely to be caused by randomness of tool invocation.
scheduling problem is easily
D DISCUSSION ON EEPPLORATION-EXPLOTIEE
TRADEOFF
A primary problem in docisionnmakinge is the tradeoff between exploration and exploitation (Berger- Tal al., 2014). AgenticIR, as a heuristic search, deals with this by greedily exploiting acceptable directions and pruning those seemingly unpromising directions in reflection. To some extent, this behavior favors exploitation and thus may suffer from early stopping, resulting in suboptimal re- sults. However, such preference is configurable. That is, reflection to suppress exploitation so as to force exploring more. We conduct an experiment that lets agent only accept tool outputs with very low severity of degradations, denoted AgenticIR".
adjust the acceptance threshold in
25

---

Published as a conference paper at ICLR 2025
Table 11: Detailed comparison between acting the agent's plan and the opposite. "Not as planned" means randomly shuffling the plan (guaranteed to be different).
Degradations
As planned PSNR SSIM LPIPS MANIQA
CLIP-IQA MUSIQ
rain, haze
/
19.00 0.8075 0.1655
0.4309
0.6015
68.25
x
16.35 0.7324 0.2250
0.4296
0.5900
68.03
motion blur,
/
20.57
0.5532
0.2575
0.4192
0.6575
66.38
low resolution
x
20.28
0.5427
0.2637
0.4374
0.6568
67.19
low light, noise
/
20.13
0.7454
0.3042
0.3516
0.4679
59.88
x
18.97
0.7175
0.2997
0.3962
0,5022
61.63
defocus blur, jpeg compression artifact
V
24.34 0.6846
0.3885
0.2163
0.2859
45,83
X
23.78
0.6580
0.4614
0.1938
0.2743
41.53
noise,
V
28.08
0.8129
0.2491
0.3537
0.5343
60.03
jpeg compression artifact
X
27.50
0.7997 0.2714
0.3484
0.5182
58.95
rain,
/
20.42
0.6005
0.3045
0.4216
0.6710
66.42
low resolution
X
21.59
0.5796
0.3801
0.3449
0.5292
64.35
motion blur,
V
17.06
0.5350
0.2849
0.2785
0.4151
60.03
low light defocus blur, haze
x
17.05
0.5442
0.2903
0.2826
0.3935
60.41
/
19.52
0.7300
0.2484
0.3033
0.4399
59.33
x
20.81
0.7477
0.2563
0.2751
0.3908
56.73
motion blur, jpeg compression artifact
/
22.12
0.6638
0.3385
0.2355
0.3784
51.24
x
21.61
0.6464
0.3831
0.1943
0.3215
45.64
haze,
/
20.20
0.7395
0.3130
0.3480
0.4645
59.81
noise
X
17.45
0.6654
0.3523
0.3785
0.5519
58.38
defocus blur, low resolution
/
22.42
0.6118
0.2475
0.4283
0.6681
67.92
23.22 x
0.6232 0.2491
0.4432
0.6777
67.74
rain,
/
19.82
0.8202
0.1743
0.4234
0.5991
68.70
low light
X
18.99
0.7892 0.2058
0.4334
0.5517
68.86
haze, motion blur,
/
17.20
0.4923
0.2951
0.4193
0.6640
65.97
low resolution
X
15.98
0.4730
0.3215
0.4084
0.6698
65.32
rain, noise, low resolution
/
20.73
0.5602
0.3785
0.4100
0.6736
62.34
X
21.14
0.5868
0.3908
0.3683
0.5610
59.89
low light, defocus blur, ipeg compression artifact
V
18.57
0.6194
0.4223
0.1978
0.2726
47.04
X
18.40
0.6108
0.4251
0.1951
0.2685
46.45
motion blur, defocus blur. noise
18.64
0.4690
0.5997
0.2202
0.3401
28.96
X
18.42
0.4402
0.6006
0.2514
0.3883
33.63
The results are shown in Tab. 13, compared with the original setting and random tool invocation. AgenticIR" does obtain slightly better results most metrics, but also consumes much more time shown in Tab. 14. Therefore, we believe it is fair to say the current setting of AgentielR strikes balance between performance and efficiency.
E LIMITATION
Our primary goal is to realize an intelligent agent for IR tool ensemble, but this paper only considers single-degradation restoration tools, limiting the application the so-called cumplex-degradation restoration problem. is worth exploring whether our paradigm can integrate more general and het- erogeneous tools, which will require higher flexibility and decision-making capabilities. generally capable agent is supposed to surmount these challenges.
26

---

Published as a conference paper at ICLR 2025
rain+ haze
motion blur-+low resolution
low iigise
defocus blur+jpeg compression artifact
Degradation combination
noise+jpeg compression artifact
rain+low resolution
motion blur+ Hlo light
defocus blur+ haze
motion blur+jpeg compression artifact
haze +noise
defocus blur+low resolution
rain light
haze-+mation blur+Hlow resolution
nain+ resolution
low right+deffcce blur+jpeg compression artifact
w/o experience
w/ experience
mation blur-defocus blur+noise
0.0
0.2
0.4
0.6
0.8
1.0
Entropy (bit)
rain+haze
motion blur+low resolution
low light+nolise
defocus tjpeg compression
Degradation combination
noise+jpegg compression artifact
rain+low resolution
motion blur+ low light
defocus blur+ haze
motion blur jpeg compression artifact
haze+noise
defocus blur+low resolution rain+ low light
haze+mmttion blur+low resolution
roin+ resolution
low light=defocuus blur+jpeg compression artifact
w/o experience w/ experience
blur+defocus blur+noise
0%
10%
20%
30%
40%
50%
Variation ratio
Figure 13: Detailed comparison between dispersion of scheduling results with and without experi- ence.
Table 12: Quantitative comparison between our framework with different LLMs and random tool invocation. "AgenticIR" has the same setting in the main text, while "AgenticIR (Llama)" "Ulama)""rplaaa"e replaces GPT-4 with Llama for (re)scheduling.
Degradations
Method
PSNR
SSIM
LPIPS
MANIQA
CLIP-IQA
MUSIQ
AgenticIR (Llama)
21.06
0.6834
0.3084
0.3123
0,4516
57.61
Group A
AgenticIR
21.04
0.6818
0.3148
0.3071
0.4474
56.88
Random
20.90
0.6642
0.3368
0.2963
0.4394
55.30
AgenticIR (Llama)
20.79
0.7019
0.3062
0.3174
0.4648
57.47
Group B
AgenticIR
20.55
0.7009
0.3072
0,3204
0.4648
57.57
Random
20.06
0.6766
0.335
0.3120
0.4514
56.15
AgenticIR (Llama)
18.80
0.5480
0.4562
0.2675
0.3859
48.13
Group C
AgenticIR
18.82
0.5474
0.4493
0.2698
0.3948
48,68
Random
18.87
0.5456
0.4796
0.2354
0.3543
44.61
27

---

Published as a conference paper at ICLR 2025
rain+haze
motion blur-+low resolution
low iiois
defocus blur+jpeg compression artifact
Degradation combination
noise+jpeg compression artifact
rain+low resolution
motion blur low light
defocus blur+haze haze
motion blur+jpeg compression artifact
haze-noiie
defocus blur+low resolution
rain+lou light
haze+motion blur+low resolution
noie resolution
low light=defocus +jaeg compression artifact
w/o experience
w/ experience
motion blur+defocus blur+noise
0.0
0.2
0.6 Entrapy difference (bit)
0.4
0.8
1.0
rain+haze
mation blur+lou resalution
low light
defocus blur jjur+ compression artifact
Degradation combination
noise+jjess compression artifact
rain+low resolution
motion blur+How light
defocus blur+haze
w/o experience
motion blur+jpeg compression artifact
w/ experience
haze+noiie-
defocus blur+low resolution
roin light
haze+mmation blur+low resolution
rain nolse low resolution
light++defocus blur+jpeg compression artifact
motion blur defocus blurtnoise
0%
10%
20%
30%
50%
Variation difference
Figure 14: Detailed comparison between sensitivity to presentation order when scheduling with and without experience.
Quantitative comparison between restoration performance AgenticIR with different acceptance thresholds in reflection, and thus different preferences for exploration.
Table 13:
Degradations
Method
PSNR
SSIM
LPIPS
MANIQA
CLIP-IQA
MUSIQ
AgenticlR AgenticIR*
21.04
0.6818
0.3148
0.3071
0.4474
56.88
Group A
20.95
0.6790
0.3169
0.3082
0.4475
56.99
AgenticIR
20.55
0.7009
0.3072
0.3204
0.4648
57.57
Group B
AgentieIR"
20.72
0.7014
0.3017
0.3223
0.4684
57.78
AgenticIR
18.82
0.5474
0.4493
0.2698
0.3948
48.68
Group C
AgenticIR*
18.90
0.5459
0.4582
0.2667
0.3902
47.71
28

---

Published as a conference paper at ICLR 2025
AirNet
PrompdIK
MIOIR
DA-CLIP
image with defocus blur JPEG
Instruci|R
AutoDDR
Random
Agentie
AirNet
PrompUR
MIOIR
DA-CLIP
Input image with haze and noise
InstrucUR
AIRoDIR
Random
AgentieIR
AirNer
PromptiR
MIOIR
DA-CLIP
Input image light
Instruct1R
AuteDIR
Random
AgenticIR
AirNet
PromptR
MiOIR
DA-CLIP
Input image haze, low resolution
InstructIR
AutoDIR
Random
AgentilIR
Figure 15: Qualitative comparison with other methods.
Table 14: Quantitative comparison between cost of AgenticIR with different acceptance thresholds in reflection.
Degradations
Method
Wall clock time (second)
# Tool invocations
AgenticIR AgenticIR"
48
3.37
Group A
137
8.07
AgenticIR
54
3.63
Group B
AgenticIR*
117
7.01
AgenticIR
4.77
Group C
AgenticIR
174
10.50
29


Reviews:
[{"review_id": "m31fjsJDmR", "reviewer": "Reviewer_oGv6", "summary": "This paper presents an agentic workflow based on LLM/VLMs for image restoration. The agentic system follows how actual humans would process images, consisting of five stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. Since the existing VLMs are not sufficiently capable of analyzing image quality or reasoning about the order of image processing tasks, the VLMs are finetuned and allowed for (self-)exploration to understand the effects of scheduling the image restoration components. Experimental results clearly demonstrate the effects of scheduling with learned experience and the other proposed components.", "questions": "1. How are the discovered workflows similar to the original motivation of following the human workflow? For instance, is the subtask scheduling by GPT-4 (w. experience) match the best practices performed by a human? It would be better if the authors could provide more insights or discussions.\n\n2. How does the proposed model perform when there is only a single type of degradation? Does it also perform competitively?\n\n3. What is the criterion for deciding whether Execution step is Failed or Successful? (I might have missed)", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Clear presentation of the benefits of the proposed methodologies. I especially liked Figure 3, 6, 7, and 8, where the authors show dramatic improvements on why choosing a good scheduler for the image restoration components is important, as well as reflection and rollback.\n\n2. Novelty in connecting the human process of IR with LLM-based agentic systems. Though the idea of mimicking the human workflow is being widely adopted more recently, application to image restoration tasks and showing effectiveness is not demonstrated before, to my knowledge.\n\n3. Thorough justification of the design choices and careful experiment designs. Reasons for the proposed workflow and the capabilities the authors are trying to give to the LLM/VLMs are well described, and the evaluations seem to be fairly performed.", "weaknesses": "1. No cost analysis. Using such agentic systems require numerous requests to the LLM/VLM APIs; if the system chooses to perform \"Reflection\" with the tree search, the worst case scenario would be extremely costly. Compared to the existing image restoration models, the proposed model uses significantly more compute. In this sense, given that many previous works (roughly) match the FLOPs when comparing the restoration quality, one might argue that the comparisons are unfair.\n\n2. Relatively subtle improvements for quantitative metrics (though qualitative improvements look quite significant). I would suggest also adding quantitative measures on the figures so that the readers can compare both aspects with a single glance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pEyFwFcNyq", "reviewer": "Reviewer_wMVY", "summary": "The proposed AgenticIR system addresses the inherent complexity of real-world image restoration (IR) tasks by emulating a human-like, multi-stage processing workflow. The system operates through distinct phases: Perception, Scheduling, Execution, Reflection, and Rescheduling. It integrates Large Language Models (LLMs) and Vision Language Models (VLMs) into a collaborative framework, allowing text-based interactions to direct the application of specialized IR models. This agentic approach builds on existing multi-modal IR frameworks by dynamically adapting its restoration strategy, actively reassessing and adjusting to handle various complex degradations.", "questions": "NA", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The proposed system offers a comprehensive approach to image restoration, addressing a broad range of degradation types through a structured, adaptable methodology.\n2. It incorporates human-interaction-inspired insights into the image restoration process, potentially enhancing adaptability and effectiveness in handling complex restoration tasks.", "weaknesses": "1. **Comparison fairness**: The comparative experiments appear to lack fairness, as the baselines (e.g., InstructIR) are designed as unified IR models trained to handle multiple degradation types in a single framework. In contrast, AgenticIR leverages specialized off-the-shelf models for each type of degradation. Therefore, it would be more appropriate to compare AgenticIR to the state-of-the-art models for each specific degradation task rather than to unified restoration models.\n  \n2. **Efficiency Concerns**: While the system is comprehensive, its workflow is notably complex and lengthy. Compared to regular image restoration models, how efficient is AgenticIR in processing images? This is a critical aspect for real-world applications of image restoration and should be addressed with precise comparative evaluations.\n\n3. **Toolbox Ablation Study**: Lines 159-160 state, \"For each degradation, we collect three to six advanced models to build the ‘toolbox’ that the intelligent agent can use.\" There is no ablation study analyzing the impact of selecting these advanced models on the system’s effectiveness. Understanding the influence of each selected model in the toolbox could provide valuable insights.\n\n4. **GPT-4 Usage and Reproducibility**: AgenticIR uses GPT-4, but GPT-4 lacks a fixed version, which raises concerns about the reproducibility of experimental results. Additionally, there is no ablation study on the effects of using alternative LLMs, particularly open-source options, on performance outcomes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n9FgRaSLrp", "reviewer": "Reviewer_sgx9", "summary": "To address the complex image restoration(IR) problems in real-world scenarios, the authors propose AgenticIR, which is an agent-based system that comprises five stages: perception, scheduling, execution, reflection, and rescheduling. The system leverages a Vision-Language Model (VLM) and a Large Language Model (LLM) to reason and connect these five stages, utilizing an IR toolbox during the execution phase to perform actual image reconstruction.\n\nThe three main elements in this process are the LLM, VLM, and the IR toolbox. The VLM primarily analyzes the image quality and summarizes the degradation issues, fine-tuning its capabilities based on existing work. The LLM is responsible for planning and strategizing based on the VLM's results, utilizing GPT-4 and employing a self-exploration method to gain experience with IR problems. The IR toolbox consists of a combination of 3-6 existing models tailored to each type of image degradation.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThis paper is well-written and easy to follow.\n-\tThe experimental setup is comprehensive, with sufficient ablation and comparative experiments demonstrating the effectiveness of their proposed methods\n-\tThe discovery that execution order is key to restoring complex degraded images is compelling.", "weaknesses": "-\tThe main concern is that this work resembles several widely-used frameworks (e.g., large language models (LLMs), vision-language models (VLMs) and image restoration models), giving it a predominantly engineering-focused approach. \n-\tAdditionally, this work appears complex, so providing statistics on the time and complexity involved in a single inference would enhance clarity.\n-\tGiven that LLMs and VLMs often struggle with the issue of 'hallucination,' does this work encounter a similar challenge? If so, how does it address this problem?\n-\tWhat are the limitations of the proposed framework?\n-\tRegarding the point that 'execution order is crucial', are the documents (knowledge) remain consistent during inference across different test samples?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qZiXQiSk2a", "reviewer": "Reviewer_hoAk", "summary": "This paper introduces AgenticIR, an intelligent system designed to handle complex image restoration tasks by emulating human-like problem-solving methods. The system operates through five stages: Perception, Scheduling, Execution, Reflection, and Rescheduling. AgenticIR leverages LLM and VLM, using their text generation capabilities to operate a set of IR tools dynamically. It relies on VLMs for image quality assessment and LLMs for step-by-step reasoning, enhancing its adaptability to various IR challenges.\n\nIt also incorporates a self-exploration mechanism that generates referenceable summaries from past restoration attempts, which improves its decision-making. Experimental results show AgenticIR’s effectiveness in complex restoration scenarios, highlighting its potential for real-world automated image processing and broader AI applications in visual processing.", "questions": "In addition to the weaknesses mentioned, I have two further questions:\n\n1. Why Not Use VLMs Exclusively Throughout the Pipeline? Given VLMs' strong capabilities in image quality assessment and reasoning, could a VLM-only approach be more efficient or effective for the entire pipeline?\n2. Would Online Updates to the Reference Data Benefit the Pipeline? Could implementing real-time updates to the experiential knowledge base further enhance the pipeline’s adaptability and performance?\n\nI hope the authors can make up for the weaknesses mentioned and address these questions.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Human-Centric Design: AgenticIR provides an image restoration approach that mirrors human actions, incorporating processes like reflection and iterative rescheduling into its pipeline. This design enhances action interpretability and facilitates meaningful human interaction with the system.\n2. Clear and Concise Expression: The paper presents complex ideas with clarity, accompanied by detailed images and diagrams that enhance comprehension and support the technical explanations.\n3. Comprehensive Experiments and Ablation Studies: Thorough experimental evaluations are provided, with well-structured ablation studies for each module. This approach effectively validates the system's design and performance.\n4. Illustrative Pipeline Examples: The pipeline is illustrated with specific cases, offering a clear understanding of how each component functions within real-world scenarios.", "weaknesses": "1. Limitations Compared to Optimal Solutions: AgenticIR tends to encounter an early stopping issue, where it may settle on a satisfactory solution prematurely, halting further exploration and potentially missing the optimal outcome. Addressing this limitation is important, and it might be beneficial to add an additional row to Table 4 to reflect this aspect.\n2. Insufficient Reporting on Iteration Count and Processing Time: Although the paper emphasizes the role of experiential information and provides illustrative examples, it lacks concrete data on the actual reduction in iterations or time consumption. Including specific metrics on these improvements would strengthen the evaluation of AgenticIR’s efficiency and practical advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)