Learning A Low-Level Vision Generalist via Visual Task Prompt

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 22:20:35

Abstract: Building a unified model for general low-level vision tasks has important research and practical value. However, existing methods still face challenges when dealing with diverse low-level vision problems. Multi-task restoration approaches can simultaneously address various degradation-to-clean restoration tasks, while their applicability to tasks with different target domains (e.g., image stylization) remains limited. Existing methods like PromptGIP that can handle tasks with multiple input-target domains mainly rely on the Masked Autoencoder (MAE) training paradigm. Unfortunately, these approaches are restricted by coupling to the ViT architecture, resulting in suboptimal image reconstruction quality. In addition, they tend to be sensitive to prompt content and often fail when handling more tasks that involve low-frequency information processing, such as color and style. In this paper, we present a Visual task Prompt-based Image Processing (VPIP) framework to address the above challenges. This framework employs the visual task prompt to process tasks with different input-target domains. Besides, it provides the flexibility to select a backbone network suitable for various low-level vision tasks. A prompt cross-attention mechanism is introduced to deal with the information interaction between the input and prompt information. Based on the VPIP framework, we train a low-level vision generalist model, namely GenLV, on 30 diverse tasks. Experimental results show that GenLV can successfully address a variety of low-level tasks, and it significantly outperforms existing methods both quantitatively and qualitatively.

Main paper: 
 related work: 
 Low-Level Vision. Over the past decade, the field of low-level vision has undergone significant advancements, largely attributable to the integration of deep learning techniques. Classic low-level vision tasks include but not limited four categories: image restoration, image enhancement, image feature extraction and stylization. Image restoration aims to restore the high-quality clean image from observations degraded by a variety of factors, including lowresolution [16] , noise [50] , blur [1] , JPEG compression [15] and bad weather, such rain [46] and haze [47] . Image enhancement [36] is concerned with the alteration of specific image attributes, such as color [19] , sharpness [2] , exposure [10] and brightness [8] , in order to make the image more suitable for a particular task or viewer. Image feature extraction, like edge detection [7] , focuses on extracting the low-level features from the image, which can help downstream enhancement and understanding tasks. Image stylization is to create visually appealing images with a specific style or aesthetic [23] . Despite the advancements, current methods often rely on specialized datasets and customized network architectures, which limits the practical application of these methods.Prompt Learning. In the NLP field, the concept of prompting is initially to supply manually selected in-context information to a pretrained model for implementing the target task [4] . Instead of using manual prompt, many follow-up works propose to treat the prompt as task-specific vectors to adapt model for various tasks [20, 25] . Prompt learning techniques have also been applied in the field of computer vision, where they have proven effective in modeling task-specific instructions across various applications [21, 51] . Notably, MAE-VQGAN [3] and Painter [40] leverage the flexibility of prompting to unify a variety of vision tasks. These models demonstrate impressive performance on high-level tasks, such as semantic segmentation, by constructing the grid-like prompt. However, their effectiveness in low-level vision tasks is insufficient [30] . In the realm of low-level vision, PromptGIP [30] proposes an MAE-based framework and use the grid-like visual prompt to handle 15 crossdomain image processing tasks. Despite this, as the number and complexity of tasks increases, the effectiveness of this method decreases significantly. Besides, the training paradigm employed by PromptGIP is deeply intertwined with the ViT architecture, which largely limits the image reconstruction quality of its output.Multi-task Image Restoration. Multi-task image restoration aims to train a single model to handle multiple restoration tasks simultaneously. Existing multi-task image restoration methods can be categorized into two groups. The first group of methods aim to process real-world images with unknown degradation, emphasizing the modeling of complex real-world degradation. The representative approaches include BSRGAN [49] and Real-ESRGAN [41] ). In contrast, the second group of methods like DASR [39] and Air-Net [26] are developed to handle several specific restoration tasks with predefined degradation. These methods mainly focus on designing better modules for multi-task learning to maximize network capability of task-specific restoration performance. Some current works such as ProRes [32] and PromptIR [35] are proposed to leverage a learnable prompt from the input image for better multi-task restoration. However, all these approaches are limited to solving the degradation-to-clean restoration problem, and lack the ability to deal with a broad range of cross-domain low-level vision tasks. Unlike these approaches, our method aims to construct a low-level vision generalist model, which is not only capable of image restoration, but also excels at handling a wider range of cross-domain tasks, including enhancement, feature detection and stylization. 
approach 3.1 representative low-level vision tasks: 
 Low-level vision tasks encompass a range of pixel-level manipulations, including image restoration, enhancement, feature extraction, stylization, etc. Each task uniquely transforms an input image space to a specific target domain. For example, the target domain of image restoration is high-quality (HQ) image space Ω 𝐻𝑄 , while the outputs of edge detection are edges maps Ω 𝐸𝑑𝑔𝑒 . Formally, given an arbitrary input image 𝐼 , the low-level vision task can be defined as T 𝑡𝑎𝑠𝑘 : Ω 𝑆 → Ω 𝑇 , where Ω 𝑆 and Ω 𝑇 denote the source image space and the target image space, respectively. According to the target domain, low-level vision tasks generally fall into the following categories as:Restoration: T 𝑅𝑒𝑠 : Ω 𝑆 → Ω 𝐻𝑄 , Enhancement: T 𝐸𝑛ℎ : Ω 𝑆 → Ω 𝐸𝑛ℎ , Edge Detection: T 𝐸𝑑𝑔 : Ω 𝑆 → Ω 𝐸𝑑𝑔 , stylization: T 𝑆𝑡 𝑦 : Ω 𝑆 → Ω 𝑆𝑡 𝑦 . (1)Each category encompasses a variety of tasks, as presented in Figure 2 . Our goal is to address all these tasks through a unified model.Existing low-level vision methods are typically designed for specific tasks, which inherently restricts their applicability to tasks with different target domains. Taking the image restoration model as an example, they accept low-quality images as input and predict the high-quality output as:EQUATIONwhere F T 𝑅𝑒𝑠 represents the restoration model parameterized by Θ.The restoration model can accommodate various restoration tasks through incorporating multiple degradations into training, such as blur, denoising and deraining, given that the output image space of these tasks are the same, i.e., Ω 𝐻𝑄 . However, this kind of model cannot be extended to simultaneously implement tasks like edge detection, which targets a completely different output modality.To train a low-level vision generalist model that can process cross-domain tasks, our approach employs a unified framework capable of handling various cross-domain tasks by utilizing additional image pair as the prompt [𝑃 Ω 𝑆 , 𝑃 Ω 𝑇 ]. It can be denoted as:EQUATIONThis formulation allows diverse task mappings to be represented by intuitive image pairs, which marks a significant difference from conventional low-level vision models, offering a more holistic and adaptable approach to broad cross-domain low-level vision tasks.In this section, we illustrate the specific design of our low-level vision generalist model, as shown in Figure 3 . The overall approach is predicated on the Visual task Prompt-based Image Processing (VPIP) framework. A powerful image processing network and a prompt encoder network are used to process the input image and the prompt images. A new prompt cross-attention mechanism is introduced to achieve the information interaction among latent representations of the input image and prompt images. VPIP Framework consists of an end-to-end image processing main network, a prompt encoder network and a prompt interaction mechanism. Given an input image 𝐼 𝑖𝑛 , it is initially processed to a high-dimensional latent feature 𝑧 𝑖𝑛 through the encoder. In parallel, the paired prompt images [𝑃 Ω 𝑆 , 𝑃 Ω 𝑇 ] are fed into the prompt encoder to generate two high-dimensional representations[𝑧 𝑃 Ω 𝑆 , 𝑧 𝑃 Ω 𝑇 ],both of which with the same spatial size as 𝑧 𝑖𝑛 . Following this, the Image Processing Backbone plays a crucial role in the image reconstruction quality for low-level vision tasks. Since different low-level vision tasks often have different requirements for network capability, the most important criterion for selecting a backbone network is its task generality. Due to the lack of work investigating task generality across a wide range of low-level tasks, we focus more on the model performance for image restoration. A recent work conduct a detailed study on the backbone network for restoration tasks and propose a general backbone network, X-Restormer [9] , suitable for multiple various restoration tasks. Therefore, we simply adopt similar architecture as our main network1 . Specifically, the main network employs a U-shape architecture, where downsampling and upsampling operations are performed three times, and skip connections are added from the encoder to the decoder at the same scale. The basic modules to construct the network are Transposed Self-Attention Block (TSAB) and Spatial Self-Attention Block (SSAB), which deal with the channel-wise global information interaction and the spatial information interaction, respectively. The structures of TSAB and SSAB are shown in Figure 3 . The implementation details of their attention mechanism can be referred to the OCA in HAT [12] and the MDTA in Restormer [48] .Prompt Cross-attention is designed to perform information interaction among the prompt and input representations. Prompt-GIP [30] demonstrates that calculating global attention in the feature space can effectively incorporate task prompt information into image processing. However, this approach is tightly coupled with the ViT architecture and is relatively inefficient in terms of the attention computation. Inspired by the Stable Diffusion [38] model, which utilizes cross-attention to apply text prompt to the denoising UNet, we adopt a similar mechanism to introduce visual prompt information into the image processing network. As depicted in Figure 3 , the Prompt Cross-Attention Block (PCAB) is implemented by adding a PCA module to the standard SSAB and is integrated at the bottom of the U architecture. To calculate the PCA, the query (Q), key (K) and value (V ) are first generated by 1 × 1 convolutions from the input representation 𝑧 𝑖𝑛 , prompt input embedding 𝑧 𝑃 Ω 𝑆 and prompt target embedding 𝑧 𝑃 Ω 𝑇 . Then, the standard attention is computed to obtain the output representation 𝑧 𝑜𝑢𝑡 . Compared to calculating global self-attention on the grid-like features consisting of four image representations across the entire network, our prompt cross-attention calculated based on the size of one image representation in just a few blocks (i.e., PCAB) is much more efficient in terms of attention computation, as shown in Figure 4 .Prompt Encoder is employed to encode the prompt images into deep representations that can be used for information interaction. We simply utilize a series of standard residual blocks spaced by multiple downsampling operations to build the encoder network. 
experiments and analysis 4.1 experimental setup: 
 Task Settings. We train the models on 30 diverse low-level tasks, as follows: 1) Image Restoration: Following PromptGIP [30] , ten classic degradation types are considered including Gaussian noise, Gaussian blur, Poisson noise, salt & pepper noise, JPEG compression, ringing artifacts, R-L algorithm [37] , inpainting, haze and rain. During the training, the on-the-fly data pairs are generated on ImageNet [14] for the first eight types. Simple mixed degradations are also considered for training. ITS dataset [27] and Rain13K [22] are utilized for dehazing and deraining. A simple additive rain model is also employed for synthetic data. For testing, a mixed dataset, Common528 [30] , composed of several low-level vision benchmark datasets is employed. 2) Image Enhancement: This category includes eight tasks: low-light enhancement (LLE), photo retouching, local Laplacian filtering [2] (LLF), multi-scale tone manipulation (MTM) [18] , underwater image contrast enhancement (ICE) based on histogram equalization, underwater image color correction (ICC) based on the DIVE+ software, image SDR-to-HDR and HDR-to-SDR [13] . The LOL dataset [42] is used for LLE, and expert-C retouched images of the Adobe-MIT Fivek dataset [5] are used for retouching, LLF, and MTM. The UIEB dataset [28] is utilized for the two underwater image enhancement tasks. 3) Image Edge Detection: This category includes three edge detection tasks: Canny operator, Laplacian operator and a perceptual edge detection (PED) [34] . 4) Image Stylization: Nine style are chosen, including pencil drawing [31] , photographic style [2] , relative total variation (RTV) [44] , Vermeer style, JOJO style, Raphael style, Fauvism style, Divisionism style and Cloisonnism style. Expert-C retouched images of Adobe-MIT Fivek dataset [5] are also used to generate the image pairs. Data of the first three styles are implemented via available toolkit, and the last six neural styles are generated by a state-of-the-art style transfer method AdaAttN [29] . Implement details. For the backbone network, we adopt the similar setting as the original X-Restormer [9] . From level-1 to level-4, the numbers of consecutive blocks (each block contains a TSAB and an SSAB) are [2, 4, 4, 4] , attention heads in TSA and SSA are both [1, 2, 4, 8] , and channel numbers are [48, 96, 192, 384] . For the prompt encoder network, we employ four residual blocks for each downsampling level. During the training, the input size is set to 256 × 256 for the input image and prompt images. 𝐿 1 loss is utilized as the loss function. AdamW optimizer with 𝛽 1 = 0.9 and 𝛽 2 = 0.99 is adopted with an initial learning rate of 1𝑒 -4 . The batch size is set to 64 and total training epochs are 30.The quantitative results for various low-level vision tasks are presented in Table 1 , Table 2 and Table 3 . Given that not all existing methods are capable to handle tasks across different target domains, our primary comparative experiments are centered on restoration tasks in Table 1 . We consider three distinct experimental settings. The first setting utilizes a pretrained model, i.e., Real-ESRGAN [41] , which is capable of handling a variety of restoration tasks. The second setting is based on the training configuration outlined in our paper, but it solely focuses on image restoration tasks. The third setting involves training on the all 30 low-level vision tasks.Ablation Study on Visual Prompt. Since the models without using task prompt cannot process tasks with different target domains, we conduct the ablation study of the visual task prompt on restoration tasks. In Table 1 , ViT ★ and X-Restormer ★ are two end-to-end models only trained on image restoration tasks, while ViT-VPIP ★ and GenLV ★ (the GenLV model can also be represented as X-Restormer-VPIP) are models based on our VPIP framework, utilizing ViT and X-Restormer as their backbone respectively. Upon the incorporation of prompt learning, both ViT-VPIP ★ and GenLV ★ exhibit substantial performance gains over ViT ★ and X-Restormer ★ in most restoration tasks. This demonstrates the effectiveness of the visual prompt in facilitating the backbone network to better handle various tasks. It is noteworthy that X-Restormer ★ , without using visual prompt, struggles with the dehazing task, achieving only 16.73dB. A similar phenomenon also occurs for the multi-task restoration method PromptIR [35] . In contrast, GenLV ★ tackles it considerably better, reaching 25.63dB. All these results show the effectiveness of our proposed VPIP framework. Influence of Backbone Network. In Table 1 , when trained on the same setting, the performance of models using X-Restormer as the backbone network (i.e., X-Restormer ★ , GenLV ★ and GenLV † ) significantly surpasses that of models using ViT (i.e., ViT ★ , ViT-VPIP ★ and ViT-VPIP ★ ). This observation suggests that an appropriate backbone network is important for low-level vision tasks generalist models, and ViT architecture may limit the model performance.Comparison with other methods. In Table 1 , GenLV ★ outperforms the state-of-the-art blind SR method Real-ESRGAN [41] and multi-task restoration method PromptIR [35] , when only considering image restoration tasks. Note that we retrain the PromptIR model on the same setting for fair comparison (the original Promp-tIR is trained only on 4 tasks). By employing the ViT network, PromptGIP ★ trained on restoration tasks performs better than ViT-VPIP ★ , due to more attention computation. However, as more tasks are involved, ViT-VPIP † outperforms PromptGIP † and Painter † instead, showing the superiority of our framework for solving more diverse tasks. In Table 2 and Table 3 , we further show the quantitative results on broader low-level vision tasks. Only methods capable of solving tasks across different target domains are considered in the comparison. The models employed VPIP framework outperform Painter and PromptGIP on a variety of low-level vision tasks.In Figure 5 , we present the visual comparison of our GenLV with Painter and PromptGIP across various low-level vision tasks. From a holistic perspective, GenLV produces results that are more consistent with the ground truth, especially in aspects such as color and brightness. In contrast, the results produced by Painter and Prompt-GIP are easily affected by errors in low-frequency information, manifesting as color anomalies or even incorrect task execution. Rather that our method where prompt information can accurately serve as task instruction, Painter and PromptGIP appear to be significantly affected by the content of the prompt image. In terms of image reconstruction quality, the images generated by GenLV have clear textures and details. Conversely, Painter and PromptGIP may suffer from blurring or blocking artifacts, particularly for image restoration tasks. Overall, the above results show the superiority of GenLV in visual quality for dealing with various low-level tasks.Table 1 : Quantitative results on image restoration tasks. #: public released model. ★: trained with only restoration tasks. †: trained with all 30 low-level vision tasks. GN: Gaussian noise. PN: Poisson noise. ViT-VPIP: ViT backbone adopted in the VPIP framework. Our GenLV can also be represented as X-Restormer-VPIP. PSNR↑ (dB) is calculated as the quantitative metric. The above results have demonstrated the advantages of our prompt mechanism compared to existing methods from quantitative and qualitative perspectives. In this section, we conduct more experiments to further illustrate the effectiveness and explore the limitation of the task prompt in our method.Influence of Different Prompts. To explore the influence on the quantitative performance for different prompt images, we randomly select 20 prompt image pairs for each task and calculate the performance on the corresponding test sets. Then, we compute the standard deviation of the 20 performance results for each task, as shown in Table 4 . We can see that except for PencilDraw, the standard deviations are around or lower than 0.1dB. This shows that our method is stable in performance for different prompts.Task Prompt on Complex Situations. We conduct further experiments to investigate the effectiveness of task prompt on complex situations. In Figure 6 (a), we exhibit the outputs for images subjected to mixed degradation. The results show that the task prompt successfully guide the mapping under this situation, and our method has the capability to deal with tasks with mixed degradation. In Figure 6 (b), we present the results for cross-domain prompt. Utilizing Canny edge detection and LLE prompts, we instruct the model to process the noisy images. We can see that our model accurately execute the target task according to the visual prompts other than perform denoising. In Figure 6 (c), we show the results on processing mixed degraded images using single-task prompts. The first row present the application of a denoising prompt to a low-light, noisy image. In the second row, we show that a deraining prompt is applied to a blurry image rain streaks. It can be see that the taskspecific prompts effectively guide the model to perform the target task. All these results demonstrate the effectiveness of the visual task prompt in our method across a variety of complex situations. Mismatch Test. We conduct mismatch test to illustrate the impact of the prompt on the model under special scenarios, as shown in Figure 7 . The first row demonstrates providing the deblurring prompt to a clean image. In the second and third rows, we provide deJPEG and denoising prompts for low-light and blurry images, respectively. Ideally, we hope that the model do not execute the wrong prompt (this is reasonable from the perspective of the prompt cross-attention mechanism). It is observed that the model ideally preserves the original input images instead of performing degradation removal in these three instances. However, the mismatch test does not consistently yield ideal outcomes. In the fourth row, the model conducts deraining when provided with an inpainting prompt. From this perspective, this indicates that the model still inevitably overfit some data or mappings during training.  
limitations and prospects: 
 Our GenLV model demonstrates commendable performance in solving a broad range of low-level vision tasks, leveraging the visual prompt-based image processing framework and a powerful backbone network. Nonetheless, there are certain limitations and potential areas for further exploration that warrant attention. The working mechanism of this method is still to divide the task space via visual prompt, to achieve multi-task low-level vision. Despite the considerable improvement in performance compared to existing methods, we have to claim that the model currently still lacks the ability to generate satisfactory results for out-of-distribution unseen tasks. Recent study about large language models (LLMs) underscore that the effectiveness of LLMs largely depends on the quality, diversity, and quantity of the training data [43] . However, the task variety, model size (~30M), data scale (~140W) are not sufficient for GenLV. We hope that future studies involving larger models, broader tasks and data will yield more surprising results. 
conclusion: 
 In this paper, we introduce a low-level vision generalist model, GenLV, which is capable of addressing various low-level vision tasks. Our approach involves the design of an image processing framework based on visual task prompt, VPIP, which enables the model to accommodate multiple tasks with different target domains. In addition, this framework allows the flexibility to incorporate a powerful backbone network that is suitable for low-level vision tasks, resulting in superior image reconstruction quality. Experimental results demonstrate that our GenLV can effectively manage 30 diverse low-level vision tasks and significantly outperform existing methods quantitatively and qualitatively.  
: 
 the custom development of task-specific models often proves to be time-consuming and labor-intensive. In recent years, there has been a significant tendency in artificial intelligence technology towards the development of general models. In the realm of Natural Language Processing (NLP), Large Language Models (LLMs) such as the GPT-series [4, 33] have exhibited remarkable performance. Analogous research has also incrementally emerged in the field of computer vision, exemplified by models like Segment Anything Model (SAM) [24] and Track Anything Model (TAM) [45] . However, these explorations have mainly focused on perceptual high-level vision tasks, while research concerning general models for low-level vision tasks remains notably insufficient.Designing a general model for low-level vision presents significant challenges in several aspects. Firstly, due to the diversity of low-level vision tasks, different types of tasks may have distinct input and target domains (e.g., image restoration and stylization). Therefore, unifying a wide range of low-level vision tasks within a single model framework is challenging. Existing models that solve multiple low-level vision tasks are often designed to deal with a specific task category. For instance, AirNet [26] and Promp-tIR [35] focus on simultaneously processing several restoration tasks, i.e., restoring degraded images with several specific degradation to clean ones. However, these models are not capable to jointly process image feature extraction or stylization tasks that accept clean images as input. Secondly, for low-level vision tasks, the pixel-level image reconstruction and generation quality is the most important factor in evaluating the model effectiveness. However, existing general vision models frequently emphasize perceptual accuracy while neglecting the model's image reconstruction capability. Consequently, when tackling low-level vision tasks, the generated results frequently suffer from unsatisfactory image quality. For example, MAE-VQGAN [3] utilizes discrete feature representations for image reconstruction, often resulting in unacceptable structural differences between the reconstructed results and the input images [30] . Painter [40] and PromptGIP [30] employs ViTbased backbone network [17] . As a result, their results often lack fine details and occasionally exhibit blocking artifacts. Furthermore, solving a broader range of low-level vision tasks involves processing high-and low-frequency information simultaneously, which also poses a significant challenge to method design. Prompt-GIP [30] performs well within a specific task range, but may fail when the number of tasks increases, especially when more lowfrequency information processing (e.g., color and style) is involved. The Painter [40] model trained under the same task settings exhibits similar or even more severe issues, as shown in Figure 5 . We attribute these phenomena to the Masked Autoencoder (MAE)-based training paradigm, which makes the models sensitive to the content, especially low-frequency information, of the visual prompt.To address the above issues of constructing a general low-level vision model, we propose a new Visual task Prompt-based Image Processing (VPIP) framework. This framework comprises three key components: an end-to-end image processing main network, an encoder sub-network for processing the visual prompt, and an information interaction mechanism that introduces prompt information into the main network to guide task-specific processing. For the specific model design, we first employ a powerful backbone network, X-Restormer [9] , which is designed for general image restoration tasks, as the main network. Similar to PromptGIP and Painter, we represent different low-level vision tasks as inputtarget image pairs, serving as the visual task prompt. The prompt encoder sub-network processes the task prompt into latent representations. Inspired by text-to-image models, such as the Stable Diffusion model [38] , which incorporate the text prompt into the denoising process, we introduce visual task prompt representations into the main network using a cross-attention mechanism.The advantages of our proposed VPIP framework are manifold: 1) By utilizing the visual task prompt, our framework effectively solve the problem of varying input-target domains across different low-level vision tasks. 2) Not restricted to the MAE training paradigm, our approach mitigates the impact of prompt content on the task representation, enhancing the robustness of the task prompt. 3) Our framework allows flexibility in selecting backbone networks suitable for low-level vision tasks, thereby improving image reconstruction quality. 4) Instead of computing global attention for grid-like representations consisting of four image features of PromptGIP, our approach, based on cross-attention calculation, effectively reduces the attention computation cost.To evaluate the effectiveness of our method, we construct 30 diverse tasks to train the model. The trained low-level vision generalist model, namely GenLV, can successfully process the various tasks with different input and target domains, as show in Figure 1 . We also conduct comprehensive experiments to compare our method and existing approaches in Section 4. Extensive results demonstrate that GenLV significantly outperforms the other methods. 
 
 Appendix: We also conduct an extensive study on the model performance of different backbone network, and the detailed results are presented in the Supplementary Material. 


Decision: This paper has been thoroughly evaluated by 5 reviewers. They have all raised many questions in their comments, which have been nicely addressed by authors in the rebuttal -- it caused one score to raise. Everyone appreciates the originality of the approach, the quality of the writing, the extensiveness of the experiments. The contribution has a series of limitations, as highlighted in the reviews, but enough merits to pass the bar.

For these reasons, the recommendation is ACCEPT, poster.

Reviews: review: paper_summary: paper_summary: This paper proposes a low-level vision generalist model GenLV, to employ the visual task prompt to process tasks with different input-target domains. A prompt cross-attention mechanism is introduced to deal with the information interaction between the input and prompt information.
strength_weakness: strength_weakness: 1. The trained low-level vision generalist model GenLV, can successfully process the various tasks with different input and target domains.
2. The quantitative and qualitative comparisons demonstrate the superiority of the proposed framework.
limitations: limitations: 1. What is the inference time, compared with previous methods? If it is too large than previous methods, it may not be a fair comparison.

2. Any failure cases to show and discuss?
paper_summary: paper_summary: This paper introduces the Visual task Prompt-based Image Processing (VPIP) framework to address challenges in handling diverse low-level vision tasks. The generalist model GenLV, built on the VPIP framework and trained on 30 different tasks, shows significant improvements over existing approaches.
strength_weakness: strength_weakness: 1. The motivation for this work is reasonable, aiming to create a unified model framework for low-level tasks.

2. The paper is well-written, logically clear, and easy to understand.

3. Experiments show that the proposed model performs well.
limitations: limitations: 1. It is hoped that the authors can provide further introduction on the selection and definition of low-level tasks.

2. The authors claim that the proposed prompt cross-attention method can improve experimental efficiency; this requires additional comparative data for support.

3. Apart from image enhancement tasks, the comparison with specific task methods is lacking for several other tasks.

4. It would be beneficial to provide further details on the specific content of the task prompts.
paper_summary: paper_summary: The paper presents a novel framework named Visual task Prompt-based Image Processing (VPIP) for addressing a variety of low-level vision tasks. The proposed model, GenLV, integrates visual prompts into the main image processing network, allowing the model to handle different input-target domain pairs flexibly. The authors employ a powerful backbone network, X-Restormer, to ensure high-quality image reconstruction and introduce a cross-attention mechanism to facilitate information interaction between the input and the prompts. Experimental results on 30 diverse tasks demonstrate the superiority of GenLV over existing methods, both quantitatively and qualitatively.
strength_weakness: strength_weakness: 1.Innovative Framework: The VPIP framework is a significant contribution, providing a flexible solution to handle various low-level vision tasks using visual prompts.
2.High Performance: The proposed GenLV model shows substantial improvement over existing methods in both image restoration and enhancement tasks.
3.Comprehensive Evaluation: The authors conduct extensive experiments on a wide range of tasks, including image restoration, enhancement, edge detection, and stylization, showcasing the versatility and robustness of GenLV.
4.Detailed Analysis: The paper provides thorough quantitative and qualitative analyses, including ablation studies and visual comparisons, to support the effectiveness of the proposed approach.
5.Efficient Attention Mechanism: The introduction of the prompt cross-attention mechanism reduces computational cost and enhances the model's capability to process diverse tasks.
limitations: limitations: 1.Dependency on Task-Specific Prompts: The model's performance heavily relies on the quality and relevance of the visual prompts, which may limit its generalization to unseen tasks or domains.
2.Training Complexity: Training the GenLV model on 30 diverse tasks requires significant computational resources and careful tuning of hyperparameters, which might be a barrier for broader adoption.
3.Model Size and Data Requirements: The current model size and the scale of training data might not be sufficient for achieving even higher performance, suggesting the need for further research with larger models and datasets.
4.Detailed description of Method: The description integration of Prompt representation and Low-level Vision Generalist Model is not detailed and clear enough.
paper_summary: paper_summary: The paper designs and trains a low-level vision model based on in-context learning, capable of performing 30 different low-level vision tasks.
strength_weakness: strength_weakness: The methodology and process of the paper are clearly described and well-organized. 

The paper uses in-context learning to address low-level vision tasks. Compared to the latest similar method, such as PromptGIP, the proposed model covers a wider range of low-level tasks. And the author also provide experiment results on different backbones and task settings.
limitations: limitations: 1. What datasets were used for training the network? How many images in total were used? Were these datasets used simultaneously for training? On which datasets were the individual tasks tested, and what were the quantities? What are the image sizes? 

2. The paper presents results in terms of PSNR. Could the authors also provide some perceptual metrics, such as LPIPS, NIQE, BRISQUE, etc.? 

3. I am curious whether the authors have analyzed the relationships between different tasks. For example, is there any mutual benefit between tasks, or are there conflicts between some tasks? For instance, the output of image feature extraction and image enhancement tasks can be quite different. Would learning these two tasks simultaneously be more beneficial than learning them separately?
paper_summary: paper_summary: The paper presents GenLV, a versatile low-level vision model that excels at various image processing tasks through a Visual Task Prompt-based Image Processing (VPIP) framework. GenLV leverages a cross-attention mechanism and a powerful backbone network, X-Restormer, to process tasks with different input-target domains effectively. Trained on 30 diverse tasks, it outperforms existing methods in both quantitative and qualitative metrics, showcasing robustness across prompts. However, it has limitations with out-of-distribution tasks.
strength_weakness: strength_weakness: 1. It adapts to various low-level vision tasks through a visual task prompt approach.
2. It shows improved performance over existing methods for the tasks it was trained on.
3. The prompt cross-attention mechanism offers a more efficient way to integrate task-specific information.
limitations: limitations: 1.  It is crucial to conduct an ablation study that compares a single unified model, trained across all subtasks, with individual models trained for each specific subtask.  It is necessary to verify whether the unified model shows a decline in performance compared to the specialized model.

3.  The author's review of pertinent literature concerning task prompts appears to be inadequate, as works such as Emu edit, PowerPaint, and Diff-Plugin have been omitted.  It is essential to establish whether the task prompting methodology proposed in this paper holds any merits when compared to the aforementioned studies.

2.  Furthermore, it is recommended that the author provide an extensive presentation of image restoration results in real-world scenarios, accompanied by user studies.  It is worth noting that the domain of image restoration tends to have limited training and testing datasets.  Consequently, researchers should prioritize assessing the practical implications of their models in genuine situations.