Supervised deep learning techniques have achieved great success in various fields due to getting rid of the limitation of handcrafted representations. However, most previous image retargeting algorithms still employ fixed design principles such as using gradient map or handcrafted features to compute saliency map, which inevitably restricts its generality. Deep learning techniques may help to address this issue, but the challenging problem is that we need to build a large-scale image retargeting dataset for the training of deep retargeting models. However, building such a dataset requires huge human efforts. In this paper, we propose a novel deep cyclic image retargeting approach, called Cycle-IR, to firstly implement image retargeting with a single deep model, without relying on any explicit user annotations. Our idea is built on the reverse mapping from the retargeted images to the given input images. If the retargeted image has serious distortion or excessive loss of important visual information, the reverse mapping is unlikely to restore the input image well. We constrain this forward-reverse consistency by introducing a cyclic perception coherence loss. In addition, we propose a simple yet effective image retargeting network (IRNet) to implement the image retargeting process. Our IRNet contains a spatial and channel attention layer, which is able to discriminate visually important regions of input images effectively, especially in cluttered images. Given arbitrary sizes of input images and desired aspect ratios, our Cycle-IR can produce visually pleasing target images directly. Extensive experiments on the standard RetargetMe dataset show the superiority of our Cycle-IR. In addition, our Cycle-IR outperforms the Multiop method and obtains the best result in the user study. Code is available at this https URL.
Different from the seam carving method in a discrete way, Panozzo @math @cite_20 propose to continuously transform the input image into the target image. Specifically, they overlay a uniform grid on the input image and assign different scaling factors to each grid cell. The scaling factors of important regions are large, and conversely, small. As a result, those important regions are preserved as much as possible, while less important regions are shrinking obviously. Wang @math @cite_31 propose a scale-and-stretch warping method. They calculate optimal scaling factors for image regions based on the edge and saliency map. Jin @math @cite_29 cast image retargeting problem into a quadratic program based on a triangular mesh. All these conventional approaches need to calculate an importance map based on low-level visual cues or other human priors. The poor quality of importance map significantly degrades the performance of these methods, which seriously limits their generality.
@cite_31 @cite_29 @cite_20
2002885083 2055307271
Saliency detection plays important roles in many image processing applications, such as regions of interest extraction and image resizing. Existing saliency detection models are built in the uncompressed domain. Since most images over Internet are typically stored in the compressed domain such as joint photographic experts group (JPEG), we propose a novel saliency detection model in the compressed domain in this paper. The intensity, color, and texture features of the image are extracted from discrete cosine transform (DCT) coefficients in the JPEG bit-stream. Saliency value of each DCT block is obtained based on the Hausdorff distance calculation and feature map fusion. Based on the proposed saliency detection model, we further design an adaptive image retargeting algorithm in the compressed domain. The proposed image retargeting algorithm utilizes multioperator operation comprised of the block-based seam carving and the image scaling to resize images. A new definition of texture homogeneity is given to determine the amount of removal block-based seams. Thanks to the directly derived accurate saliency information from the compressed domain, the proposed image retargeting algorithm effectively preserves the visually important regions for images, efficiently removes the less crucial regions, and therefore significantly outperforms the relevant state-of-the-art algorithms, as demonstrated with the in-depth analysis in the extensive experiments. We present a "scale-and-stretch" warping method that allows resizing images into arbitrary aspect ratios while preserving visually prominent features. The method operates by iteratively computing optimal local scaling factors for each local region and updating a warped image that matches these scaling factors as closely as possible. The amount of deformation of the image content is guided by a significance map that characterizes the visual attractiveness of each pixel; this significance map is computed automatically using a novel combination of gradient and salience-based measures. Our technique allows diverting the distortion due to resizing to image regions with homogeneous content, such that the impact on perceptually important features is minimized. Unlike previous approaches, our method distributes the distortion in all spatial directions, even when the resizing operation is only applied horizontally or vertically, thus fully utilizing the available homogeneous regions to absorb the distortion. We develop an efficient formulation for the nonlinear optimization involved in the warping function computation, allowing interactive image resizing. Image retargeting adjusts images to arbitrary sizes such that they can be viewed on different displays. Content-aware image retargeting has been receiving increased attention. In particular, researchers have improved a patch-wise scaling method for image retargeting at the object level. The scaling partitions the image into rectangular patches of adaptive sizes, which are comparable to the sizes of the salient objects in the image. This partitioning is based on a visual-saliency map; accordingly, the method labels the patches as important or unimportant. Then, the method scales the important patches as uniformly as possible and stretches or squeezes the unimportant patches to fit the target size. A patch-based image-similarity measure finds the optimal set of scaling factors. In experiments, the improved method performed well for three image types: lines and edges, foreground objects, and geometric structures. We present a novel method for content-aware image resizing based on optimization of a well-defined image distance function, which preserves both the important regions and the global visual effect (the background or other decorative objects) of an image. The method operates by joint use of seam carving and image scaling. The principle behind our method is the use of a bidirectional similarity function of image Euclidean distance (IMED), while cooperating with a dominant color descriptor (DCD) similarity and seam energy variation. The function is suitable for the quantitative evaluation of the resizing result and the determination of the best seam carving number. Different from the previous simplex-mode approaches, our method takes the advantages of both discrete and continuous methods. The technique is useful in image resizing for both reduction retargeting and enlarging. We also show that this approach can be extended to indirect image resizing.