Supervised deep learning techniques have achieved great success in various fields due to getting rid of the limitation of handcrafted representations. However, most previous image retargeting algorithms still employ fixed design principles such as using gradient map or handcrafted features to compute saliency map, which inevitably restricts its generality. Deep learning techniques may help to address this issue, but the challenging problem is that we need to build a large-scale image retargeting dataset for the training of deep retargeting models. However, building such a dataset requires huge human efforts. In this paper, we propose a novel deep cyclic image retargeting approach, called Cycle-IR, to firstly implement image retargeting with a single deep model, without relying on any explicit user annotations. Our idea is built on the reverse mapping from the retargeted images to the given input images. If the retargeted image has serious distortion or excessive loss of important visual information, the reverse mapping is unlikely to restore the input image well. We constrain this forward-reverse consistency by introducing a cyclic perception coherence loss. In addition, we propose a simple yet effective image retargeting network (IRNet) to implement the image retargeting process. Our IRNet contains a spatial and channel attention layer, which is able to discriminate visually important regions of input images effectively, especially in cluttered images. Given arbitrary sizes of input images and desired aspect ratios, our Cycle-IR can produce visually pleasing target images directly. Extensive experiments on the standard RetargetMe dataset show the superiority of our Cycle-IR. In addition, our Cycle-IR outperforms the Multiop method and obtains the best result in the user study. Code is available at this https URL.
: Because the visual quality of retargeted images highly depends on the subjective assessment, it is hard to build a large-scale image retargeting dataset for supervisory learning a deep model. Thus, there are few supervised retargeting methods. Cho @math . @cite_10 introduce deep learning technique into the image retargeting task. Their deep model learns a shift map to implement pixel-wise mapping from the source image to the target image. They define a loss function containing two terms: content loss and structure loss. The content loss enables the retargeted image outputted by the deep network to have the same class as the source image. Therefore, their approach requires image-level annotations to construct the content loss. This approach may fail to deal with those images that do not belong to any class in the training set. Besides, in order to reduce distortions in the target image, the method proposes to use 1D duplicate convolution to smooth the learned shift map. This operation is helpful for reducing visual artifacts in the target image but requires the input image to be a fixed size.
@cite_10
2744093089
This paper proposes a weakly- and self-supervised deep convolutional neural network (WSSDCNN) for content-aware image retargeting. Our network takes a source image and a target aspect ratio, and then directly outputs a retargeted image. Retargeting is performed through a shift map, which is a pixel-wise mapping from the source to the target grid. Our method implicitly learns an attention map, which leads to a content-aware shift map for image retargeting. As a result, discriminative parts in an image are preserved, while background regions are adjusted seamlessly. In the training phase, pairs of an image and its image-level annotation are used to compute content and structure losses. We demonstrate the effectiveness of our proposed method for a retargeting application with insightful analyses. Image retargeting aims to adapt images to displays of small sizes and different aspect ratios. Effective retargeting requires emphasizing the important content while retaining surrounding context with minimal visual distortion. In this paper, we present such an effective image retargeting method using saliency-based mesh parametrization. Our method first constructs a mesh image representation that is consistent with the underlying image structures. Such a mesh representation enables easy preservation of image structures during retargeting since it captures underlying image structures. Based on this mesh representation, we formulate the problem of retargeting an image to a desired size as a constrained image mesh parametrization problem that aims at finding a homomorphous target mesh with desired size. Specifically, to emphasize salient objects and minimize visual distortion, we associate image saliency into the image mesh and regard image structure as constraints for mesh parametrization. Through a stretch-based mesh parametrization process we obtain the homomorphous target mesh, which is then used to render the target image by texture mapping. The effectiveness of our algorithm is demonstrated by experiments. Image retargeting adjusts images to arbitrary sizes such that they can be viewed on different displays. Content-aware image retargeting has been receiving increased attention. In particular, researchers have improved a patch-wise scaling method for image retargeting at the object level. The scaling partitions the image into rectangular patches of adaptive sizes, which are comparable to the sizes of the salient objects in the image. This partitioning is based on a visual-saliency map; accordingly, the method labels the patches as important or unimportant. Then, the method scales the important patches as uniformly as possible and stretches or squeezes the unimportant patches to fit the target size. A patch-based image-similarity measure finds the optimal set of scaling factors. In experiments, the improved method performed well for three image types: lines and edges, foreground objects, and geometric structures. We propose the space of axis-aligned deformations as the meaningful space for content-aware image retargeting. Such deformations exclude local rotations, avoiding harmful visual distortions, and they are parameterized in 1D. We show that standard warping energies for image retargeting can be minimized in the space of axis-aligned deformations while guaranteeing that bijectivity constraints are satisfied, leading to high-quality, smooth and robust retargeting results. Thanks to the 1D parameterization, our method only requires solving a small quadratic program, which can be done within a few milliseconds on the CPU with no precomputation overhead. We demonstrate how the image size and the saliency map can be changed in real time with our approach, and present results on various input images, including the RetargetMe benchmark. We compare our results with six other algorithms in a user study to demonstrate that the space of axis-aligned deformations is suitable for the problem at hand. © 2012 Wiley Periodicals, Inc.