nils-herrmann/clean_scirepeval huggingface dataset:
Inference With Collaborative Model for Interactive Tumor Segmentation in Medical Image Sequences

Segmenting organisms or tumors from medical data (e.g., computed tomography volumetric images, ultrasound, or magnetic resonance imaging images/image sequences) is one of the fundamental tasks in medical image analysis and diagnosis, and has received long-term attentions. This paper studies a novel computational framework of interactive segmentation for extracting liver tumors from image sequences, and it is suitable for different types of medical data. The main contributions are twofold. First, we propose a collaborative model to jointly formulate the tumor segmentation from two aspects: 1) region partition and 2) boundary presence. The two terms are complementary but simultaneously competing: the former extracts the tumor based on its appearance/texture information, while the latter searches for the palpable tumor boundary. Moreover, in order to adapt the data variations, we allow the model to be discriminatively trained based on both the seed pixels traced by the Lucas-Kanade algorithm and the scribbles placed by the user. Second, we present an effective inference algorithm that iterates to: 1) solve tumor segmentation using the augmented Lagrangian method and 2) propagate the segmentation across the image sequence by searching for distinctive matches between images. We keep the collaborative model updated during the inference in order to well capture the tumor variations over time. We have verified our system for segmenting liver tumors from a number of clinical data, and have achieved very promising results. The software developed with this paper can be found at
Detecting photographic composites using two-view geometrical constraints

In this work, we describe a new technique for detecting image composites by enforcing two-view geometrical constrains: H and F constraints on image pairs, where H denotes the planar homography matrix and F the fundamental matrix. Our approach detects fake regions efficiently on pictures taken at the same scene but with different camera configurations. Performance of this approach is demonstrated on real image pairs with visually plausible composites.
A Global Spatio-Temporal Representation for Action Recognition

In this paper we introduce an effective method to construct a global spatio-temporal representation for action recognition. This representation is inspired by the fact that human actions can be treated as 3D shapes induced by the silhouettes in the space-time volume. We estimate the silhouettes which contain detailed shape information of the action, and present an efficient sampling method to extract interest points along the silhouettes. The local interest point is represented by a spatio-temporal descriptor based on 2D DAISY. Our global space-time representation is the integration of these local descriptors in an order along the silhouette. In this manner, we not only utilize the static shape information, but also the spatial-temporal cue. We have obtained impressive results on publicly available action datasets.
Identifying Image Composites Through Shadow Matte Consistency

In this paper, we propose a framework for detecting tampered digital images based on photometric consistency of illumination in shadows. In particular, we formulate color characteristics of shadows measured by the shadow matte value. The shadow boundaries and the penumbra shadow region in an image are first extracted. Then a simple and efficient method is used to estimate shadow matte values of shadows. Our approach efficiently extracts these constraints from a single view of a target scene and makes use of them for the digital forgery detection. Experimental results on both simulated photos and visually plausible real images demonstrate the effectiveness of the proposed method.
Deep multi-context Network for FINE-GRAINED VISUAL RECOGNITION

In this paper, we tackle the FINE-GRAINED VISUAL RECOGNITION problem by proposing a deep multi-context framework. We employ deep Convolutional Neural Networks to model features of objects in images. Global context and local context are both taken into consideration, and are jointly modeled in a unified multi-context deep learning framework. To cleanse the relatively dirty data for training, a regional proposal method is designed to make the multi-context modeling suited for fine-grained visual recognition in the real world. Furthermore, recently proposed contemporary deep models are used, and their combination is investigated. Our approaches are evaluated on MSR-IRC 2016 and further assessed on the more complex validation set. The results show significant and consistent improvements over the baseline.
3D Object Transfer Between Non-Overlapping Videos

Given two video sequences of different scenes acquired with moving cameras, it is interesting to seamlessly transfer a 3D object from one sequence to the other. In this paper, we present a video-based approach to extract the alpha mattes of rigid or approximately rigid 3D objects from one or more source videos, and then geometrycorrectly transfer them into another target video of a different scene. Our framework builds upon techniques in camera pose estimation, 3D spatiotemporal video alignment, depth recovery, key-frame editing, natural video matting, and image-based rendering. Based on the explicit camera pose estimation, the camera trajectories of the source and target videos are aligned in 3D space. Combinied with the estimated dense depth information, this allows us to significantly relieve the burdens of key-frame editing and efficiently improve the quality of video matting. During the transfer, our approach not only correctly restores the geometric deformation of the 3D object due to the different camera trajectories, but also effectively retains the soft shadow and environmental lighting properties of the object to ensure that the augmenting object is in harmony with the target scene.
Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation.

Co-saliency detection aims at extracting the common salient regions from an image group containing two or more relevant images. It is a newly emerging topic in computer vision community. Different from the most existing co-saliency methods focusing on RGB images, this paper proposes a novel co-saliency detection model for RGBD images, which utilizes the depth information to enhance identification of co-saliency. First, the intra saliency map for each image is generated by the single image saliency model, while the inter saliency map is calculated based on the multi-constraint feature matching, which represents the constraint relationship among multiple images. Then, the optimization scheme, namely cross label propagation, is used to refine the intra and inter saliency maps in a cross way. Finally, all the original and optimized saliency maps are integrated to generate the final co-saliency result. The proposed method introduces the depth information and multi-constraint feature matching to improve the performance of co-saliency detection. Moreover, the proposed method can effectively exploit any existing single image saliency model to work well in co-saliency scenarios. Experiments on two RGBD co-saliency datasets demonstrate the effectiveness of our proposed model.
Multimodal Gesture Recognition Based on the ResC3D Network

Gesture recognition is an important issue in computer vision. Recognizing gestures with videos remains a challenging task due to the barriers of gesture-irrelevant factors. In this paper, we propose a multimodal gesture recognition method based on a ResC3D network. One key idea is to find a compact and effective representation of video sequences. Therefore, the video enhancement techniques, such as Retinex and median filter are applied to eliminate the illumination variation and noise in the input video, and a weighted frame unification strategy is utilized to sample key frames. Upon these representations, a ResC3D network, which leverages the advantages of both residual and C3D model, is developed to extract features, together with a canonical correlation analysis based fusion scheme for blending features. The performance of our method is evaluated in the Chalearn LAP isolated gesture recognition challenge. It reaches 67.71% accuracy and ranks the 1st place in this challenge.
Device identification based on H.264 cues

With the rapid development of multimedia techniques, the video can easily be edited and tempered. To determine whether a video has been modified from its original recording has become a critical challenge. Thanks to the flexibility of H.264, we observe that different H.264 video parameters are used by various manufacturers. We propose a video capturing device identification approach by exploring the H.264 cues. Two main video dependent features are exploited: 1) H.264 encoding parameters which rely on motion-vector analysis, entropy coding, encoding quality and level, GOP information, etc. 2) The other is the video encapsulation formats. The former is to judge whether the video is taken by cellphone, while the latter is utilized to identify the video source. In the experimental part, we evaluate our proposed approach on our collected video dataset, and demonstrate the efficiency of our method.
Shape-Preserving Object Depth Control for Stereoscopic Images

In the field of 3-D technology, it is interesting as well as meaningful issue to control object depth in 3-D space. Recently, some depth control methods for stereoscopic images have been proposed, which usually employ depth map or directly process color images to implement depth control. There are two main disadvantages for these methods. First, the results of these methods usually suffer from object deformation and holes. Second, these methods are prone to cause undesired object size changing in 3-D space. To address these issues, we propose a shape-preserving object depth control method for stereoscopic images. First, a novel depth mapping model is presented for calculating the ideal coordinates of the key points in depth control, so that the shape of the object can be well preserved. Afterward, the image content-based constraints are used to further preserve the structure of the object and its background. Finally, the warping technology is introduced to deal with images optimally as well as to avoid holes. Experimental results show that the proposed method can control object depth and preserve the shape of the object effectively without sensible background distortion.
Augmenting Image Descriptions Using Structured Prediction Output

The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.
Camera calibration without metric information using 1D objects

This paper addresses the problem of calibrating a pin-hole camera from images of 1D objects. Assuming a unit aspect ratio and zero skew, we introduce a novel and simple approach that uses four observations of a 1D object and requires no information about the distances between the points on the object. This is in contrast to existing methods that use two images, but impose more restrictive configurations that require measured distances on the calibrating object. The key features of the proposed technique are its simplicity and ease of use due to the lack of need for any metric information. To demonstrate the effectiveness of the algorithm, we present the processing results on synthetic and real images.
A new framework for video cut and paste

In this paper, we describe a novel framework to cut and paste objects among different video shots. First, based on geometric analysis of the camera motion of a shot, we classify the shot into either simple camera motion shot, e.g. panning, tracking and zooming, or complex camera motion shot, e.g. hand-held shots. Next, for the simple camera motion shots, we temporally align them captured by cameras undergoing the same motions based on the quantization in the geometric analysis. In the case of complex camera motion, we recover the camera poses for each frame of both the source and target shots and, thus, for each frame in the target shot, we find the corresponding source frame with the closest viewing direction. Then, the foreground objects in the source shots are automatically cut by combining the merits of motion layer segmentation and alpha matting techniques. Finally, the extracted foreground mattes can be directly blended into the corresponding target frames for simple motion shots. For complex shots, combining the estimated rough depths of the foreground objects, foreground layers are rendered and blended into the target frames. Results on various content types, e.g. home videos and feature films, and different camera motions are reported for validating the proposed framework
Synthesizing Reflections of Inserted Objects

The aim of reflection synthesis of inserted objects is to generate reflections which would be seen by the same camera capturing the target scene and be reflected by the true reflective media in the target scene. This problem is inherently difficult because we are typically given only several, sometimes even just one, views of the target scene. In this paper, we explore the geometric constraints to synthesize geometrically correct reflections. We also demonstrate how to constraint the synthesized reflections to be photometrically consistent with those in the original target views. The proposed method, therefore, advances the current image-based compositing techniques one step further toward the scenes with variations in both lighting conditions and viewpoints. We demonstrate our approach for real scenes
Flexible Multi-View Dimensionality Co-Reduction

Dimensionality reduction aims to map the high-dimensional inputs onto a low-dimensional subspace, in which the similar points are close to each other and vice versa. In this paper, we focus on unsupervised dimensionality reduction for the data with multiple views, and propose a novel method, called Multi-view Dimensionality co-Reduction. Our method flexibly exploits the complementarity of multiple views during the dimensionality reduction and respects the similarity relationships between data points across these different views. The kernel matching constraint based on Hilbert-Schmidt Independence Criterion enhances the correlations and penalizes the disagreement of different views. Specifically, our method explores the correlations within each view independently, and maximizes the dependence among different views with kernel matching jointly. Thus, the locality within each view and the consistence between different views are guaranteed in the subspaces corresponding to different views. More importantly, benefiting from the kernel matching, our method need not depend on a common low-dimensional subspace, which is critical to reduce the influence of the unbalanced dimensionalities of multiple views. Specifically, our method explicitly produces individual low-dimensional projections for individual views, which could be applied for new coming data in the out-of-sample manner. Experiments on both clustering and recognition tasks demonstrate the advantages of the proposed method over the state-of-the-art approaches.
Configuring Mixed Reality Environment

We present a practical framework for registering a mixed reality (MR) environment of an arbitrary number of agents. Each agent consists of a head mounted display (HMD), which consists of a pair of stereo cameras. Each agent is assumed to be moving freely in 3D space and multiple HMDs need not have a common field of view (FoV). We show that the plane at infinity and a common vertical vanishing point can be use to determine the exact orientation of all HMDs with respect to each other, and establish a common reference frame. Our method generalizes previous work which considers restricted camera motions. Using minimal assumptions, we are able to successfully demonstrate promising results on real data
A Study on Detection and Recovery of Speech Signal Tampering

In this paper, a watermark scheme for detecting and recovering the tampering of speech signal is proposed. We embed an approximate version of the origin information into the less significant bits (LSBs). When the watermarked signal is tampered, we can localize the tampered area and exclude from the area, recovery the original one from the approximate version. Based the embedded information, we estimate the origin signal by solving a liner equation with Least Square QR-factorization (LSQR) method. The result of an informal listening test shows that 83.7% listening material is intelligible after recovery with 20% part tampered.
Symmetry Constraint for Foreground Extraction

Symmetry as an intrinsic shape property is often observed in natural objects. In this paper, we discuss how explicitly taking into account the symmetry constraint can enhance the quality of foreground object extraction. In our method, a symmetry foreground map is used to represent the symmetry structure of the image, which includes the symmetry matching magnitude and the foreground location prior. Then, the symmetry constraint model is built by introducing this symmetry structure into the graph-based segmentation function. Finally, the segmentation result is obtained via graph cuts. Our method encourages objects with symmetric parts to be consistently extracted. Moreover, our symmetry constraint model is applicable to weak symmetric objects under the part-based framework. Quantitative and qualitative experimental results on benchmark datasets demonstrate the advantages of our approach in extracting the foreground. Our method also shows improved results in segmenting objects with weak, complex symmetry properties.
Output Feature Augmented Lasso

Lasso simultaneously conducts variable selection and supervised regression. In this paper, we extend Lasso to multiple output prediction, which belongs to the categories of structured learning. Though structured learning makes use of both input and output simultaneously, the joint feature mapping in current framework of structured learning is usually application-specific. As a result, ad hoc heuristics have to be employed to design different joint feature mapping functions for different applications, which results in the lackness of generalization ability for multiple output prediction. To address this limitation, in this paper, we propose to augment Lasso with output by decoupling the joint feature mapping function of traditional structured learning. The contribution of this paper is three-fold: 1) The augmented Lasso conducts regression and variable selection on both the input and output features, and thus the learned model could fit an output with both the selected input variables and the other correlated outputs. 2) To be more general, we set up nonlinear dependencies among output variables by generalized Lasso. 3) Moreover, the Augmented Lagrangian Method (ALM) with Alternating Direction Minimizing (ADM) strategy is used to find the optimal model parameters. The extensive experimental results demonstrate the effectiveness of the proposed method.
The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations Through Sexual Evolutionary Synthesis

Evolutionary deep intelligence was recently proposed as a method for achieving highly efficient deep neural network architectures over successive generations. Drawing inspiration from nature, we propose the incorporation of sexual evolutionary synthesis. Rather than the current asexual synthesis of networks, we aim to produce more compact feature representations by synthesizing more diverse and general-izable offspring networks in subsequent generations via the combination of two parent networks. Experimental results were obtained using the MNIST and CIFAR-10 datasets, and showed improved architectural efficiency and comparable testing accuracy relative to the baseline asexual evolutionary neural networks. In particular, the network synthesized via sexual evolutionary synthesis for MNIST had approximately double the architectural efficiency (cluster efficiency of 34.29× and synaptic efficiency of 258.37×) in comparison to the network synthesized via asexual evolutionary synthesis, with both networks achieving a testing accuracy of ~97%.
Hierarchical network flow phase unwrapping

The well-studied interferometric synthetic aperture radar (InSAR) problem for digital elevation map generation involves the derivation of topography from the radar phase. The topography is a function of the full phase, whereas the measured phase is known modulo 2/spl pi/, necessitating the process of recovering full phase values via phase unwrapping. This mathematical process becomes difficult through the presence of noise and phase discontinuities. This paper is motivated by research which models phase unwrapping as a network-flow minimization problem. A major limitation is that often a substantial computational effort is required to find solutions. Commonly, these phase images are huge (/spl Gt/10 million pixels), and obviously the sheer size of the problem itself makes phase unwrapping challenging. This paper addresses the development of a computationally efficient hierarchical algorithm, based on a "divide-and-conquer" approach. We have shown that the phase-unwrapping problem can first be partitioned into independent phase-unwrapping subproblems, which can further be recombined to produce the unwrapped phase. Interestingly, the recombination step itself can be interpreted as an unwrapping problem, for which a modified network-flow solution applies! In short, this paper develops a parallelization of the network-flow algorithm, allowing images of virtually unlimited size to be unwrapped and leading to dramatic decreases in the algorithm execution time.
Parametric contour estimation by simulated annealing

Virtually all implementations of simulated annealing are simplified by assuming discrete unknowns, however continuous-parameter annealing has many potential applications to image processing. Widely scattered problems such as formant tracking, boundary estimation and phase- unwrapping can all be approached as the annealed minimizations of continuous B-spline parameters. The benefits of simulated annealing are well known, including an insensitivity to initial conditions and the ability to solve problems with many local minima. Discrete variable annealing has seen broad application, however continuous-variable annealing is limited by the computational challenge of Gibbs sampling. In this paper we develop efficient approaches to sampling, illustrated in the context of contour tracking in noisy images.