nils-herrmann/clean_scirepeval huggingface dataset:
Reliability Map Estimation for CNN-Based Camera Model Attribution

Among the image forensic issues investigated in the last few years, great attention has been devoted to blind camera model attribution. This refers to the problem of detecting which camera model has been used to acquire an image by only exploiting pixel information. Solving this problem has great impact on image integrity assessment as well as on authenticity verification. Recent advancements that use convolutional neural networks (CNNs) in the media forensic field have enabled camera model attribution methods to work well even on small image patches. These improvements are also important for determining forgery localization. Some patches of an image may not contain enough information related to the camera model (e.g., saturated patches). In this paper, we propose a CNN-based solution to estimate the camera model attribution reliability of a given image patch. We show that we can estimate a reliabilitymap indicating which portions of the image contain reliable camera traces. Testing using a well known dataset confirms that by using this information, it is possible to increase small patch camera model attribution accuracy by more than 8% on a single patch.
3D object modeling with a voxelset carving approach

In the past few years several systems for object reconstruction based on the analysis of 2D images have been proposed. In order for such systems to be of practical use, the 3D data extraction process is expected to be fast and reliable. In this paper we propose a general approach for the reconstruction of complete 3D objects based on a mesh fusion algorithm. Every surface patch is obtained as a depth map using an algorithm based on graph cuts theory. Each depth map is then triangulated before using it in a fusion algorithm based on a voxel-set carving approach. The result of the process is a closed mesh representing the object surface with sub-voxel resolution.
Video Quality Assessment from the Perspective of a Network Service Provider

In this paper we consider the problem of estimating the quality of a decoded video that is transmitted over an unreliable packet-switched network. The goal is to allow the encoder to calculate the expected decoder distortion without any direct comparison between the original and the decoded sequence. This work is based on ROPE algorithm for the video quality estimate. ROPE calculates, at the encoder side, the expected decoded video distortion: therefore it is reasonably performing in estimating the average obtained distortion. The main question this paper would like to answer is how accurate ROPE distortion estimate is in the case of a single realization, i.e., in the case of distortion estimate for a single user
Hash-Based Motion Modeling in Wyner-Ziv Video Coding

Generally, distributed video coding (DVC) schemes perform motion estimation at the decoder side, without the current frame being available. In order to generate the side-information reliably, one solution consists in allocating a limited bit budget to send a hash of the current frame. At the decoder, this auxiliary hash is used to perform motion estimation. This paper studies the accuracy of hash-based motion estimation and compares it to conventional encoder-side motion estimation. We show that, at low rates, the very limited bit-budget of the hash does not ensure a reliable motion estimation, while at medium to high rates the motion accuracy is comparable with the finite precision used to represent motion vectors. Then, we derive the rate-distortion characteristic, which combines the cost of encoding the hash and the prediction residuals after decoder-side motion compensation. We show that, at high rates, hash-based motion modeling can virtually achieve the same coding efficiency as motion-compensated predictive coding. Instead, at medium-to-low rates we observe a significant coding loss. Experimental results on real video sequences validate the results of the proposed model.
Action modeling with volumetric data

In this paper we propose and test an action recognition algorithm in which the images of the scene captured by a significant number of cameras are first used to generate a volumetric representation of a moving human body in terms of voxsets by means of volumetric intersection. The recognition stage is then performed directly on 3D data, allowing the system to avoid critical problems like viewpoint dependence and motion trajectory variability. Suitable features are extracted from the voxset representing the body and fed to a classical hidden Markov model to produce a finite-state description of the motion.
Combining MCTF with distributed source coding

Motion compensated temporal filtering (MCTF) has proved to be an efficient coding tool in the design of open-loop scalable video codecs. In this paper we propose a MCTF video coding scheme based on lifting where the prediction step is implemented using PRISM (power efficient, robust, high compression syndrome-based multimedia coding), a video coding framework built on distributed source coding principles. We study the effect of integrating the update step at the encoder or at the decoder side. We show that the latter approach allows improving the quality of the side information exploited during decoding. We present the analytical results obtained by modeling the video signal along the motion trajectories as an AR(1) process showing that the update step at the decoder allows to half the contribution of the quantization noise. We also include experimental results with real video data that demonstrate the potential of this approach when the video sequences are coded at low bitrates.
Unsupervised feature learning for bootleg detection using deep learning architectures

The widespread diffusion of portable devices capable of capturing high-quality multimedia data, together with the rapid proliferation of media sharing platforms, has determined an incredible growth of user-generated content available online. Since it is hard to strictly regulate this trend, illegal diffusion of copyrighted material is often likely to occur. This is the case of audio bootlegs, i.e., concerts illegally recorded and redistributed by fans. In this paper, we propose a bootleg detector, with the aim of disambiguating between: i) bootlegs unofficially recorded; ii) live concerts officially published; iii) studio recordings from officially released albums. The proposed method is based on audio feature analysis and machine learning techniques. We exploit a deep learning paradigm to extract highly characterizing features from audio excerpts, and a supervised classifier for detection. The method is validated against a dataset of nearly 500 songs, and results are compared to a state-of-the-art detector. The conducted experiments confirm the capability of deep learning techniques to outperform classic feature extraction approaches.
Hybrid coding of visual content and local image features

Distributed visual analysis applications, such as mobile visual search or Visual Sensor Networks (VSNs) require the transmission of visual content on a bandwidth-limited network, from a peripheral node to a processing unit. Traditionally, a “Compress-Then-Analyze” approach has been pursued, in which sensing nodes acquire and encode the pixel-level representation of the visual content, that is subsequently transmitted to a sink node in order to be processed. This approach might not represent the most effective solution, since several analysis applications leverage a compact representation of the content, thus resulting in an inefficient usage of network resources. Furthermore, coding artifacts might significantly impact the accuracy of the visual task at hand. To tackle such limitations, an orthogonal approach named “Analyze-Then-Compress” has been proposed . According to such a paradigm, sensing nodes are responsible for the extraction of visual features, that are encoded and transmitted to a sink node for further processing. In spite of improved task efficiency, such paradigm implies the central processing node not being able to reconstruct a pixel-level representation of the visual content. In this paper we propose an effective compromise between the two paradigms, namely “Hybrid-Analyze-Then-Compress” (HATC) that aims at jointly encoding visual content and local image features. Furthermore, we show how a target tradeoff between image quality and task accuracy might be achieved by accurately allocating the bitrate to either visual content or local features.
Modeling closed surfaces in a multi-resolution fashion

In this paper we discuss two image-based 3D modeling methods based on a multi-resolution evolution of a volumetric function's levelset. In the former the role of the levelset implosion is to fuse ("sew" and "stitch") together several partial reconstructions (depth maps) into a closed model. In the latter the levelset's implosion is steered directly by the texture mismatch between views. Both solutions share the characteristic of operating in an adaptive multi-resolution fashion, in order to boost computational efficiency and robustness.
A Dimensional Contextual Semantic Model for music description and retrieval

Several paradigms for high-level music descriptions have been proposed to develop effective system for browsing and retrieving musical content in large repositories. Such paradigms are based on either categorical or dimensional models. The interest in dimensional models has recently grown a great deal, as they define a semantic relation between concepts through graded descriptions. One problem that affects semantic descriptions is the ambiguity that often arises from using the same descriptor in different contexts. In order to overcome this difficulty, it is important to model and address polysemy, which is the property of words to take on different meanings depending on the use-context. In this paper we propose a Dimensional Contextual Semantic Model for defining semantic relations among descriptors in a context-aware fashion. This model is here used for developing a semantic music search engine. In order to evaluate the effectiveness of our model, we compare this engine with two systems that are based on different description models.
The ORIGAMI project: advanced tools and techniques for high-end mixing and interaction between real and virtual content

ORIGAMI is an EU-funded IST project with the goal to develop advanced tools and new production techniques for high-quality mixing of real and virtual content for film and TV productions. In particular, the project proposes new techniques for 3D camera calibration and tracking and for 3D scene reconstruction and plenoptic modelling. Such solutions will support a variety of different object classes and application scenarios, including single-camera modelling of static environments, and multi-camera modelling of dynamic studio scenes. The developed methods will be integrated into a new software framework, which is being developed within the project. ORIGAMI also proposes novel techniques that enable real and virtual actors to interact with each other and with the real and virtual elements of the extended set.
Robust Interactive Image Segmentation Using Convex Active Contours

The state-of-the-art interactive image segmentation algorithms are sensitive to the user inputs and often unable to produce an accurate boundary with a small amount of user interaction. They frequently rely on laborious user editing to refine the segmentation boundary. In this paper, we propose a robust and accurate interactive method based on the recently developed continuous-domain convex active contour model. The proposed method exhibits many desirable properties of an effective interactive image segmentation algorithm, including robustness to user inputs and different initializations, the ability to produce a smooth and accurate boundary contour, and the ability to handle topology changes. Experimental results on a benchmark data set show that the proposed tool is highly effective and outperforms the state-of-the-art interactive image segmentation algorithms.
Re-examination of applying wavelet based progressive image coder for 3D semi-regular mesh compression

The latest wavelet based 3D mesh coding schemes convert an irregular mesh into a semi-regular mesh and directly apply the zerotree-like image coders to compress the wavelet vectors generated in the remeshing process. The major problem of such type of approaches is that the particular properties of semi-regular meshes are not being considered in the zerotree-like image coders. In this paper, we propose an improved wavelet based 3D mesh coder. The basic idea is to introduce a preprocessing step to scale up the vector wavelets generated in remeshing so that the inherent dependency of wavelets can be truly understood by the zerotree-like image compression algorithms. The weights used in the scaling process are carefully designed through thoroughly analyzing the distortions of wavelets at different refinement levels. Experimental results show that our proposed mesh coder significantly outperforms the state-of-the-art wavelet based 3D mesh compression scheme.
Interactive Mesh Cutting Using Constrained Random Walks

This paper considers the problem of interactively finding the cutting contour to extract components from an existing mesh. First, we propose a constrained random walks algorithm that can add constraints to the random walks procedure and thus allows for a variety of intuitive user inputs. Second, we design an optimization process that uses the shortest graph path to derive a nice cut contour. Then a new mesh cutting algorithm is developed based on the constrained random walks plus the optimization process. Within the same computational framework, the new algorithm provides a novel user interface for interactive mesh cutting that supports three typical user inputs and also their combinations: 1) foreground/background seed inputs: the user draws strokes specifying seeds for “foreground” (i.e., the part to be cut out) and “background” (i.e., the rest); 2) soft constraint inputs: the user draws strokes on the mesh indicating the region which the cuts should be made nearby; and 3) hard constraint inputs: the marks which the cutting contour must pass. The algorithm uses feature sensitive metrics that are based on surface geometric properties and cognitive theory. The integration of the constrained random walks algorithm, the optimization process, the feature sensitive metrics, and the varieties of user inputs makes the algorithm intuitive, flexible, and effective as well. The experimental examples show that the proposed cutting method is fast, reliable, and capable of producing good results reflecting user intention and geometric attributes.
Accurate and Efficient Approximation of Clothoids Using Bézier Curves for Path Planning

An accurate and efficient clothoid approximation approach is presented in this paper using Bézier curves based on the minimization of curvature profile difference. Compared with existing methods, the proposed approach is able to guarantee higher order geometric continuity with smaller approximation error in terms of position, orientation, and curvature. The approximation scheme takes place in three stages. First, a subset of clothoids with specific winding angle constraints referred to as elementary clothoids is approximated using quintic Bézier curves. Then, a basic clothoid defined in the first quadrant is formulated, which is composed of a series of transformed elementary clothoids. An adaptive sampling stra-tegy is applied to ensure that the resulting Bézier segments are computed within a specified accuracy and all the required information can be obtained offline and stored in a lookup table. Finally, a general clothoid with arbitrary parameters can be conveniently approximated based on the lookup table through appropriate geometric transformations. A comparison with the recent circular interpolation and rational Bézier curve based approximation shows that the proposed approach is able to achieve equivalent or greater computational efficiency in most scenarios.
Multiple Human Identification and Cosegmentation: A Human-Oriented CRF Approach With Poselets

Localizing, identifying, and extracting humans with consistent appearance jointly from a personal photo stream is an important problem and has wide applications. The strong variations in foreground and background and irregularly occurring foreground humans make this realistic problem challenging. Inspired by advancements in object detection, scene understanding, and image cosegmentation, we explore explicit constraints to label and segment human objects rather than other nonhuman objects and “stuff.” We refer to such a problem as multiple human identification and cosegmentation (MHIC). To identify specific human subjects, we propose an efficient human instance detector by combining an extended color line model with a poselet-based human detector. Moreover, to capture high-level human shape information, a novel soft shape cue is proposed. It is initialized by the human detector, then further enhanced through a generalized geodesic distance transform, and finally refined with a joint bilateral filter. We also propose to capture the rich feature context around each pixel by using an adaptive cross-region data structure, which gives a higher discriminative power than a single pixel-based estimation. The high-level object cues from the detector and the shape are then integrated with the low-level pixel cues and midlevel contour cues into a principled conditional random field (CRF) framework, which can be efficiently solved by using fast graph cut algorithms. We evaluate our method over a newly created NTU-MHIC human dataset, which contains 351 images with manually annotated groundtruth segmentation. Both visual and quantitative results demonstrate that our method achieves state-of-the-art performance for the MHIC task.
Adaptive T-spline surface approximation of triangular meshes

Digital 3D models represented by triangular meshes are now broadly available and they are becoming a new multimedia data type after sound, images and video. For many applications there is a need to convert such mesh models into spline surfaces. This paper describes an adaptive algorithm that automatically converts a triangular mesh into a T-spline surface based on optimization techniques. The key issues of the algorithm include the parameterization of input data points, the determination of topology of the T-spline surface, and the calculation of the least squares T-spline surface control points. Some examples are provided to demonstrate the efficiency of the algorithm.
Constrained active contours for boundary refinement in interactive image segmentation

The state-of-the-art interactive image segmentation algorithms are often not able to produce accurate segmentation results with one-shot user input, and they frequently rely on laborious user editing to refine the segmentation boundary. In this paper, we propose a constrained active contour method for boundary refinement, which can be used to improve the segmentation results of many existing region-based interactive segmentation algorithms. Our constrained active contour model exhibits many desired properties for a good boundary refinement tool, including the robustness to user inputs, the ability to produce a smooth and accurate boundary contour, and the ability to handle topology changes. Experimental results show that the proposed refinement tool is highly effective and can significantly improve initial segmentation results without additional user inputs.
Poselet-based multiple human identification and cosegmentation

Localizing, identifying and extracting human groups with consistent appearance jointly from a personal photo stream is an important problem and has wide applications. Inspired by recent advances in object detection, scene understanding and image cosegmentation, in this paper we explore explicit constraints to label and segment human objects rather than other non-human objects and “stuff”. We propose a novel soft human shape cue, which is initialized by color line poselet-based human part detection, further processed through a generalized geodesic distance transform, and refined finally with a joint bilateral filter. Such a high-level object cue is then integrated with other low-level unary and pairwise terms into a principled conditional random field framework, which can be efficiently solved by fast graph cut algorithms. We evaluate our algorithm over the FlickrMFC human dataset, and show that it achieves state-of-the-art performance for this challenging task.
Segmentation-Based View-Dependent 3-D Graphics Model Transmission

For wireless network based graphics applications, a key challenge is how to efficiently transmit complex 3-D models over bandwidth-limited wireless channels. Most existing 3-D mesh transmission systems do not consider such a view-dependent delivery issue, and thus transmit unnecessary portions of 3-D mesh models, which leads to the waste in precious wireless network bandwidth. In this paper, we propose a novel view-dependent 3-D model transmission scheme, where a 3-D model is partitioned into a number of segments, each segment is then independently coded using the MPEG-4 3DMC coding algorithm, and finally only the visible segments are selected and delivered to the client. Moreover, we also propose analytical models to find the optimal number of segments so as to minimize the average transmission size. Simulation results show that such a view-based 3-D model transmission is able to substantially save the transmission bandwidth and therefore has a significant impact on wireless graphics applications.
A diffusion approach to seeded image segmentation

Seeded image segmentation is a popular type of supervised image segmentation in computer vision and image processing. Previous methods of seeded image segmentation treat the image as a weighted graph and minimize an energy function on the graph to produce a segmentation. In this paper, we propose to conduct the seeded image segmentation according to the result of a heat diffusion process in which the seeded pixels are considered to be the heat sources and the heat diffuses on the image starting from the sources. After the diffusion reaches a stable state, the image is segmented based on the pixel temperatures. It is also shown that our proposed framework includes the RandomWalk algorithm for image segmentation as a special case which diffuses only along the two coordinate axes. To better control diffusion, we propose to incorporate the attributes (such as the geometric structure) of the image into the diffusion process, yielding an anisotropic diffusion method for image segmentation. The experiments show that the proposed anisotropic diffusion method usually produces better segmentation results. In particular, when the method is tested using the groundtruth dataset of Microsoft Research Cambridge (MSRC), an error rate of 4.42% can be achieved, which is lower than the reported error rates of other state-of-the-art algorithms.