nils-herrmann/clean_scirepeval huggingface dataset:
On The Encoding Of The Anchor Frame In Video Coding

In this paper we determine the number of bits to be used for the encoding of the anchor frame in low bit rate video coding in order to improve the quality of the next and subsequent frames to be encoded. This problem is important in a real time video communication system. We use a progressive method for the transmission of the anchor frame. We develop two methods for determining the optimal number of bits to be allocated to the first frame in on line video communication applications.
Automated line flattening of Atomic Force Microscopy images

In this paper, an automated algorithm to flatten lines from Atomic Force Microscopy (AFM) images is presented. Due to the mechanics of the AFM, there is a curvature distortion (bowing effect) present in the acquired images. At present, flattening such images requires human intervention to manually segment object data from the background, which is time consuming and highly inaccurate. The proposed method classifies the data into objects and background, and fits convex lines in an iterative fashion. Results on real images from DNA wrapped carbon nanotubes (DNA-CNTs) and synthetic experiments are presented, demonstrating the effectiveness of the proposed algorithm in increasing the resolution of the surface topography.
Recursive displacement estimation for use in multiple object tracking

The authors develop a recursive model-based maximum a posteriori (MAP) estimator that estimates the displacement vector field (DVF) from a noisy image sequence. To model the DVF, they develop a nonstationary vector field model called the vector coupled Gauss-Markov (VCGM) model. The VCGM model consists of two levels, an upper level, which is made up of several submodels with various characteristics, and a lower level or line process which governs the transitions between the submodels. By estimating the line process the authors segment the DVF along its motion boundaries, which is a critical step in the tracking of objects through a video sequence.<<ETX>>
Automatic pigment identification on roman Egyptian paintings by using sparse modeling of hyperspectral images

In this paper, we study the problem of automatic identification of pigments applied to paintings using hyperspectral reflectance data. Here, we cast the problem of pigment identification in a novel way by decomposing the spectrum into pure pigments. The pure pigment exemplars, chosen and prepared in our laboratory based on historic sources and archaeological examples, closely resemble the materials used to make ancient paintings. To validate our algorithm, we created a set of mock-up paintings in our laboratory consisting of a broad palette of mixtures of pure pigments. Our results clearly demonstrate more accurate estimation of pigment composition than purely distance-based methods such as spectral angle mapping (SAM) and spectral correlation mapping (SCM). In addition, we studied hyperspectral imagery acquired of a Roman-Egyptian portrait, excavated from the site of Tebtunis in the Fayum region of Egypt, and dated to about the 2nd century CE. Using ground truth information obtained using Raman spectroscopy, we show qualitatively that our method accurately detects pigment composition for the specific pigments hematite and indigo.
Motion estimation in high resolution image reconstruction from compressed video sequences

In order to obtain a high resolution image from a compressed video sequence it is essential to correctly estimate the motion vectors in the sequence. Most of the approaches reported in the literature address this problem using standard motion estimation techniques. In this paper we tackle the correct estimation of the motion vectors by consistently estimating the optical flow across multiple images. Consistency is achieved by adding a regularization term to the classical Lucas-Kanade approach to motion estimation. The proposed algorithm is tested on real video sequences.
Adaptive multiple-input constrained pel-recursive displacement estimation

An adaptive multiple-input pel-recursive algorithm is presented. The displacement vector field (DVF) is estimated by minimizing the linearized displaced frame difference (DFD) using nu submasks of causal mask around the working point. Then, nu corresponding systems of equations are formed and the set theoretic regularization approach results in a weighted constrained least-squares estimation of the DVF by using information about the variance of the linearization error (noise) and the solution. The prior information about the solution is incorporated into the algorithm using a causal oriented smoothness constraint (OSC), which also provides a spatial prediction model for the estimated DVF. The improved performance of the proposed algorithm with respect to accuracy, robustness to occlusion, and smoothness of the estimated DVF is demonstrated.<<ETX>>
Speech-to-video synthesis using facial animation parameters

The presence of visual information in addition to audio could improve speech understanding in noisy environments. This additional information could be especially useful for people with impaired hearing who are able to speechread. This paper focuses on the problem of synthesizing the facial animation parameters (FAPs), supported by the MPEG-4 standard for the visual representation of speech, from a narrowband acoustic speech (telephone) signal. A correlation hidden Markov model (CHMM) system for performing visual speech synthesis is proposed. The CHMM system integrates an independently trained acoustic HMM (AHMM) system and a visual HMM (VHMM) system, in order to realize speech-to-video synthesis. Analyzing the synthesized FAPs and computing the time alignment errors perform objective experiments. Time alignment errors are reduced by 40.5% compared to the conventional temporal scaling method.
A non uniform segmentation optimal hybrid fractal/DCT image compression algorithm

In this paper a hybrid fractal and discrete cosine transform (DCT) coder is developed. Drawing on the ability of DCT to remove inter-pixel redundancies and on the ability of fractal transforms to capitalize on long-range correlations in the image, the hybrid coder performs an optimal, in the rate-distortion sense, bit allocation among coding parameters. An orthogonal basis framework is used within which an image segmentation and a hybrid block-based transform are selected jointly. A Lagrangian multiplier approach is used to optimize the hybrid parameters and the segmentation. Differential encoding of the DC coefficient is employed, with the scanning path based on a 3rd-order Hilbert curve. Simulation results show a significant improvement in quality with respect to the JPEG standard.
Digital image restoration

The article introduces digital image restoration to the reader who is just beginning in this field, and provides a review and analysis for the reader who may already be well-versed in image restoration. The perspective on the topic is one that comes primarily from work done in the field of signal processing. Thus, many of the techniques and works cited relate to classical signal processing approaches to estimation theory, filtering, and numerical analysis. In particular, the emphasis is placed primarily on digital image restoration algorithms that grow out of an area known as "regularized least squares" methods. It should be noted, however, that digital image restoration is a very broad field, as we discuss, and thus contains many other successful approaches that have been developed from different perspectives, such as optics, astronomy, and medical imaging, just to name a few. In the process of reviewing this topic, we address a number of very important issues in this field that are not typically discussed in the technical literature.
Joint Data Filtering and Labeling Using Gaussian Processes and Alternating Direction Method of Multipliers

Sequence labeling aims at assigning a label to every sample of a signal (or pixel of an image) while considering the sequentiality (or vicinity) of the samples. To perform this task, many works in the literature first filter and then label the data. Unfortunately, the filtering, which is performed independently from the labeling, is far from optimal and frequently makes the latter task harder. In this paper, a novel approach that trains a Gaussian process classifier and estimates the coefficients of an optimal filter jointly is presented. The new approach, based on Bayesian modeling and alternating direction method of multipliers (ADMMs) optimization, performs both tasks simultaneously. All unknowns are treated as stochastic variables, which are estimated using variational inference and filtering and labeling are linked with the use of ADMM. In the experimental section, synthetic and real experiments are presented to compare the proposed method with other existing approaches.
Multi-model robust error correction for face recognition

In this work we present a general framework for robust error estimation in face recognition. The proposed formulation allows the simultaneous use of various loss functions for modeling the residual in face images, which usually follows non-standard distributions, depending on the image capturing conditions. Our method extends the current vast literature offering flexibility in the selection of the residual modeling characteristics but, at the same time, considering many existing algorithms as special cases. As such, it proves robust for a range of error inducing factors, such as, varying illumination, occlusion, pixel corruption, disguise or their combinations. Extensive simulations document the superiority of selecting multiple models for representing the noise term in face recognition problems, allowing the algorithm to achieve near-optimal performance in most of the tested face databases. Finally, the multi-model residual representation offers useful insights into understanding how different noise types affect face recognition rates.
Unequal Error Protection for Robust Streaming of Scalable Video Over Packet Lossy Networks

Efficient bit stream adaptation and resilience to packet losses are two critical requirements in scalable video coding for transmission over packet-lossy networks. Various scalable layers have highly distinct importance, measured by their contribution to the overall video quality. This distinction is especially more significant in the scalable H.264/advanced video coding (AVC) video, due to the employed prediction hierarchy and the drift propagation when quality refinements are missing. Therefore, efficient bit stream adaptation and unequal protection of these layers are of special interest in the scalable H.264/AVC video. This paper proposes an algorithm to accurately estimate the overall distortion of decoder reconstructed frames due to enhancement layer truncation, drift/error propagation, and error concealment in the scalable H.264/AVC video. The method recursively computes the total decoder expected distortion at the picture-level for each layer in the prediction hierarchy. This ensures low computational cost since it bypasses highly complex pixel-level motion compensation operations. Simulation results show an accurate distortion estimation at various channel loss rates. The estimate is further integrated into a cross-layer optimization framework for optimized bit extraction and content-aware channel rate allocation. Experimental results demonstrate that precise distortion estimation enables our proposed transmission system to achieve a significantly higher average video peak signal-to-noise ratio compared to a conventional content independent system.
Optimal sensor selection for video-based target tracking in a wireless sensor network

The use of wireless sensor networks for target tracking is an active area of research. Imaging sensors that obtain video-rate images of a scene can have a significant impact in such networks, as they can measure vital information on the identity, position, and velocity of moving targets. Since wireless networks must operate under stringent energy constraints, it is important to identify the optimal set of imagers to be used in a tracking scenario such that the network lifetime is maximized. We formulate this problem as one of maximizing the information utility gained from a set of sensors subject to a constraint on the average energy consumption in the network. We use an unscented Kalman filter framework to solve the tracking and data fusion problem with multiple imaging sensors in a computationally efficient manner, and use a lookahead algorithm to optimize the sensor selection based on the predicted trajectory of the target. Simulation results show the effectiveness of this method of sensor selection.
A deep symmetry convnet for stroke lesion segmentation

Stroke is one of the leading causes of death and disability. Clinically, to establish stroke patient prognosis, an accurate delineation of brain lesion is essential, which is time consuming and prone to subjective errors. In this paper, we propose a novel method call Deep Lesion Symmetry ConvNet to automatically segment chronic stroke lesions using MRI. An 8-layer 3D convolutional neural network is constructed to handle the MRI voxels. An additional CNN stream using the corresponding symmetric MRI voxels is combined, leading to a significant improvement in system performance. The high average dice coefficient achieved on our dataset based on data collected from three research labs demonstrates the effectiveness of our method.
Shape Error Concealment Based on a Shape-Preserving Boundary Approximation

In object-based video representation, video scenes are composed of several arbitrarily shaped video objects (VOs), defined by their texture, shape and motion. In error-prone communications, packet loss results in missing information at the decoder. The impact of transmission errors is minimized through error concealment. In this paper, we propose a spatial error concealment technique for recovering lost shape data. We consider a geometric shape representation consisting of the object boundary, which can be extracted from the α-plane. Missing macroblocks result in a broken boundary. A B-spline curve is constructed to replace a missing boundary segment, based on a T-spline representation of the received boundary. We use T-splines because they produce shape-preserving approximations and do not change the characteristics of the original boundary. The representation ensures a good estimation of the first derivatives at the points touching the missing segment. Applying smoothing conditions, we manage to construct a new spline that joins smoothly with the received boundary, leading to successful concealment results. Experimental results on object shapes with different concealment difficulty demonstrate the performance of the proposed method. Comparisons with prior proposed methods are also presented.
Bayesian image restoration using a wavelet-based subband decomposition

The subband decomposition of a single channel image restoration problem is examined. The decomposition is carried out in the image model (prior model) in order to take into account the frequency activity of each band of the original image. The hyperparameters associated with each band together with the original image are rigorously estimated within the Bayesian framework. Finally, the proposed method is tested and compared with other methods on real images.
Simultaneous identification and restoration of images using maximum likelihood estimation

'I'his paper deals with simultaneous identification and restoration of images. By idqitification, we mean the estimation of the parameters characterizing the degradation mechanisms. The original image and the additive noise are assumed to be zeroniean Gaussian random processes. Their autocovariance ma trices arc unknown parameters. Blurring is part of the degradation. It is specified by its point spread function, which is also an unknown parameter to be estimated. Maximum likelihood estimation is used to find those unknown parameters. In turn, the EM algorithm is used to find the maximum likelihood estimates. In applying the EM algorithm, the observed image is treated as the incomplete data, which turns out to be a linear transformation of the complete data. Diffefent choices of complete data are investigated. Under the assumptioil that the image covariance and distortion matrices are circulant, the estimation of the unknown parameters becomes feasible. Explicit iterative expressions are derived for the estimation. The restored image is computed in the E-step of the EM algorithm.
Application of the motion vector constraint to the regularized enhancement of compressed video

We present a novel fidelity constraint for the image enhancement problem by exploiting the motion vectors of a compressed video bit-stream. These vectors establish a correspondence between image pixels across a series of frames, and our goal is to maintain this relationship during processing. In our past work, we considered algorithms that relied on the sum-of-absolute differences as the match criteria. As we show in this paper, this metric is problematic for the enhancement problem. We then pose the constraint within the context of a sum-of-squared errors criterion for matching. This allows for a more rigorous treatment of the fidelity constraint. Finally, experimental results illustrate the performance of the new constraint.
Single camera-based full depth map estimation using color shifting property of a multiple color-filter aperture

A multiple color-filter aperture (MCA) camera can provide depth information as well as color and intensity in the single-camera framework, where the MCA generates misalignment between color channels depending on the distance of a region-of-interest. In this paper, we present a single camera-based estimation of the full depth map using the color shifting property of the MCA. For estimating the color shifting vectors (CSVs) among red, green, and blue color channels, edges are extracted at each color channel. At the edge, we estimate CSVs using normalized cross correlation combined with color shifting mask map. A full depth map is then generated by depth interpolation using the matting Laplacian method from sparsely estimated CSVs at an edge location. Experimental results show that the proposed method can not only estimate the full depth map but also correct the misaligned color image to generate photorealistic color images using a single camera equipped with MCA.
Hierarchical Bayesian image restoration from partially-known blurs

A number of restoration filters have been proposed for the restoration problem from partially-known blurs. Previously we proposed the regularized constrained least-squares filter (RCTLS) and showed that it has a number of advantages over previous ones (Mesarovic et al. 1995). However, the problem of estimating the parameters that define the RCTLS filter has not yet been addressed. In this paper we propose a two-step algorithm based on the hierarchical Bayesian approach to simultaneously restore the image and estimate the parameters of the RCTLS restoration filter. The algorithm is derived in the DFT domain; thus, it is very efficient even for very large images.
Jointly optimal inter-mode shape coding and VLC selection

This paper investigates how the between frame correlation of shape information can be exploited within the framework of an operationally rate-distortion (ORD) optimized coder. Contours are approximated either by connected second-order spline segments, each defined by three consecutive control points, or by segments of the motion-compensated reference contours. Consecutive control points are then encoded predictively using angle and run temporal contexts or by tracking the reference contour. We employ a novel criterion for selecting global object motion vectors, which further improves efficiency. The problem is formulated as Lagrangian minimization and solved using Dynamic Programming (DP). Furthermore, we employ an iterative technique to remove dependency on a particular VLC and jointly arrive at the OPD globally optimal solution and an optimized conditional parameter distribution.
Toward Dynamic Scene Understanding by Hierarchical Motion Pattern Mining

Our work addresses the problem of analyzing and understanding dynamic video scenes. A two-level motion pattern mining approach is proposed. At the first level, activities are modeled as distributions over patch-based features, including spatial location, moving direction, and speed. At the second level, traffic states are modeled as distributions over activities. Both patterns are shared among video clips. Compared to other works, one advantage of our method is that moving speed is considered to describe visual word. The other advantage is that traffic states are detected and assigned to every video frame. These enable finer semantic interpretation, more precise video segmentation, and anomaly detection. Specifically, every video frame is labeled by a certain traffic state, and the video is segmented frame by frame accordingly. Moving pixels in each frame, which do not belong to any activity or cannot exist in the corresponding traffic state, are detected as anomalies. We have successfully tested our approach on some challenging traffic surveillance sequences containing both pedestrian and vehicle motions.
Video compressive sensing with on-chip programmable subsampling

The maximum achievable frame-rate for a video camera is limited by the sensor's pixel readout rate. The same sensor may achieve either a slow frame-rate at full resolution (e.g., 60 fps at 4 Mpixel resolution) or a fast frame-rate at low resolution (e.g., 240 fps at 1 Mpixel resolution). Higher frame-rates are achieved using pixel readout modes (e.g., subsampling or binning) that sacrifice spatial for temporal resolution within a fixed bandwidth. A number of compressive video cameras have been introduced to overcome this fixed bandwidth constraint and achieve high frame-rates without sacrificing spatial resolution. These methods use electro-optic components (e.g., LCoS, DLPs, piezo actuators) to introduce high speed spatio-temporal multiplexing in captured images. Full resolution, high speed video is then restored by solving an undetermined system of equations using a sparse regularization framework. In this work, we introduce the first all-digital temporal compressive video camera that uses custom subsampling modes to achieve spatio-temporal multiplexing. Unlike previous compressive video cameras, ours requires no additional optical components, enabling it to be implemented in a compact package such as a mobile camera module. We demonstrate results using a TrueSense development kit with a 12 Mpixel sensor and programmable FPGA read out circuitry.
Visualization of feature evolution during convolutional neural network training

Convolutional neural networks (CNNs) are a staple in the fields of computer vision and image processing. These networks perform visual tasks with state-of-the-art accuracy; yet, the understanding behind the success of these algorithms is still lacking. In particular, the process by which CNNs learn effective task-specific features is still unclear. This work elucidates such phenomena by applying recent deep visualization techniques during different stages of the training process. Additionally, this investigation provides visual justification to the benefits of transfer learning. The results are in line with previously discussed notions of feature specificity, and show a new facet of a particularly vexing machine learning pitfall: overfitting.
Parallel iterative image restoration algorithms

Three parallel iterative image restoration algorithms with and without preconditioning are proposed and analyzed. The first algorithm corresponds to a coarse-grained general-purpose multiprocessor computer, the second to a massively parallel computer with synchronization, and the third to a massively parallel computer without synchronization. It is shown that the second and third algorithms can give a speed-up proportional to the number of processors when proper assumptions are satisfied, while the first one performs the best when simulated by a uniprocessor computer.<<ETX>>