nils-herrmann/clean_scirepeval huggingface dataset:
Deep Learning with Hierarchical Convolutional Factor Analysis

Unsupervised multilayered (“deep”) models are considered for imagery. The model is represented using a hierarchical convolutional factor-analysis construction, with sparse factor loadings and scores. The computation of layer-dependent model parameters is implemented within a Bayesian setting, employing a Gibbs sampler and variational Bayesian (VB) analysis that explicitly exploit the convolutional nature of the expansion. To address large-scale and streaming data, an online version of VB is also developed. The number of dictionary elements at each layer is inferred from the data, based on a beta-Bernoulli implementation of the Indian buffet process. Example results are presented for several image-processing applications, with comparisons to related models in the literature.
Color and Illuminant Voting

A geometric-vision approach to color constancy and illuminant estimation is presented in this paper. We show a general framework, based on ideas from the generalized probabilistic Hough transform, to estimate the illuminant and reflectance of natural images. Each image pixel "votes" for possible illuminants and the estimation is based on cumulative votes. The framework is natural for the introduction of physical constraints in the color constancy problem. We show the relationship of this work to previous algorithms for color constancy and present examples.
A Variational Framework for Simultaneous Motion Estimation and Restoration of Motion-Blurred Video

The problem of motion estimation and restoration of objects in a blurred video sequence is addressed in this paper. Fast movement of the objects, together with the aperture time of the camera, result in a motion-blurred image. The direct velocity estimation from this blurred video is inaccurate. On the other hand, an accurate estimation of the velocity of the moving objects is critical for restoration of motion-blurred video. Therefore, restoration needs accurate motion estimation and vice versa, and a joint process is called for. To address this problem we derive a novel model of the blurring process and propose a Mumford-Shah type of variational framework, acting on consecutive frames, for joint object deblurring and velocity estimation. The proposed procedure distinguishes between the moving object and the background and is accurate also close to the boundary of the moving object. Experimental results both on simulated and real data show the importance of this joint estimation and its superior performance when compared to the independent estimation of motion and restoration.
Regularized Mixed Dimensionality and Density Learning in Computer Vision

A framework for the regularized estimation of nonuniform dimensionality and density in high dimensional data is introduced in this work. This leads to learning stratifications, that is, mixture of manifolds representing different characteristics and complexities in the data set. The basic idea relies on modeling the high dimensional sample points as a process of Poisson mixtures, with regularizing restrictions and spatial continuity constraints. Theoretical asymptotic results for the model are presented as well. The presentation of the framework is complemented with artificial and real examples showing the importance of regularized stratification learning in computer vision applications.
Alignment with intra-class structure can improve classification

High dimensional data is modeled using low-rank subspaces, and the probability of misclassification is expressed in terms of the principal angles between subspaces. The form taken by this expression motivates the design of a new feature extraction method that enlarges inter-class separation, while preserving intra-class structure. The method can be tuned to emphasize different features shared by members within the same class. Classification performance is compared to that of state-of-the-art methods on synthetic data and on the real face database. The probability of misclassification is decreased when intra-class structure is taken into account.
Generalization error of deep neural networks: Role of classification margin and data structure

Understanding the generalization properties of deep learning models is critical for their successful usage in many applications, especially in the regimes where the number of training samples is limited. We study the generalization properties of deep neural networks (DNNs) via the Jacobian matrix of the network. Our analysis is general to arbitrary network structures, types of non-linearities and pooling operations. We show that bounding the spectral norm of the Jacobian matrix in the network reduces the generalization error. In addition, we tie this error to the invariance in the data and the network. Experiments on the MNIST and ImageNet datasets support these findings. This short paper summarizes our generalization error theorems for DNNs and for general invariant classifiers , .
A nonintrusive system for behavioral analysis of children using multiple RGB+depth sensors

In developmental disorders such as autism and schizophrenia, observing behavioral precursors in very early childhood can allow for early intervention and can improve patient outcomes. While such precursors open the possibility of broad and large-scale screening, until now they have been identified only through experts' painstaking examinations and their manual annotations of limited, unprocessed video footage. Here we introduce a system to automate and assist in such procedures. Employing multiple inexpensive real-time rgb+depth (rgb+d) sensors recording from multiple viewpoints, our non-invasive system - now installed at the Shirley G. Moore Lab School, a research preschool - is being developed to monitor and reconstruct the play and interactions of preschoolers. The system's role is to help in assessing the growing volumes of its on-site recordings and to provide the data needed to uncover additional neuromotor behavioral markers via techniques such as data mining.
Gaussian mixture model for video compressive sensing

A Gaussian Mixture Model (GMM)-based algorithm is proposed for video reconstruction from temporal compressed measurements. The GMM is used to model spatio-temporal video patches, and the reconstruction can be efficiently computed based on analytic expressions. The developed GMM reconstruction method benefits from online adaptive learning and parallel computation. We demonstrate the efficacy of the proposed GMM with videos reconstructed from simulated compressive video measurements and from a real compressive video camera.
Three-dimensional point cloud recognition via distributions of geometric distances

A geometric framework for the recognition of three-dimensional objects represented by point clouds is introduced in this paper. The proposed approach is based on comparing distributions of intrinsic measurements on the point cloud. In particular, intrinsic distances are exploited as signatures for representing the point clouds. The first signature we introduce is the histogram of pairwise diffusion distances between all points on the shape surface. These distances represent the probability of traveling from one point to another in a fixed number of random steps, the average intrinsic distances of all possible paths of a given number of steps between the two points. This signature is augmented by the histogram of the actual pairwise geodesic distances, as well as the distribution of the ratio between these two distances. These signatures are not only geometric but also invariant to bends. We further augment these signatures by the distribution of a curvature function and the distribution of a curvature weighted distance. These histograms are compared using the chi2 or other common distance metrics for distributions. The presentation of the framework is accompanied by theoretical justification and state-of-the-art experimental results with the standard Princeton 3D shape benchmark and ISDB datasets, as well as a detailed analysis of the particular relevance of each one of the different histogram-based signatures. Finally, we briefly discuss a more local approach where the histograms are computed for a number of overlapping patches from the object rather than the whole shape, thereby opening the door to partial shape comparisons.
Cross-modality pose-invariant facial expression

In this work, we present a dictionary learning based framework for robust, cross-modality, and pose-invariant facial expression recognition. The proposed framework first learns a dictionary that i) contains both 3D shape and morphological information as well as 2D texture and geometric information, ii) enforces coherence across both 2D and 3D modalities and different poses, and iii) is robust in the sense that a learned dictionary can be applied across multiple facial expression datasets. We demonstrate that enforcing domain specific block structures on the dictionary, given a test expression sample, we can transform such sample across different domains for tasks such as pose alignment. We validate our approach on the task of pose-invariant facial expression recognition on the standard BU3D-FE and MultiPie datasets, achieving state of the art performance.
Intersecting 2D lines: A simple method for detecting vanishing points

We present a simple and powerful technique for testing with a prescribed precision whether a set of 2D lines meet at a given point. The method is based on a probabilistic framework and has a fundamental geometric interpretation. We use this technique for detecting vanishing points in images. We developed a very simple algorithm that yields state-of-the-art results at a much lower computational cost than its competitors. The presentation of the proposed formulation is complemented with numerous examples.
A geometric method for automatic extraction of sulcal fundi

Sulcal fundi are 3D curves that lie in the depths of the cerebral cortex and are often used as landmarks for downstream computations in brain imaging. We present a sequence of geometric algorithms which automatically extract the sulcal fundi from magnetic resonance images and represent them as smooth polygons lying on the cortical surface. First we compute a geometric depth measure for each point on the cortical surface, and based on this information we extract sulcal regions by checking the connectivity above a depth threshold. We then extract the endpoints of each fundus and delineate the fundus by thinning each connected region keeping the endpoints fixed. The curves thus defined are smoothed using weighted splines on the gray-matter surface to yield high-quality representations of the sulcal fundi
Projection based image and video inpainting using wavelets

In this paper, we present a technique for automatic color image inpainting, the art of modifying an image-region in a nondetectable form. The main algorithm is based on the theory of projections onto convex sets (POCS). The image and its wavelet transform are projected onto each other after applying suitable constraints in each domain. This technique exploits the frequency-spatial representation provided by wavelets and utilizes the correlation between the damaged area in the image and its neighborhood. The resulting restored area is homogeneous with its surrounding and preserves the aesthetics of the image. The same technique is used for simple video restoration problems. Video frames are stacked and treated as a 3D volume, making a natural use of interframe correlation.
Discriminative sparse representations in hyperspectral imagery

Recent advances in sparse modeling and dictionary learning for discriminative applications show high potential for numerous classification tasks. In this paper, we show that highly accurate material classification from hyperspectral imagery (HSI) can be obtained with these models, even when the data is reconstructed from a very small percentage of the original image samples. The proposed supervised HSI classification is performed using a measure that accounts for both reconstruction errors and sparsity levels for sparse representations based on class-dependent learned dictionaries. Combining the dictionaries learned for the different materials, a linear mixing model is derived for sub-pixel classification. Results with real hyperspectral data cubes are shown both for urban and non-urban terrain.
Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images

Nonparametric Bayesian methods are considered for recovery of imagery based upon compressive, incomplete, and/or noisy measurements. A truncated beta-Bernoulli process is employed to infer an appropriate dictionary for the data under test and also for image recovery. In the context of compressive sensing, significant improvements in image recovery are manifested using learned dictionaries, relative to using standard orthonormal image expansions. The compressive-measurement projections are also optimized for the learned dictionary. Additionally, we consider simpler (incomplete) measurements, defined by measuring a subset of image pixels, uniformly selected at random. Spatial interrelationships within imagery are exploited through use of the Dirichlet and probit stick-breaking processes. Several example results are presented, with comparisons to other methods in the literature.
Image modeling and enhancement via structured sparse model selection

An image representation framework based on structured sparse model selection is introduced in this work. The corresponding modeling dictionary is comprised of a family of learned orthogonal bases. For an image patch, a model is first selected from this dictionary through linear approximation in a best basis, and the signal estimation is then calculated with the selected model. The model selection leads to a guaranteed near optimal denoising estimator. The degree of freedom in the model selection is equal to the number of the bases, typically about 10 for natural images, and is significantly lower than with traditional overcomplete dictionary approaches, stabilizing the representation. For an image patch of size √N × √N, the computational complexity of the proposed framework is O (N2), typically 2 to 3 orders of magnitude faster than estimation in an overcomplete dictionary. The orthogonal bases are adapted to the image of interest and are computed with a simple and fast procedure. State-of-the-art results are shown in image denoising, deblurring, and inpainting.
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?

Three important properties of a classification machinery are i) the system preserves the core information of the input data; ii) the training examples convey information about unseen data; and iii) the system is able to treat differently points from different classes. In this paper, we show that these fundamental properties are satisfied by the architecture of deep neural networks. We formally prove that these networks with random Gaussian weights perform a distance-preserving embedding of the data, with a special treatment for in-class and out-of-class data. Similar points at the input of the network are likely to have a similar output. The theoretical analysis of deep networks here presented exploits tools used in the compressed sensing and dictionary learning literature, thereby making a formal connection between these important topics. The derived results allow drawing conclusions on the metric learning properties of the network and their relation to its structure, as well as providing bounds on the required size of the training set such that the training examples would represent faithfully the unseen data. The results are validated with state-of-the-art trained networks.
REGULARIZATION FOR INVERTING THE RADON TRANSFORM WITH WEDGE CONSIDERATION

In limited angle tomography, with applications such as electron microscopy, medical imaging, and industrial testing, the object of interest is scanned over a limited angular range, which is less than the full 180deg mathematically required for density reconstruction. The use of standard full-range reconstruction algorithms produces results with notorious "butterfly" or "wedge" artifacts. In this work we propose a reconstruction technique with a regularization term that takes into account the orientation of the missing angular range, also denoted as missing wedge. We show that a regularization that penalizes non-uniformly in the orientation space produces reconstructions with less artifacts, thereby improving the recovery of the "invisible" edges due to the missing wedge. We present the underlying framework and results for a challenging phantom and real cryo-electron microscopy data
Sparse Representation for Computer Vision and Pattern Recognition

Techniques from sparse signal representation are beginning to see significant impact in computer vision, often on nontraditional applications where the goal is not just to obtain a compact high-fidelity representation of the observed signal, but also to extract semantic information. The choice of dictionary plays a key role in bridging this gap: unconventional dictionaries consisting of, or learned from, the training samples themselves provide the key to obtaining state-of-the-art results and to attaching semantic meaning to sparse signal representations. Understanding the good performance of such unconventional dictionaries in turn demands new algorithmic and analytical techniques. This review paper highlights a few representative examples of how the interaction between sparse signal representation and computer vision can enrich both fields, and raises a number of open questions for further study.
C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework

Sparse modeling is a powerful framework for data analysis and processing. Traditionally, encoding in this framework is performed by solving an l1-regularized linear regression problem, commonly referred to as Lasso or Basis Pursuit. In this work we combine the sparsity-inducing property of the Lasso at the individual feature level, with the block-sparsity property of the Group Lasso, where sparse groups of features are jointly encoded, obtaining a sparsity pattern hierarchically structured. This results in the Hierarchical Lasso (HiLasso), which shows important practical advantages. We then extend this approach to the collaborative case, where a set of simultaneously coded signals share the same sparsity pattern at the higher (group) level, but not necessarily at the lower (inside the group) level, obtaining the collaborative HiLasso model (C-HiLasso). Such signals then share the same active groups, or classes, but not necessarily the same active set. This model is very well suited for applications such as source identification and separation. An efficient optimization procedure, which guarantees convergence to the global optimum, is developed for these new models. The underlying presentation of the framework and optimization approach is complemented by experimental examples and theoretical results regarding recovery guarantees.
Inpainting the colors

A framework for automatic image colorization, the art of adding color to a monochrome image or movie, is presented in this paper. The approach is based on considering the geometry and structure of the monochrome luminance input, given by its gradient information, as representing the geometry and structure of the whole colored version. The color is then obtained by solving a partial differential equation that propagates a few color scribbles provided by the user or by side information, while considering the gradient information brought in by the monochrome data. This way, the color is inpainted, constrained both by the monochrome image geometry and the provided color samples. We present the underlying framework and examples for still images and movies.
Direction diffusion

In a number of disciplines, directional data provides a fundamental source of information. A novel framework for isotropic and anisotropic diffusion of directions is presented in this paper. The framework can be applied both to regularize directional data and to obtain multiscale representations of it. The basic idea is to apply and extend results from the theory of harmonic maps in liquid crystals. This theory deals with the regularization of vectorial data, while satisfying the unit norm constraint of directional data. We show the corresponding variational and partial differential equations formulations for isotropic diffusion, obtained from an L/sub 2/ norm, and edge preserving diffusion, obtained from an L/sub 1/ norm. In contrast with previous approaches, the framework is valid for directions in any dimensions, supports non-smooth data, and gives both isotropic and anisotropic formulations. We present a number of theoretical results, open questions, and examples for gradient vectors, optical flow, and color images.
A multi-sensor visual tracking system for behavior monitoring of at-risk children

Clinical studies confirm that mental illnesses such as autism, Obsessive Compulsive Disorder (OCD), etc. show behavioral abnormalities even at very young ages; the early diagnosis of which can help steer effective treatments. Most often, the behavior of such at-risk children deviate in very subtle ways from that of a normal child; correct diagnosis of which requires prolonged and continuous monitoring of their activities by a clinician, which is a difficult and time intensive task. As a result, the development of automation tools for assisting in such monitoring activities will be an important step towards effective utilization of the diagnostic resources. In this paper, we approach the problem from a computer vision standpoint, and propose a novel system for the automatic monitoring of the behavior of children in their natural environment through the deployment of multiple non-invasive sensors (cameras and depth sensors). We provide details of our system, together with algorithms for the robust tracking of the activities of the children. Our experiments, conducted in the Shirley G. Moore Laboratory School, demonstrate the effectiveness of our methodology.
Evaluation of JPEG-LS, the new lossless and controlled-lossy still image compression standard, for compression of high-resolution elevation data

The compression of elevation data is studied. The performance of JPEG-LS, the new international ISO/ITU standard for lossless and near-lossless (controlled-lossy) still-image compression, is investigated both for data from the USGS digital elevation model (DEM) database and the navy-provided digital terrain model (DTM) data. Using JPEG-LS has the advantage of working with a standard algorithm. Moreover, in contrast with algorithms like the popular JPEG-lossy standard, this algorithm permits the completely lossless compression of the data as well as a controlled lossy mode where a sharp upper bound on the elevation error is selected by the user. All these are achieved at a very low computational complexity. In addition to these algorithmic advantages, they show that JPEG-LS achieves significantly better compression results than those obtained with other (nonstandard) algorithms previously investigated for the compression of elevation data. The results here reported suggest that JPEG-LS can immediately be adopted for the compression of elevation data for a number of applications.
Anisotropic 2-D and 3-D averaging of fMRI signals

A novel method for denoising functional magnetic resonance imaging temporal signals is presented in this note. The method is based on progressively enhancing the temporal signal by means of adaptive anisotropic spatial averaging. This average is based on a new metric for comparing temporal signals corresponding to active fMRI regions. Examples are presented both for simulated and real two and three-dimensional data. The software implementing the proposed technique is publicly available for the research community.