nils-herrmann/clean_scirepeval huggingface dataset:
Recognizing Facial Expressions in Videos Using a Facial Action Analysis-Synthesis Scheme

In this paper, we propose a novel approach for facial expression analysis and recognition. The proposed approach relies on tracked facial actions provided by an appearance-based 3D face tracker. For each universal expression, a dynamical model for facial actions given by an auto-regressive process is learned from training data. We classify a given image in an unseen video into one of the universal facial expression categories using an analysis-synthesis scheme. This scheme uses all models and select the one that provides the most consistent synthesized spatio-temporal facial actions. The dynamical models can be utilized in the tasks of synthesis and prediction. Experiments using unseen videos demonstrated the effectiveness of the developed method.
Constructing panoramic views through facial gaze tracking

This paper describes a human machine interaction application for building panoramic views easily and efficiently. The panoramas are not limited to the 1D problem (one axis of rotation). The viewing direction of the camera acquiring snapshots is directly controlled by the userpsilas tracked gaze direction through a 3D face tracker. Natural face motions can be used to control local or remote camera in order to build panoramic views. The resulting system may find applications in online environment mapping as well as in video surveillance. The developed system was applied to map some indoor and outdoor scenes.
Single snapshot-based 3D head pose initialization for tracking in a HRI scenario

This paper presents an automatic 3D head pose initialization scheme for a real-time face tracker with application to human-robot interaction. It has two main contributions. First, we propose an automatic 3D head pose and person specific face shape estimation, based on a 3D deformable model. The proposed approach serves to initialize our real-time 3D face tracker. What makes this contribution very attractive is that the initialization step can cope with faces under arbitrary pose, so it is not limited only to near-frontal views. Second, the previous framework is used to develop an application in which the orientation of an AIBO's camera can be controlled through the imitation of user's head pose. In our scenario, this application is used to build panoramic images from overlapping snapshots. Experiments on real videos confirm the robustness and usefulness of the proposed methods.
Fast Automatic Step Size Estimation for Gradient Descent Optimization of Image Registration

Fast automatic image registration is an important prerequisite for image-guided clinical procedures. However, due to the large number of voxels in an image and the complexity of registration algorithms, this process is often very slow. Stochastic gradient descent is a powerful method to iteratively solve the registration problem, but relies for convergence on a proper selection of the optimization step size. This selection is difficult to perform manually, since it depends on the input data, similarity measure and transformation model. The Adaptive Stochastic Gradient Descent (ASGD) method is an automatic approach, but it comes at a high computational cost. In this paper, we propose a new computationally efficient method (fast ASGD) to automatically determine the step size for gradient descent methods, by considering the observed distribution of the voxel displacements between iterations. A relation between the step size and the expectation and variance of the observed distribution is derived. While ASGD has quadratic complexity with respect to the transformation parameters, fast ASGD only has linear complexity. Extensive validation has been performed on different datasets with different modalities, inter/intra subjects, different similarity measures and transformation models. For all experiments, we obtained similar accuracy as ASGD. Moreover, the estimation time of fast ASGD is reduced to a very small value, from 40 s to less than 1 s when the number of parameters is 105, almost 40 times faster. Depending on the registration settings, the total registration time is reduced by a factor of 2.5-7 × for the experiments in this paper.
A virtual exploring mobile robot for left ventricle contour tracking

In this paper we describe a totally new and original approach for combining global and local information in medical image processing. We implemented a virtual mobile robot and trained it using fuzzy neural networks to recognize segments of the myocardium while he navigates autonomously around the left ventricle (LV) of the heart. On its journey around the heart, the virtual exploring robot applies appropriate local edge detection to delineate fully automatically the borders of the myocardium. This may sound unconventional but it has proven effective enough to be integrated in a clinical analytical software tool.
Atlas-based organ & bone approximation for ex-vivo μMRI mouse data: A pilot study

In this paper we propose a novel semi-automated atlas-based approach for organ and bone approximation for micro-Magnetic Resonance Imaging (μMRI) data of mice. Based on a set of 18 manually indicated landmarks at specific joint & bone locations, individual atlas bones (pelvis, limb bones and sternum) are mapped to the target in a first step and a sparse set of corresponding landmarks on a skin surface representation is determined in a second step. Subsequently, this sparse set on the skin is used to derive a dense set of correspondences relying on matching spectra of local geodesic distances. Finally, determined by the skin correspondence, a Thin-Plate-Spline (TPS) approximation of major organs (heart, lungs, liver, spleen, stomach, kidneys) is performed. The method was tested using 3 µMRI mouse datasets and the MOBY atlas. The performance of the organ approximation algorithm was estimated using manual segmentations of 6 organs for each MRI dataset and calculating Dice indices of organ-volume overlap for each dataset and the atlas. The obtained results indicate excellent fitting of heart and kidneys and moderate fitting of spleen, lungs, liver and stomach. These initial results are satisfactory and comparable to other organ mapping studies using different approaches and μComputed Tomography (CT) mouse data.
DeepEyes: Progressive Visual Analytics for Designing Deep Neural Networks

Deep neural networks are now rivaling human accuracy in several pattern recognition problems. Compared to traditional classifiers, where features are handcrafted, neural networks learn increasingly complex features directly from the data. Instead of handcrafting the features, it is now the network architecture that is manually engineered. The network architecture parameters such as the number of layers or the number of filters per layer and their interconnections are essential for good performance. Even though basic design guidelines exist, designing a neural network is an iterative trial-and-error process that takes days or even weeks to perform due to the large datasets used for training. In this paper, we present DeepEyes, a Progressive Visual Analytics system that supports the design of neural networks during training. We present novel visualizations, supporting the identification of layers that learned a stable set of patterns and, therefore, are of interest for a detailed analysis. The system facilitates the identification of problems, such as superfluous filters or layers, and information that is not being captured by the network. We demonstrate the effectiveness of our system through multiple use cases, showing how a trained network can be compressed, reshaped and adapted to different problems.
Statistical Shape Model-Based Femur Kinematics From Biplane Fluoroscopy

Studying joint kinematics is of interest to improve prosthesis design and to characterize postoperative motion. State of the art techniques register bones segmented from prior computed tomography or magnetic resonance scans with X-ray fluoroscopic sequences. Elimination of the prior 3D acquisition could potentially lower costs and radiation dose. Therefore, we propose to substitute the segmented bone surface with a statistical shape model based estimate. A dedicated dynamic reconstruction and tracking algorithm was developed estimating the shape based on all frames, and pose per frame. The algorithm minimizes the difference between the projected bone contour and image edges. To increase robustness, we employ a dynamic prior, image features, and prior knowledge about bone edge appearances. This enables tracking and reconstruction from a single initial pose per sequence. We evaluated our method on the distal femur using eight biplane fluoroscopic drop-landing sequences. The proposed dynamic prior and features increased the convergence rate of the reconstruction from 71% to 91%, using a convergence limit of 3 mm. The achieved root mean square point-to-surface accuracy at the converged frames was 1.48 ± 0.41 mm.The resulting tracking precision was 1-1.5 mm, with the largest errors occurring in the rotation around the femoral shaft (about 2.5° precision).
Multistage hybrid active appearance model matching: segmentation of left and right ventricles in cardiac MR images

A fully automated approach to segmentation of the left and right cardiac ventricles from magnetic resonance (MR) images is reported. A novel multistage hybrid appearance model methodology is presented in which a hybrid active shape model/active appearance model (AAM) stage helps avoid local minima of the matching function. This yields an overall more favorable matching result. An automated initialization method is introduced making the approach fully automated. The authors' method was trained in a set of 102 MR images and tested in a separate set of 60 images. In all testing cases, the matching resulted in a visually plausible and accurate mapping of the model to the image data. Average signed border positioning errors did not exceed 0.3 mm in any of the three determined contours-left-ventricular (LV) epicardium, LV and right-ventricular (RV) endocardium. The area measurements derived from the three contours correlated well with the independent standard (r=0.96, 0.96, 0.90), with slopes and intercepts of the regression lines close to one and zero, respectively. Testing the reproducibility of the method demonstrated an unbiased performance with small range of error as assessed via Bland-Altman statistic. In direct border positioning error comparison, the multistage method significantly outperformed the conventional AAM (p<0.001). The developed method promises to facilitate fully automated quantitative analysis of LV and RV morphology and function in clinical setting.
Fast linear geodesic shape regression using coupled logdemons registration

Longitudinal brain image series offers the possibility to study individual brain anatomical changes over time. Mathematical models are needed to study such developmental trajectories in detail. In this paper, we present a novel approach to study the individual brain anatomy over time via a linear geodesic shape regression method. In our method, we integrate separate pairwise registrations between the baseline image and the follow-up images into a unified spatial registration plus temporal regression framework. Different from previous geodesic shape regression approaches, which use the LDDMM framework to estimate the brain anatomical change over time, our method is based on the LogDemons method to decrease the computation cost, while maintaining the diffeomorphic property of the deformation over time. Moreover, a temporal regression constraint is explicitly implemented in each optimization iteration to make sure that the entire developmental trajectory can be compactly represented by the baseline image and an optimal stationary velocity field. Our method is mathematically well founded in the Alternating Direction Method of Multipliers (ADMM), which for our image regression application is interpreted in diffeomorphic space instead of Euclidean space. We evaluate our new method on 2D synthetic images and real 3D brain longitudinal image series, and the experiments show promising results in regression accuracy as well as estimated deformations.
A multiresolution image segmentation technique based on pyramidal segmentation and fuzzy clustering

In this paper, an unsupervised image segmentation technique is presented, which combines pyramidal image segmentation with the fuzzy c-means clustering algorithm. Each layer of the pyramid is split into a number of regions by a root labeling technique, and then fuzzy c-means is used to merge the regions of the layer with the highest image resolution. A cluster validity functional is used to find the optimal number of objects automatically. Segmentation of a number of synthetic as well as clinical images is illustrated and two fully automatic segmentation approaches are evaluated, which determine the left ventricular volume (LV) in 140 cardiovascular magnetic resonance (MR) images. First fuzzy c-means is applied without pyramids. In the second approach the regions generated by pyramidal segmentation are merged by fuzzy c-means. The correlation coefficients of manually and automatically defined LV lumen of all 140 and 20 end-diastolic images were equal to 0.86 and 0.79, respectively, when images were segmented with fuzzy c-means alone. These coefficients increased to 0.90 and 0.93 when the pyramidal segmentation was combined with fuzzy c-means. This method can be applied to any dimensional representation and at any resolution level of an image series. The evaluation study shows good performance in detecting LV lumen in MR images.
Fully Automated Motion Correction in First-Pass Myocardial Perfusion MR Image Sequences

This paper presents a novel method for registration of cardiac perfusion magnetic resonance imaging (MRI). The presented method is capable of automatically registering perfusion data, using independent component analysis (ICA) to extract physiologically relevant features together with their time-intensity behavior. A time-varying reference image mimicking intensity changes in the data of interest is computed based on the results of that ICA. This reference image is used in a two-pass registration framework. Qualitative and quantitative validation of the method is carried out using 46 clinical quality, short-axis, perfusion MR datasets comprising 100 images each. Despite varying image quality and motion patterns in the evaluation set, validation of the method showed a reduction of the average right ventricle (LV) motion from 1.26plusmn0.87 to 0.64plusmn0.46 pixels. Time-intensity curves are also improved after registration with an average error reduced from 2.65plusmn7.89% to 0.87plusmn3.88% between registered data and manual gold standard. Comparison of clinically relevant parameters computed using registered data and the manual gold standard show a good agreement. Additional tests with a simulated free-breathing protocol showed robustness against considerable deviations from a standard breathing protocol. We conclude that this fully automatic ICA-based method shows an accuracy, a robustness and a computation speed adequate for use in a clinical environment.
Neuro-fuzzy systems for computer-aided myocardial viability assessment

This paper describes a multimodality framework for computer-aided myocardial viability assessment based on neurofuzzy techniques. The proposed approach distinguishes two main levels: the modality-independent inference level and the modality-dependent application level. This two-level distinction releases the hard constraint of multimodality image registration. An abstract description template is used to describe the different myocardial functions (contractile function, perfusion, metabolism). Parameters extracted from different image modalities are combined to derive a diagnostic image. The neuro-fuzzy techniques make our system transparent, adaptive and easily extendable. Its effectiveness and robustness are demonstrated in a positron emission tomography/magnetic resonance imaging data fusion application.
Anatomical model matching with fuzzy implicit surfaces for segmentation of thoracic volume scans

Many segmentation methods for thoracic volume data require manual input in the form of a seed point, initial contour, volume of interest etc. The aim of the work presented here is to further automate this segmentation initialization step. In this paper an anatomical modeling and matching method is proposed to coarsely segment thoracic volume data into anatomically labeled regions. An anatomical model of the thorax is constructed in two steps: (1) individual organs are modeled with blended fuzzy implicit surfaces and (2) the single organ models are grouped into a tree structure with a solid modeling technique named constructive solid geometry (CSG). The combination of CSG with fuzzy implicit surfaces allows a hierarchical scene description by means of a boundary model, which characterizes the scene volume as a boundary potential function. From this boundary potential, an energy function is defined which is minimal when the model is registered to the tissue-air transitions in thoracic magnetic resonance imaging (MRI) data. This allows automatic registration in three steps: feature detection, initial positioning and energy minimization. The model matching has been validated in phantom simulations and on 15 clinical thoracic volume scans from different subjects. In 13 of these sets the matching method accurately partitioned the image volumes into a set of volumes of interest for the heart, lungs, cardiac ventricles, and thorax outlines. The method is applicable to segmentation of various types of thoracic MR-images, provided that a large part of the thorax is contained in the image volume.
Approximated and User Steerable tSNE for Progressive Visual Analytics

Progressive Visual Analytics aims at improving the interactivity in existing analytics techniques by means of visualization as well as interaction with intermediate results. One key method for data analysis is dimensionality reduction, for example, to produce 2D embeddings that can be visualized and analyzed efficiently. t-Distributed Stochastic Neighbor Embedding (tSNE) is a well-suited technique for the visualization of high-dimensional data. tSNE can create meaningful intermediate results but suffers from a slow initialization that constrains its application in Progressive Visual Analytics. We introduce a controllable tSNE approximation (A-tSNE), which trades off speed and accuracy, to enable interactive data exploration. We offer real-time visualization techniques, including a density-based solution and a Magic Lens to inspect the degree of approximation. With this feedback, the user can decide on local refinements and steer the approximation level during the analysis. We demonstrate our technique with several datasets, in a real-world research scenario and for the real-time analysis of high-dimensional streams to illustrate its effectiveness for interactive data analysis.
Organ approximation in μCT data with low soft tissue contrast using an articulated whole-body atlas

In this article, we present an approach for organ approximation in low contrast muCT data of mice using a whole-body mouse atlas (Segars et al. ). Starting from a set of landmarks on bone and joint locations, further correspondences are derived on surface representations of the lung by atlas-based registration and on the skin by employing a local geodesic shape context. Subsequently, landmarks on the skeleton, the lung and the skin are used to constrain a Thin-Plate-Spline (TPS) based mapping of major organs from the atlas to the subject domain. The feasibility of the method has been tested by means of 26 CT mouse datasets and a different whole-body mouse atlas (Digimouse ). Proper mapping of the lung and the skin as well as major organs could be achieved in all cases yielding a mean Euclidean distance between surface nodes of 0.42 plusmn 0.068 mm for the lung and 0.34 plusmn 0.036 mm for the skin. The performance of the organ interpolation has been assessed on basis of manual segmentations of two CT datasets of mice with injected contrast agent and the Digimouse. The calculated dice indices of volume overlap show significant improvement compared to earlier studies.
Fully Automated Attenuation Measurement and Motion Correction in FLIP Image Sequences

Fluorescence loss in photobleaching (FLIP) is a method to study compartment connectivity in living cells. A FLIP sequence is obtained by alternatively bleaching a spot in a cell and acquiring an image of the complete cell. Connectivity is estimated by comparing fluorescence signal attenuation in different cell parts. The measurements of the fluorescence attenuation are hampered by the low signal to noise ratio of the FLIP sequences, by sudden sample shifts and by sample drift. This paper describes a method that estimates the attenuation by modeling photobleaching as exponentially decaying signals. Sudden motion artifacts are minimized by registering the frames of a FLIP sequence to target frames based on the estimated model and by removing frames that contain deformations. Linear motion (sample drift) is reduced by minimizing the entropy of the estimated attenuation coefficients. Experiments on 16 in vivo FLIP sequences of muscle cells in Drosophila show that the proposed method results in fluorescence attenuations similar to the manually identified gold standard, but with standard deviations of approximately 50 times smaller. As a result of this higher precision, cell compartment edges and details such as cell nuclei become clearly discernible. The main value of this method is that it uses a model of the bleaching process to correct motion and that the model based fluorescence intensity and attenuation estimates can be interpreted easily. The proposed method is fully automatic, and runs in approximately one minute per sequence, making it suitable for unsupervised batch processing of large data series.
Active appearance motion model segmentation

An adaptive method for temporal sequence segmentation was developed and its performance assessed in the segmentation of cardiac motion image sequences. The primary contribution of this paper is the development of a novel, 2D+time active appearance motion model (AAMM) that represents the dynamics of the cardiac cycle in combination with the shape and image appearance of the heart. Cootes' 2D active appearance model (AAM) framework was extended by considering a complete image sequence as a single shape/intensity sample. This way, the proven strength of AAMs, like robustness and ability to capture observer preference, are augmented with temporal consistency over an image sequence.