nils-herrmann/clean_scirepeval huggingface dataset:
A Service-oriented Framework for Context-aware Semantic Media Applications

As we move closer to the vision of ubiquitous, invisible computing, we feel the need for a framework that supports the integration of visual and auditory inputs with context sourced from various ambient 'sense organs' such as temperature and pressure sensors etc, as well as other factual information about persons, locations, activity schedules and events that are of significance to human beings. In this paper we describe our framework for integrating visual inputs at the human-semantics level, so that these inputs can be used for reasoning and inference. We borrow techniques from media and knowledge representation as well as work in pervasive context-aware computing to build a unifying service-oriented framework that can support applications with differing needs.
Genome-Wide Association study for glaucoma

The Genome-Wide Association (GWA) study is the latest approach in the development of genetic studies and is renowned for its widespread success in identifying disease variants within the genome for various common diseases. It is a highly popular study amongst geneticists worldwide, evident from the numerous GWA studies conducted in laboratories all over the world. This paper introduces various GWA study designs currently recognized, including other aspects such as the software tools and its progress thus far. Especially, the paper reviews the genetic studies for glaucoma, an ocular disease which can lead to irreversible and permanent vision loss. Glaucomatous progression can be slowed or even halted if detected early; however, genetic information on glaucoma has not been well established yet. Therefore, by conducting a GWA study on glaucoma to find comprehensive associated genetic variants, the early detection of glaucoma through GWA may finally be seen as a possibility.
Learning cell geometry models for cell image simulation: An unbiased approach

Computer generation of cell images can provide annotated data to simulate various imaging conditions with controllable parameters. Synthesized images based on simple models cannot reflect the complicated parameter constraints in simulating real objects in terms of their deformation with appropriate probabilities. Learning-based techniques can provide insight to these properties and impose constraints on deformation selections. In this work, we discuss the simulation of gray level images of healthy red blood cell populations. Different from existing techniques, we learn the unbiased average shape and deformation models of the cells. Both models are used to guide the selection of possible deformations. We also learn cell color models to govern the texture generation of simulated cells. We apply this technique to simulate cell populations and validate the results using cell segmentation and counting algorithms. The proposed learning and simulation technique is generic and can be applied to other types of cells as well.
A Computer-Aided Diagnosis System of Nuclear Cataract

Cataracts are the leading cause of blindness worldwide, and nuclear cataract is the most common form of cataract. An algorithm for automatic diagnosis of nuclear cataract is investigated in this paper. Nuclear cataract is graded according to the severity of opacity using slit lamp lens images. Anatomical structure in the lens image is detected using a modified active shape model. On the basis of the anatomical landmark, local features are extracted according to clinical grading protocol. Support vector machine regression is employed for grade prediction. This is the first time that the nucleus region can be detected automatically in slit lamp images. The system is validated using clinical images and clinical ground truth on >5000 images. The success rate of structure detection is 95% and the average grading difference is 0.36 on a 5.0 scale. The automatic diagnosis system can improve the grading objectivity and potentially be used in clinics and population studies to save the workload of ophthalmologists.
An empirical approach for automatic face clustering on personal lifelogging images

Life-logging applications generate a vast amount of personalized data that provides vital insights into the user's daily life. One such key insight is the people whom the user has come across/interacted with during regular life. This can be obtained from the faces extracted from images acquired by a wearable life-logging camera. However, manual inspection and tagging of the life-logging images is cumbersome and highly subjective. Therefore, in this paper, a fully automatic method to extract and cluster the faces from the images obtained from a life-logging camera is designed and evaluated. It is shown that such a practical system designed using commercial off-the-shelf devices and commercially available face recognition APIs is able to obtain human like precision, while the recall may be lower compared to human performance.
Automatic fundus image classification for computer-aided diagonsis

With the advances of computer technology, more and more computer-aided diagnosis (CAD) systems have been developed to provide the “second opinion”. This paper reports an automatic fundus image classification technique that is designed to screen out the severely degraded fundus images that cannot be processed by traditional CAD systems. The proposed technique classifies fundus images based on the image range property. In particular, it first calculates a number of range images from a fundus image at different resolutions. A feature vector is then constructed based on the histogram of the calculated range images. Finally, fundus images can be classified by a linear discriminant classifier that is built by learning from a large number of normal and abnormal training fundus images. Experiments over 644 fundus images of different qualities show that the classification accuracy of the proposed technique reaches above 96%.
Automated nuclei clump decomposition for image analysis in neuronal cell fluorescent microscopy

Automated clump decomposition is crucial for the analysis of neuronal cell fluorescent microscopic images due to inevitable nuclei and cell clumping. Existing techniques are not satisfactory in terms of accuracy and efficiency. In the current work, we propose a new method to automatically decompose nuclei clumps, using multiple visual cues from seeds, skeletons, and object boundaries and nuclei shape models to deliver robust and efficient analysis. The use of shape models enhances its robustness in handling complicated clumps against those bottom-up approaches without prior knowledge. We adopt a strategy of model verification in allowable local shape changes to improve the computational efficiency over template matching. Validation experiments in the analysis of 50 images containing over 2000 nuclei have demonstrated accuracy improvements over existing techniques.
Multimodal Multi-Stream Deep Learning for Egocentric Activity Recognition

In this paper, we propose a multimodal multi-stream deep learning framework to tackle the egocentric activity recognition problem, using both the video and sensor data. First, we experiment and extend a multi-stream Convolutional Neural Network to learn the spatial and temporal features from egocentric videos. Second, we propose a multistream Long Short-Term Memory architecture to learn the features from multiple sensor streams (accelerometer, gyroscope, etc.). Third, we propose to use a two-level fusion technique and experiment different pooling techniques to compute the prediction results. Experimental results using a multimodal egocentric dataset show that our proposed method can achieve very encouraging performance, despite the constraint that the scale of the existing egocentric datasets is still quite limited.
Image retrieval using spatial icons

We propose an adaptive semantic image indexing framework based on the notion of semantic support regions. Semantic support regions are salient image regions that exhibit semantic meanings and that can be derived statistically to span a new indexing space. We perform the learning and indexing of semantic support regions for 2400 heterogeneous consumer photos using support vector machines. We demonstrate the uniqueness and effectiveness of query by spatial icons, supported by our semantic indexing framework, with 12 queries.
Scene Identification using Discriminative Patterns

With the proliferation of camera phones, new information retrieval applications will emerge. The image of a scene captured by a camera phone can be a query to a remote server to identify the scene and return relevant information. But unconstrained scene identification is an open problem. In this paper, we propose a discriminative measure to rank image patterns sampled from target scene classes. Support vector classifiers are then trained using top discriminative patterns for scene identification using voting. We demonstrate our generic approach on two scene databases (ZuBuD and STOIC) with promising results
Convex hull based neuro-retinal optic cup ellipse optimization in glaucoma diagnosis

Glaucoma is the second leading cause of blindness. Glaucoma can be diagnosed through measurement of neuro-retinal optic cup-to-disc ratio (CDR). Automatic calculation of optic cup boundary is challenging due to the interweavement of blood vessels with the surrounding tissues around the cup. A Convex Hull based Neuro-Retinal Optic Cup Ellipse Optimization algorithm improves the accuracy of the boundary estimation. The algorithm’s effectiveness is demonstrated on 70 clinical patient’s data set collected from Singapore Eye Research Institute. The root mean squared error of the new algorithm is 43% better than the ARGALI system which is the state-of-the-art. This further leads to a large clinical evaluation of the algorithm involving 15 thousand patients from Australia and Singapore.
Learning cognitive manifolds of faces

Inspired by the studies in psychology and neuroscience, we propose a computational model of cognitive face representation that mimics the mechanism of human face perception. We propose to learn two separate manifolds for facial identity and facial expression. A new t-SNE algorithm is designed to find a way to achieve the learning purpose and a spectral regression algorithm is implemented accordingly to learn the cognitive manifolds of neutral and smiling faces from real face images. The association between the two manifolds is established by a weighted k-NN fusion algorithm. An iterative linear algorithm is derived to refine the association between the two manifolds. We evaluate our model on two large datasets containing real-world images of neutral and wide smiling faces. The experimental results show both the accuracy of the cognitive manifold representation and the superiority over Eigenfaces and Fisherfaces on the tough task of matching a wide smiling face to a neutral face of an individual.
Deep Future Gaze: Gaze Anticipation on Egocentric Videos Using Adversarial Networks

We introduce a new problem of gaze anticipation on egocentric videos. This substantially extends the conventional gaze prediction problem to future frames by no longer confining it on the current frame. To solve this problem, we propose a new generative adversarial neural network based model, Deep Future Gaze (DFG). DFG generates multiple future frames conditioned on the single current frame and anticipates corresponding future gazes in next few seconds. It consists of two networks: generator and discriminator. The generator uses a two-stream spatial temporal convolution architecture (3D-CNN) explicitly untangling the foreground and the background to generate future frames. It then attaches another 3D-CNN for gaze anticipation based on these synthetic frames. The discriminator plays against the generator by differentiating the synthetic frames of the generator from the real frames. Through competition with discriminator, the generator progressively improves quality of the future frames and thus anticipates future gaze better. Experimental results on the publicly available egocentric datasets show that DFG significantly outperforms all well-established baselines. Moreover, we demonstrate that DFG achieves better performance of gaze prediction on current frames than state-of-the-art methods. This is due to benefiting from learning motion discriminative representations in frame generation. We further contribute a new egocentric dataset (OST) in the object search task. DFG also achieves the best performance for this challenging dataset.
Faceted topic retrieval of news video using joint topic modeling of visual features and speech transcripts

Because of the inherent ambiguity in user queries, an important task of modern retrieval systems is faceted topic retrieval (FTR), which relates to the goal of returning diverse or novel information elucidating the wide range of topics or facets of the query need. We introduce a generative model for hypothesizing facets in the (news) video domain by combining the complementary information in the visual keyframes and the speech transcripts. We evaluate the efficacy of our multimodal model on the standard TRECVID-2005 video corpus annotated with facets. We find that: (1) the joint modeling of the visual and text (speech transcripts) information can achieve significant F-score improvement over a text-alone system; (2) our model compares favorably with standard diverse ranking algorithms such as the MMR . Our FTR model has been implemented on a news search prototype that is undergoing commercial trial.
Robust and Efficient Saliency Modeling from Image Co-Occurrence Histograms

This paper presents a visual saliency modeling technique that is efficient and tolerant to the image scale variation. Different from existing approaches that rely on a large number of filters or complicated learning processes, the proposed technique computes saliency from image histograms. Several two-dimensional image co-occurrence histograms are used, which encode not only "how many" (occurrence) but also "where and how" (co-occurrence) image pixels are composed into a visual image, hence capturing the "unusualness" of an object or image region that is often perceived by either global "uncommonness" (i.e., low occurrence frequency) or local "discontinuity" with respect to the surrounding (i.e., low co-occurrence frequency). The proposed technique has a number of advantageous characteristics. It is fast and very easy to implement. At the same time, it involves minimal parameter tuning, requires no training, and is robust to image scale variation. Experiments on the AIM dataset show that a superior shuffled AUC (sAUC) of 0.7221 is obtained, which is higher than the state-of-the-art sAUC of 0.7187.
Learning Deep Hierarchical Visual Feature Coding

In this paper, we propose a hybrid architecture that combines the image modeling strengths of the bag of words framework with the representational power and adaptability of learning deep architectures. Local gradient-based descriptors, such as SIFT, are encoded via a hierarchical coding scheme composed of spatial aggregating restricted Boltzmann machines (RBM). For each coding layer, we regularize the RBM by encouraging representations to fit both sparse and selective distributions. Supervised fine-tuning is used to enhance the quality of the visual representation for the categorization task. We performed a thorough experimental evaluation using three image categorization data sets. The hierarchical coding scheme achieved competitive categorization accuracies of 79.7% and 86.4% on the Caltech-101 and 15-Scenes data sets, respectively. The visual representations learned are compact and the model's inference is fast, as compared with sparse coding methods. The low-level representations of descriptors that were learned using this method result in generic features that we empirically found to be transferrable between different image data sets. Further analysis reveal the significance of supervised fine-tuning when the architecture has two layers of representations as opposed to a single layer.
Unsupervised learning for image classification based on distribution of hierarchical feature tree

The classification image into one of several categories is a problem arisen naturally under a wide range of circumstances. In this paper, we present a novel unsupervised model for the image classification based on featurepsilas distribution of particular patches of images. Our method firstly divides an image into grids and then constructs a hierarchical tree in order to mine the feature information of the image details. According to our definition, the root of the tree contains the global information of the image, and the child nodes contain detail information of image. We observe the distribution of features on the tree to find out which patches are important in term of a particular class. The experiment results show that our performances are competitive with the state of art in image classification in term of recognition rate.
Determining the best attributes for surveillance video keywords generation

Automatic video keyword generation is one of the key ingredients in reducing the burden of security officers in analyzing surveillance videos. Keywords or attributes are generally chosen manually based on expert knowledge of surveillance. Most existing works primarily aim at either supervised learning approaches relying on extensive manual labelling or hierarchical probabilistic models that assume the features are extracted using the bag-of-words approach; thus limiting the utilization of the other features. To address this, we turn our attention to automatic attribute discovery approaches. However, it is not clear which automatic discovery approach can discover the most meaningful attributes. Furthermore, little research has been done on how to compare and choose the best automatic attribute discovery methods. In this paper, we propose a novel approach, based on the shared structure exhibited amongst meaningful attributes, that enables us to compare between different automatic attribute discovery approaches. We then validate our approach by comparing various attribute discovery methods such as PiCoDeS on two attribute datasets. The evaluation shows that our approach is able to select the automatic discovery approach that discovers the most meaningful attributes. We then employ the best discovery approach to generate keywords for videos recorded from a surveillance system. This work shows it is possible to massively reduce the amount of manual work in generating video keywords without limiting ourselves to a particular video feature descriptor.
Towards robust face recognition for Intelligent-CCTV based surveillance using one gallery image

In recent years, the use of Intelligent Closed-Circuit Television (ICCTV) for crime prevention and detection has attracted significant attention. Existing face recognition systems require passport-quality photos to achieve good performance. However, use of CCTV images is much more problematic due to large variations in illumination, facial expressions and pose angle. In this paper we propose a pose variability compensation technique, which synthesizes realistic frontal face images from non-frontal views. It is based on modelling the face via Active Appearance Models and detecting the pose through a correlation model. The proposed technique is coupled with adaptive principal component analysis (APCA), which was previously shown to perform well in the presence of both lighting and expression variations. Experiments on the FERET dataset show up to 6 fold performance improvements. Finally, in addition to implementation and scalability challenges, we discuss issues related to on-going real life trials in public spaces using existing surveillance hardware.
Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching

A convenient way of dealing with image sets is to represent them as points on Grassmannian manifolds. While several recent studies explored the applicability of discriminant analysis on such manifolds, the conventional formalism of discriminant analysis suffers from not considering the local structure of the data. We propose a discriminant analysis approach on Grassmannian manifolds, based on a graph-embedding framework. We show that by introducing within-class and between-class similarity graphs to characterise intra-class compactness and inter-class separability, the geometrical structure of data can be exploited. Experiments on several image datasets (PIE, BANCA, MoBo, ETH-80) show that the proposed algorithm obtains considerable improvements in discrimination accuracy, in comparison to three recent methods: Grassmann Discriminant Analysis (GDA), Kernel GDA, and the kernel version of Affine Hull Image Set Distance. We further propose a Grassmannian kernel, based on canonical correlation between subspaces, which can increase discrimination accuracy when used in combination with previous Grassmannian kernels.
Experimental Analysis of Face Recognition on Still and CCTV Images

Although automatic identity inference based on faces has shown success when using high quality images, for CCTV based images it is hard to attain similar levels of performance. Furthermore, compared to recognition based on static images, relatively few studies have been done for video based face recognition. In this paper, we present an empirical analysis and comparison of face recognition using high quality and CCTV images in several important aspects: image quality (including resolution, noise, blurring and interlacing) as well as geometric transformations (such as translations, rotations and scale changes). The results show that holistic face recognition can be tolerant to image quality degradation but can also be highly influenced by geometric transformations. In addition, we show that camera intrinsics have much influence - when using different cameras for collecting gallery and probe images the recognition rate is considerably reduced. We also show that the classification performance can be considerably improved by straightforward averaging of consecutive face images from a CCTV video sequence.
Texture classification using nonparametric Markov random fields

We present a nonparametric Markov random field model for classifying texture in images. This model can capture the characteristics of a wide variety of textures, varying from the highly structured to the stochastic. The power of our modelling technique is evident in that only a small training image is required, even when the training texture contains long range characteristics. We show how this model can be used for unsupervised segmentation and classification of images containing textures for which we have no prior knowledge of the constituent texture types. This technique can therefore be used to find a specific texture in a background of unknown textures.
Image-set face recognition based on transductive learning

In this paper we consider the problem of face recognition in a scenario when the query consists of a set of images and the gallery contains a single still image per subject. This is a more challenging problem compared to image-set to image-set matching and has wider applications in advanced surveillance, smart access control and human-computer interaction. Unfortunately most of the previous matching strategies in literature fail to work or deteriorate drastically if they are provided with one sample per class as the gallery data. In this paper we demonstrate how transductive learning can be utilized to map the image-set to single image matching problem into the recently-studied framework of set matching using canonical correlations. Experimental results on different challenging datasets reveal the efficiency of the proposed method against existing approaches.