Reviewerly/exHarmony huggingface dataset:
Eye-dominance-guided Foveated Rendering

Optimizing rendering performance is critical for a wide variety of virtual reality (VR) applications. Foveated rendering is emerging as an indispensable technique for reconciling interactive frame rates with ever-higher head-mounted display resolutions. Here, we present a simple yet effective technique for further reducing the cost of foveated rendering by leveraging ocular dominance - the tendency of the human visual system to prefer scene perception from one eye over the other. Our new approach, eye-dominance-guided foveated rendering (EFR), renders the scene at a lower foveation level (with higher detail) for the dominant eye than the non-dominant eye. Compared with traditional foveated rendering, EFR can be expected to provide superior rendering performance while preserving the same level of perceived visual quality.
Students’ Academic Use of Mobile Technology and Higher-Order Thinking Skills: The Role of Active Engagement

The academic use of mobile technology engages students beyond traditional classroom contexts. Over the past few years, higher education institutions have promoted students’ learning and growth by supporting their use of mobile technology. Mobile technology offers educational possibilities that can enhance students’ growth in higher education. The aim of this study is to examine the relationship between college students’ academic use of mobile technology and higher-order thinking skills through their active engagement and learning effort. The sample consisted of 456 students at a university in South Korea, and the data were analyzed using partial least squares structural equation modeling. The results suggest that the academic use of mobile technology influences students’ higher-order thinking skills directly, in addition to their learning effort and active engagement in courses. These findings provide valuable information for higher education institutions that seek to introduce interactive and technology-integrated environments.
Research on the application of spatial partial differential equation in user oriented information mining

In this paper, the application of spatial partial differential equation in user intent oriented information mining is proposed. The hybrid mining algorithm is used to avoid the defects of a single algorithm, which makes the discovery and identification of user intent efficient and practical. The v-svc algorithm can suppress the classification of errors, the hierarchical generation based on attributes can easily get the focus of services, and the fuzzy clustering method can get the service level or service quality of users. This method can be used for any complex object research similar to user intention and the evaluation mechanism of a research object. In view of the problems of traditional methods, this paper puts forward the technology of association mining based on the mathematical model of partial differential classification, and makes an experimental comparative analysis. The results show that the convergence, stability and performance of this method are good.
Topic-driven toxicity: Exploring the relationship between online toxicity and news topics

Hateful commenting, also known as 'toxicity', frequently takes place within news stories in social media. Yet, the relationship between toxicity and news topics is poorly understood. To analyze how news topics relate to the toxicity of user comments, we classify topics of 63,886 online news videos of a large news channel using a neural network and topical tags used by journalists to label content. We score 320,246 user comments from those videos for toxicity and compare how the average toxicity of comments varies by topic. Findings show that topics like Racism, Israel-Palestine, and War & Conflict have more toxicity in the comments, and topics such as Science & Technology, Environment & Weather, and Arts & Culture have less toxic commenting. Qualitative analysis reveals five themes: Graphic videos, Humanistic stories, History and historical facts, Media as a manipulator, and Religion. We also observe cases where a typically more toxic topic becomes non-toxic and where a typically less toxic topic becomes "toxicified" when it involves sensitive elements, such as politics and religion. Findings suggest that news comment toxicity can be characterized as topic-driven toxicity that targets topics rather than as vindictive toxicity that targets users or groups. Practical implications suggest that humanistic framing of the news story (i.e., reporting stories through real everyday people) can reduce toxicity in the comments of an otherwise toxic topic.
An oversampling framework for imbalanced classification based on Laplacian eigenmaps

Imbalanced classification is a challenging problem in machine learning and data mining. Oversampling methods, such as the Synthetic Minority Oversampling Technique (SMOTE), generate synthetic data to achieve data balance for imbalanced classification. However, such kind of oversampling methods generates unnecessary noise when the data are not well separated. On the other hand, there are many applications with inadequate training data and vast testing data, making the imbalanced classification much more challenging. In this paper, we propose a novel oversampling framework to achieve the following two objectives. (1) Improving the classification results of the SMOTE based oversampling methods; (2) Making the SMOTE based oversampling methods applicable when the training data are inadequate. The proposed framework utilizes the Laplacian eigenmaps to find an optimal dimensional space, where the data are well separated and the generation of noise by SMOTE based oversampling methods can be avoided. The construction of graph Laplacian not only explores the useful information from the unlabeled testing data to facilitate imbalanced learning, but also makes the learning process incremental. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed framework.
Genetic Algorithm: An Approach on Optimization

Solutions for both constrained and unconstrained problems of optimization pose a challenge from the past till date. The genetic algorithm is a technique for solving such optimization problems based on biological laws of evolution particularly natural selection. In simple terms, a genetic algorithm is a successor to the traditional evolutionary algorithm where at each step it will select random solutions from the present population and labels those as parents and uses them to reproduce to the next generation as children with a series of biological operations, namely reproduction, selection, crossover and mutation.
Relational Deep Feature Learning for Heterogeneous Face Recognition

Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity's relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function (C-softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database.
Improved Initialization of the EM Algorithm for Mixture Model Parameter Estimation

A commonly used tool for estimating the parameters of a mixture model is the Expectation–Maximization (EM) algorithm, which is an iterative procedure that can serve as a maximum-likelihood estimator. The EM algorithm has well-documented drawbacks, such as the need for good initial values and the possibility of being trapped in local optima. Nevertheless, because of its appealing properties, EM plays an important role in estimating the parameters of mixture models. To overcome these initialization problems with EM, in this paper, we propose the Rough-Enhanced-Bayes mixture estimation (REBMIX) algorithm as a more effective initialization algorithm. Three different strategies are derived for dealing with the unknown number of components in the mixture model. These strategies are thoroughly tested on artificial datasets, density–estimation datasets and image–segmentation problems and compared with state-of-the-art initialization methods for the EM. Our proposal shows promising results in terms of clustering and density-estimation performance as well as in terms of computational efficiency. All the improvements are implemented in the rebmix R package.
Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks

We present a method for weakly-supervised action localization based on graph convolutions. In order to find and classify video time segments that correspond to relevant action classes, a system must be able to both identify discriminative time segments in each video, and identify the full extent of each action. Achieving this with weak video level labels requires the system to use similarity and dissimilarity between moments across videos in the training data to understand both how an action appears, as well as the subactions that comprise the action's full extent. However, current methods do not make explicit use of similarity between video moments to inform the localization and classification predictions. We present a novel method that uses graph convolutions to explicitly model similarity between video moments. Our method utilizes similarity graphs that encode appearance and motion, and pushes the state of the art on THUMOS'14, ActivityNet 1.2, and Charades for weakly- supervised action localization.
Multimodal sentiment analysis based on multi-head attention mechanism

Multimodal sentiment analysis is still a promising area of research, which has many issues needed to be addressed. Among them, extracting reasonable unimodal features and designing a robust multimodal sentiment analysis model is the most basic problem. This paper presents some novel ways of extracting sentiment features from visual, audio and text, furthermore use these features to verify the multimodal sentiment analysis model based on multi-head attention mechanism. The proposed model is evaluated on Multimodal Opinion Utterances Dataset (MOUD) corpus and CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus for multimodal sentiment analysis. Experimental results prove the effectiveness of the proposed approach. The accuracy of the MOUD and MOSI datasets is 90.43% and 82.71%, respectively. Compared to the state-of-the-art models, the improvement of the performance are approximately 2 and 0.4 points.
Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation

Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand. Current approaches often rely on a popular body pose algorithm, such as the Convolutional Pose Machine (CPM), to learn 2D keypoint features. These algorithms cannot adequately address the unique challenges of hand pose estimation, because they are trained solely based on keypoint positions without seeking to explicitly model structural relationship between them. We propose a novel Nonparametric Structure Regularization Machine (NSRM) for 2D hand pose estimation, adopting a cascade multi-task architecture to learn hand structure and keypoint representations jointly. The structure learning is guided by synthetic hand mask representations, which are directly computed from keypoint positions, and is further strengthened by a novel probabilistic representation of hand limbs and an anatomically inspired composition strategy of mask synthesis. We conduct extensive studies on two public datasets - OneHand 10k and CMU Panoptic Hand. Experimental results demonstrate that explicitly enforcing structure learning consistently improves pose estimation accuracy of CPM baseline models, by 1.17% on the first dataset and 4.01% on the second one. The implementation and experiment code is freely available online <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> . Our proposal of incorporating structural learning to hand pose estimation requires no additional training information, and can be a generic add-on module to other pose estimation models.
A Geometric Approach to Obtain a Bird's Eye View From an Image

The objective of this paper is to rectify any monocular image by computing a homography matrix that transforms it to a geometrically correct bird's eye (overhead) view. We make the following contributions: (i) we show that the homography matrix can be parameterised with only four geometric parameters that specify the horizon line and the vertical vanishing point, or only two if the field of view or focal length is known; (ii) We introduce a novel representation for the geometry of a line or point (which can be at infinity) that is suitable for regression with a convolutional neural network (CNN); (iii) We introduce a large synthetic image dataset with ground truth for the orthogonal vanishing points, that can be used for training a CNN to predict these geometric entities; and finally (iv) We achieve state-of-the-art results on horizon detection, with 74.52% AUC on the Horizon Lines in the Wild dataset. Our method is fast and robust, and can be used to remove perspective distortion from videos in real time.
Noise Reduction in Coherence Scanning Interferometry for Surface Topography Measurement

Abstract Coherence scanning interferometry is one of the most accurate surface measuring technologies, and it is increasingly applied to challenging surface structures, such as additive manufactured parts and transparent films, directly in environments that resemble production areas more than metrology labs. Environmental disturbances may further compromise measurement accuracy. Data acquisition strategies to reduce measurement noise in coherence scanning interferometry include averaging a sequence of repeated topography measurements or increasing the sampling frequency of the fringe signal during a single data acquisition—sometimes referred to as oversampling. In this paper, we improve the understanding of the mechanisms of the two noise reduction methods and compare their effects on surface topography measurement in the presence of environment-induced vibration. The results provide guidance for good practice in the reduction of uncertainty in surface measurement for a wide range of applications.
Prague: High-Performance Heterogeneity-Aware Asynchronous Decentralized Training

Distributed deep learning training usually adopts All-Reduce as the synchronization mechanism for data parallel algorithms due to its high performance in homogeneous environment. However, its performance is bounded by the slowest worker among all workers. For this reason, it is significantly slower in heterogeneous settings. AD-PSGD, a newly proposed synchronization method which provides numerically fast convergence and heterogeneity tolerance, suffers from deadlock issues and high synchronization overhead. Is it possible to get the best of both worlds --- designing a distributed training method that has both high performance like All-Reduce in homogeneous environment and good heterogeneity tolerance like AD-PSGD?
Domain Adaptation for Vehicle Detection from Bird's Eye View LiDAR Point Cloud Data

Point cloud data from 3D LiDAR sensors are one of the most crucial sensor modalities for versatile safety-critical applications such as self-driving vehicles. Since the annotations of point cloud data is an expensive and time-consuming process, therefore recently the utilisation of simulated environments and 3D LiDAR sensors for this task started to get some popularity. However, the generated synthetic point cloud data are still missing the artefacts usually exist in point cloud data from real 3D LiDAR sensors. Thus, in this work, we are proposing a domain adaptation framework for bridging this gap between synthetic and real point cloud data. Our proposed framework is based on the deep cycle-consistent generative adversarial networks (CycleGAN) architecture. We have evaluated the performance of our proposed framework on the task of vehicle detection from a bird's eye view (BEV) point cloud images coming from real 3D LiDAR sensors. The framework has shown competitive results with an improvement of more than 7% in average precision score over other baseline approaches when tested on real BEV point cloud images.
Dual-Stream Multi-Path Recursive Residual Network for JPEG Image Compression Artifacts Reduction

JPEG is the most widely used lossy image compression standard. When using JPEG with high compression ratios, visual artifacts cannot be avoided. These artifacts not only degrade the user experience but also negatively affect many low-level image processing tasks. Recently, convolutional neural network (CNN)-based compression artifact removal approaches have achieved significant success, however, at the cost of high computational complexity due to an enormous number of parameters. To address this issue, we propose a dual-stream recursive residual network (STRRN) which consists of structure and texture streams for separately reducing the specific artifacts related to high-frequency or low-frequency image components. The outputs of these streams are combined and fed into an aggregation network to further enhance the restored images. By using parameter sharing, the proposed network reduces the total number of training parameters significantly. Moreover, experiments conducted on five commonly used datasets confirm that the proposed STRRN can efficiently reduce the compression artifacts, while using up to 4.6 times less training parameters and 5 times less running time compared to the state-of-the-art approaches.
CEREBRUM: a fast and fully-volumetric Convolutional Encoder-decodeR for weakly-supervised sEgmentation of BRain strUctures from out-of-the-scanner MRI

Many functional and structural neuroimaging studies call for accurate morphometric segmentation of different brain structures starting from image intensity values of MRI scans. Current automatic (multi-) atlas-based segmentation strategies often lack accuracy on difficult-to-segment brain structures and, since these methods rely on atlas-to-scan alignment, they may take long processing times. Alternatively, recent methods deploying solutions based on Convolutional Neural Networks (CNNs) are enabling the direct analysis of out-of-the-scanner data. However, current CNN-based solutions partition the test volume into 2D or 3D patches, which are processed independently. This process entails a loss of global contextual information, thereby negatively impacting the segmentation accuracy. In this work, we design and test an optimised end-to-end CNN architecture that makes the exploitation of global spatial information computationally tractable, allowing to process a whole MRI volume at once. We adopt a weakly supervised learning strategy by exploiting a large dataset composed of 947 out-of-the-scanner (3 Tesla T1-weighted 1mm isotropic MP-RAGE 3D sequences) MR Images. The resulting model is able to produce accurate multi-structure segmentation results in only a few seconds. Different quantitative measures demonstrate an improved accuracy of our solution when compared to state-of-the-art techniques. Moreover, through a randomised survey involving expert neuroscientists, we show that subjective judgements favour our solution with respect to widely adopted atlas-based software.
PRNU‐based detection of facial retouching

Nowadays, many facial images are acquired using smart phones. To ensure the best outcome, users frequently retouch these images before sharing them, e.g. via social media. Modifications resulting from used retouching algorithms might be a challenge for face recognition technologies. Towards deploying robust face recognition as well as enforcing anti-photoshop legislations, a reliable detection of retouched face images is needed. In this work, the effects of facial retouching on face recognition are investigated. A qualitative assessment of 32 beautification apps is conducted. Based on this assessment five apps are chosen which are used to create a database of 800 beautified face images. Biometric performance is measured before and after retouching using a commercial face recognition system. Subsequently, a retouching detection system based on the analysis of photo response non-uniformity (PRNU) is presented. Specifically, scores obtained from analysing spatial and spectral features extracted from PRNU patterns across image cells are fused. In a scenario, in which unaltered bona fide images are compressed to the average sizes of the retouched images using JPEG, the proposed PRNU-based detection scheme is shown to robustly distinguish between bona fide and retouched images achieving an average detection equal error rate of 13.7% across all retouching algorithms.
Towards Area-Efficient Optical Neural Networks: An FFT-based Architecture

As a promising neuromorphic framework, the optical neural network (ONN) demonstrates ultra-high inference speed with low energy consumption. However, the previous ONN architectures have high area overhead which limits their practicality. In this paper, we propose an area-efficient ONN architecture based on structured neural networks, leveraging optical fast Fourier transform for efficient computation. A two-phase software training flow with structured pruning is proposed to further reduce the optical component utilization. Experimental results demonstrate that the proposed architecture can achieve 2.2~3.7× area cost improvement compared with the previous singular value decomposition-based architecture with comparable inference accuracy.