Reviewerly/exHarmony huggingface dataset:
Graph Embedding using Multi-Layer Adjacent Point Merging Model

For graph classification tasks, many traditional kernel methods focus on measuring the similarity between graphs. These methods have achieved great success on resolving graph isomorphism problems. However, in some classification problems, the graph class depends on not only the topological similarity of the whole graph, but also constituent subgraph patterns. To this end, we propose a novel graph embedding method using a multi-layer adjacent point merging model. This embedding method allows us to extract different sub-graph patterns from train-data. Then we present a flexible loss function for feature selection which enhances the robustness of our method for different classification problems. Finally, numerical evaluations demonstrate that our proposed method outperforms many state-of-the-art methods.
A Comparative Study on Fuzzy Clustering for Cloud Computing. Taking Web Service as a case

This study addresses the increasingly encountered challenge of data clustering. We present a comparative study to data clustering for cloud computing using Fuzzy C-MEANS and Adaptive Resonance Theory. To reduce variance and improve generalization ability, we used a resampling method based on 10-fold cross-validation. The typical initialization scheme is applied to improve the convergence speed of training and thus, reach the optimal solution. Experimental results on cloud computing datasets showed that the typical initialization-based Fuzzy Adaptive Resonance Theory model is effective and achieves improved accuracy for pattern recognition task compared to Fuzzy C-MEANS.
Stress-Lysis: An IoMT-Enabled Device for Automatic Stress Level Detection from Physical Activities

This is an extended abstract for a Research Demo Session based on our published article [1]. Physiological signals inside the human body depend on various factors. One of those important factors is psychological stress. Long term exposure to stress has many negative effects which may lead to major health issues such as cancer. Monitoring such long term, high impact stress is very important to maintain a healthy emotional balance. Keeping this in mind, Stress-Lysis, a smart healthcare framework is proposed. Through Stress-Lysis, an approach is proposed to not just monitor stress but also allow the user to live a happy, stress-free life. This is achieved with a wearable, edge level processing device.
Bayesian Hierarchical Model for Change Point Detection in Multivariate Sequences

Motivated by the wind turbine anomaly detection, we propose a Bayesian hierarchical model (BHM) for the mean-change detection in multivariate sequences. By combining the exchange random order distribution induced from the Poisson–Dirichlet process and nonlocal priors, BHM exhibits satisfactory performance for mean-shift detection with multivariate sequences under different error distributions. In particular, BHM yields the smallest detection error compared with other competitive methods considered in the article. We use a local scan procedure to accelerate the computation, while the anomaly locations are determined by maximizing the posterior probability through dynamic programming. We establish consistency of the estimated number and locations of the change points and conduct extensive simulations to evaluate the BHM approach. Among the popular change point detection algorithms, BHM yields the best performance for most of the datasets in terms of the F1 score for the wind turbine anomaly detection.
History Utterance Embedding Transformer LM for Speech Recognition

History utterances contain rich contextual information; however, better extracting information from the history utterances and using it to improve the language model (LM) is still challenging. In this paper, we propose the history utterance embedding Transformer LM (HTLM), which includes an embedding generation network for extracting contextual information contained in the history utterances and a main Transformer LM for current prediction. In addition, the two-stage attention (TSA) is proposed to encode richer contextual information into the embedding of history utterances (h-emb) while supporting GPU parallel training. Furthermore, we combine the extracted h-emb and embedding of current utterance (c-emb) through the dot-product attention and a fusion method for HTLM's current prediction. Experiments are conducted on the HKUST dataset and achieve a 23.4% character error rate (CER) on the test set. Compared with the baseline, the proposed method yields 12.86 absolute perplexity reduction and 0.8% absolute CER reduction.
Sharing the Load Online: Virtual Presentations with Virtual Co-Presenter Agents

The pandemic has caused a significant increase in the use of videoconferencing for oral presentations. Prior work demonstrated that an embodied conversational agent that co-delivers an oral presentation could be used in face-to-face presentations to reduce public speaking anxiety and increase presentation quality. In this work, we evaluate the use of a co-presenter agent in the delivery of virtual presentations given over a videoconferencing system, comparing them to presentations given without the agent. We found that participants were satisfied with the co-presenter agent, and those who liked the agent (scoring above the mean on a composite self-report measure of satisfaction) rated the presentations they gave with the agent as having significantly higher quality compared to those given without the agent. There was evidence the agent helped participants feel less nervous about their talks. Interviews confirmed these findings, and identified additional advantages and disadvantages of using co-presenter agents in virtual presentations.
Towards a Measurement Scale of Organizational Readiness for Personas

User studies have found persona application challenging. We argue that a potential reason for the challenges is the organization's readiness to apply personas. This research reports the on-going effort of developing the Persona Readiness Scale, a survey instrument for organizations' readiness for personas. The scale involves twenty-two items from seven dimensions: Need Readiness, Culture Readiness, Knowledge Readiness, Resource Readiness, Data and Systems Readiness, Capability Readiness, and Goal Readiness. Organizations can apply the current scale to evaluate their persona readiness but using the dimensions for statistical analyses requires further empirical validation.
Small Footprint Text-Independent Speaker Verification For Embedded Systems

Deep neural network approaches to speaker verification have proven successful, but typical computational requirements of State-Of-The-Art (SOTA) systems make them unsuited for embedded applications. In this work, we present a two-stage model architecture orders of magnitude smaller than common solutions (237.5K learning parameters, 11.5MFLOPS) reaching a competitive result of 3.31% Equal Error Rate (EER) on the well established VoxCeleb1 verification test set. We demonstrate the possibility of running our solution on small devices typical of IoT systems such as the Raspberry Pi 3B with a latency smaller than 200ms on a 5s long utterance. Additionally, we evaluate our model on the acoustically challenging VOiCES corpus. We report a limited increase in EER of 2.6 percentage points with respect to the best scoring model of the 2019 VOiCES from a Distance Challenge, against a reduction of 25.6 times in the number of learning parameters.
Deep Ensemble Siamese Network For Incremental Signal Classification

Incremental Signal Classification (ISC) aims to continuously identify and classify unknown signal categories, which is essentially an open-set classification task. In this paper, a new Deep Ensemble Siamese Network (DESN) is constructed for unknown category detection and incremental accumulation of signals from the detected category. Then the accumulated samples are used to update a One-dimensional Convolution Network (OCN) for incremental learning of new signal categories. Experimental results show that the proposed method can achieve accurate detection and accumulation of unknown signals, and is feasible for practical ISC.
A Nonparametric Hawkes Model for Forecasting California Seismicity

ABSTRACT A variety of nonparametric models have been proposed for estimating earthquake triggering. We investigate the ability of the model-independent stochastic declustering method developed by Marsan and Lengliné (2008) to estimate variable spatial triggering that can vary with direction, magnitude, and region. We develop an approach for local fault estimation and demonstrate forecasting methods that use the nonparametric estimates. Simulation studies are conducted to verify the effectiveness of the method, and the nonparametric estimates are applied to a California earthquake catalog. Model forecast performance is evaluated retrospectively by comparing our models with the long-term forecast of Helmstetter et al. (2007), using both deviance and Voronoi residuals. We show improved performance compared with Helmstetter et al. (2007) in various regions while using a full nonparametric estimation and forecasting approach.
Siamese-structure Deep Neural Network Recognizing Changes in Facial Expression According to the Degree of Smiling

A smile is a representative expression of happiness or high quality-of-life; however, automatic recognition of a smile according to happiness remains a challenging task. Because expressions of happiness are strongly dependent upon physical condition and occurrence of other emotions, and similar facial expression often occur under different emotions, we consider that there is no absolute visual pattern of a smile corresponding to happiness. Therefore, in this study, we assumed that a “smile with happiness” is observed as the temporal ascent in the degree of smiling and attempted to recognize this by capturing changes in facial expression within temporally sequential images. As an implementation of this scheme, we proposed a Siamese-structure deep neural network to compare facial expressions in two input images and estimate the existence of smile ascension or descension. For primal analysis of the proposed network, we developed a unique smiling dataset containing image pairs with various changes in smiling degree, including slight changes. The results demonstrated that the proposed method achieved nearly perfect recognition with > 0.95 accuracy when recognizing changes in the degree of smiling that humans certainly recognize. Attention regions that contributed to the predicted labels were concentrated on the mouth, cheeks, and tail of the eyes, which indicates a reasonable function for recognizing changes in smiling degree was constructed by the proposed method.
LINN: Lifting Inspired Invertible Neural Network for Image Denoising

In this paper, we propose an invertible neural network for image denoising (DnINN) inspired by the transform-based denoising framework. The proposed DnINN consists of an invertible neural network called LINN whose architecture is inspired by the lifting scheme in wavelet theory and a sparsity-driven denoising network which is used to remove noise from the transform coefficients. The denoising operation is performed with a single soft-thresholding operation or with a learned iterative shrinkage thresholding network. The forward pass of LINN produces an over-complete representation which is more suitable for denoising. The denoised image is reconstructed using the backward pass of LINN using the output of the denoising network. The simulation results show that the proposed DnINN method achieves results comparable to the DnCNN method while only requiring 1/4 of learnable parameters.
Enriching Video Captions With Contextual Text

Understanding video content and generating caption with context is an important and challenging task. Unlike prior methods that typically attempt to generate generic video captions without context, our architecture contextualizes captioning by infusing extracted information from relevant text data. We propose an end-to-end sequence-to-sequence model which generates video captions based on visual input, and mines relevant knowledge such as names and locations from contextual text. In contrast to previous approaches, we do not preprocess the text further, and let the model directly learn to attend over it. Guided by the visual input, the model is able to copy words from the contextual text via a pointer-generator network, allowing to produce more specific video captions. We show competitive performance on the News Video Dataset and, through ablation studies, validate the efficacy of contextual video captioning as well as individual design choices in our model architecture.
An Empirical Analysis of Visual Features for Multiple Object Tracking in Urban Scenes

This paper addresses the problem of selecting appearance features for multiple object tracking (MOT) in urban scenes. Over the years, a large number of features has been used for MOT. However, it is not clear whether some of them are better than others. Commonly used features are color histograms, histograms of oriented gradients, deep features from convolutional neural networks and re-identification (ReID) features. In this study, we assess how good these features are at discriminating objects enclosed by a bounding box in urban scene tracking scenarios. Several affinity measures, namely the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and the Bhattacharyya distances, Rank-1 counts and the cosine similarity, are also assessed for their impact on the discriminative power of the features. Results on several datasets show that features from ReID networks are the best for discriminating instances from one another regardless of the quality of the detector. If a ReID model is not available, color histograms may be selected if the detector has a good recall and there are few occlusions; otherwise, deep features are more robust to detectors with lower recall.
Robust Binary Loss for Multi-Category Classification with Label Noise

Deep learning has achieved tremendous success in image classification. However, the corresponding performance leap relies heavily on large-scale accurate annotations, which are usually hard to collect in reality. It is essential to explore methods that can train deep models effectively under label noise. To address the problem, we propose to train deep models with robust binary loss functions. To be specific, we tackle the K-class classification task by using K binary classifiers. We can immediately use multi-category large margin classification approaches, e.g., Pairwise-Comparison (PC) or One-Versus-All (OVA), to jointly train the binary classifiers for multi-category classification. Our method can be robust to label noise if symmetric functions, e.g., the sigmoid loss or the ramp loss, are employed as the binary loss function in the framework of risk minimization. The learning theory reveals that our method can be inherently tolerant to label noise in multi-category classification tasks. Extensive experiments over different datasets with different types of label noise are conducted. The experimental results clearly confirm the effectiveness of our method.
Gaze Estimation Using Neural Network And Logistic Regression

Abstract Currently, a large number of mature methods are available for gaze estimation. However, most regular gaze estimation approaches require additional hardware or platforms with professional equipment for data collection or computing that typically involve high costs and are relatively tedious. Besides, the implementation is particularly complex. Traditional gaze estimation approaches usually require systematic prior knowledge or expertise for practical operations. Moreover, they are primarily based on the characteristics of pupil and iris, which uses pupil shapes or infrared light and iris glint to estimate gaze, requiring high-quality images shot in special environments and other light source or professional equipment. We herein propose a two-stage gaze estimation method that relies on deep learning methods and logistic regression, which can be applied to various mobile platforms without additional hardware devices or systematic prior knowledge. A set of automatic and fast data collection mechanism is designed for collecting gaze images through a mobile platform camera. Additionally, we propose a new annotation method that improves the prediction accuracy and outperforms the traditional gridding annotation method. Our method achieves good results and can be adapted to different applications.
Denoising Synthesis: A module for fast image synthesis using denoising-based models

Many different types of generative models have been applied to image synthesis, the task of producing new images from a certain dataset. One newer class of generative models is denoising-based models, which take random noise and create an image from this by gradually removing the noise. We introduce a module called Denoising Synthesis to train these denoising-based generative models and generate new images from them. Our module is both flexible and simple to use, and provides helpful functionality, including a method of greatly increasing the image generation speed of the models.
Toward Intelligence in Communication Networks: A Deep Learning Identification Strategy for Radio Frequency Fingerprints

In the modern privacy-sensitive environment, identifying a particular radio device among a larger pool of Internet-of-Thing (IoT) facilities has become more difficult than ever. In particular, the identification process is susceptible to security, mobility, and environment changes. Machine learning techniques, on the other hand, offer a unique learning mechanism to reconstruct corrupted signals from noise and interference at wireless transceiver chains, providing a stable and accurate system for applications on wireless communication. In this paper, we exploit a Deep Learning (DL)-based identification strategy for Radio Frequency (RF) Fingerprints. Specifically, we introduce an Echo State Network (ESN), uniquely suited for nonlinear information processing, to discover dissimilar RF Fingerprints directly from raw transmitted signals, allowing the network to discriminate radio devices. Through an over-the-air WiFi transmission dataset, numerical evaluations demonstrate advantages of the ESN over the cutting-edge DL-based identification approaches, yielding an average classification accuracy of 98.11% while significantly reducing the training overhead. Furthermore, compared to our baseline models with various DL architectures, the ESN can be trained even with a very limited training set without degrading its accuracy.
Comparative Analysis of Question Answering Models for HRI Tasks with NAO in Spanish

Recent studies on Human Robot Interaction (HRI) have shown that different types of applications that combine metrics and techniques can help achieve a more efficient and organic interaction. This applications can be related to human care or go further and use a humanoid robot for nonverbal communication tasks. Similarly, for verbal communication, we found Question Answering, a Natural Language Processing task, that is in charge of capturing and interpret a question automatically and return a good representation of an answer. Also, recent work on creating Question Answering models, based on the Transformer architecture, have obtained state-of-the-art results. Our main goal in this project is to build a new Human Robot Interaction technique which uses a Question Answering system where we will test with college students. In the creation of the Question Answering model, we get results from state-of-the-art pre-trained models like BERT or XLNet, but also multilingual ones like m-BERT or XLM. We train them with a new Spanish dataset translated from the original SQuAD getting our best results with XLM-R, obtaining 68.1 F1 and 45.3 EM in the MLQA test dataset, and, 77.9 F1 and 58.3 EM for XQuAD test dataset. To validate the results obtained, we evaluated the project based on HRI metrics and a survey. The results demonstrate a high degree of acceptance in the students about the type of interaction that has been proposed.
Scene Completeness-Aware Lidar Depth Completion for Driving Scenario

This paper introduces Scene Completeness-Aware Depth Completion (SCADC) to complete raw lidar scans into dense depth maps with fine and complete scene structures. Recent sparse depth completion for lidars only focuses on the lower scenes and produces irregular estimations on the upper because existing datasets, such as KITTI, do not provide groundtruth for upper areas. These areas are considered less important since they are usually sky or trees of less scene understanding interest. However, we argue that in several driving scenarios such as large trucks or cars with loads, objects could extend to the upper parts of scenes. Thus depth maps with structured upper scene estimation are important for RGBD algorithms. SCADC adopts stereo images that produce disparities with better scene completeness but are generally less precise than lidars, to help sparse lidar depth completion. To our knowledge, we are the first to focus on scene completeness of sparse depth completion. We validate our SCADC on both depth estimate precision and scene-completeness on KITTI. Moreover, we experiment on less-explored outdoor RGBD semantic segmentation with scene completeness-aware D-input to validate our method.
CoasterChat: Exploring Digital Communication between People with Early Stage Dementia and Family Members Embedded in a Daily Routine

People with dementia living at home experience difficulties in participating in social interactions, while staying in contact with family adds quality to their lived experience. Initiating communication can be challenging for family, since there is a natural disparity between their life patterns and those of people with dementia. In this paper, we present CoasterChat, an exploration in design that embeds asynchronous digital communication in a daily coffee routine to support social sharing between people with early stage dementia and their family. In this preliminary study, the aim is to explore suitable interaction design that provides an opportunity for technology to be embedded in existing individual routines. Initial results show the importance of personalization in digital guidance. Benefit for family who engage in the interaction lies in the flexibility and accessibility of asynchronous communication. Finally, we discuss that these routines enhance social relationships which demonstrates the opportunities of creating a meaningful interaction between people with dementia and family members.
Persona Analytics: Implementing Mouse-Tracking for an Interactive Persona System

Observing user interactions with interactive persona systems offers important insights for the design and application of such systems. Using an interactive persona system, user behavior and interaction with personas can be tracked with high precision, addressing the scarcity of behavioral persona user studies. In this research, we introduce and evaluate an implementation of persona analytics based on mouse tracking, which offers researchers new possibilities for conducting persona user studies, especially during times when in-person user studies are challenging to carry out.