Reviewerly/exHarmony huggingface dataset:
Scalable Kernel $k$-Means With Randomized Sketching: From Theory to Algorithm

Kernel <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means is a fundamental unsupervised learning in data mining. Its computational requirements are typically at least quadratic in the number of data, which are prohibitive for large-scale scenarios. To address these issues, we propose a novel randomized sketching approach SKK based on the circulant matrix. SKK projects the kernel matrix left and right according to the proposed sketch matrices to obtain a smaller one and accelerates the matrix-matrix product by the fast Fourier transform based on the circulant matrix, which can greatly reduce the computational requirements of the approximate kernel <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means estimator with the same generalization bound as the exact kernel <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means in the statistical setting. In particular, theoretical analysis shows that taking the sketch dimension of <inline-formula><tex-math notation="LaTeX">$\sqrt{n}$</tex-math></inline-formula> is sufficient for SKK to achieve the optimal excess risk bound with only a fraction of computations, where <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> is the number of data. The extensive experiments verify our theoretical analysis, and SKK achieves the state-of-the-art performances on 12 real-world datasets. To the best of our knowledge, in randomized sketching, this is the first time that unsupervised learning makes such a significant breakthrough.
Combining User Specific and Global News Features for Neural News Recommendation

News recommendation engines play a vital role in online news services. Such systems help users discover news articles they may want to read, thus increasing user experience and engagement. Existing news recommendation methods usually model users' interest from their reading history by learning user and news representations. Those methods represent news by features specific to users to create personalized user interest models and recommend news similar to user interest. However, in many cases, users may also read the news that is not in their usual interest, for example, breaking news. Such news often has global characteristics that are independent of specific users. In this paper, we propose a neural news recommendation method that combines user-specific news models with global news models to better deal with those situations. To create informative news models, we adopt an attentive neural approach that can learn representations of the news from different news aspects such as title, category, and news contents. The attentive mechanism allows the model to focus on an aspect that attracts global interest from different users. Experiments on a real-world dataset show that our model achieves improved performance over the state-of-the-art news recommendation method and is able to recommend candidate news of both types: those are relevant to a specific user and those are of common interest for different users.
Publishing Climate Data as Linked Data Via Virtual Knowledge Graphs

With the active development of ICT and Internet technologies in climate research, individuals often need to gather different and disparate datasets and preprocess them in preparation for downstream data analysis in order to have a more full understanding of the challenges. This preparatory procedure is often lengthy due to the primary issue that data providers can-not ensure a homogeneous data format for data integration purposes. To overcome this problem, this study proposes enhancing existing relational climate data by layering a virtual knowledge graph on top of the original databases provided by various data vendors. The primary benefit of doing this is that data consumers are able to simply integrate climate data with other data sources using Linked Data principles, and climate data producers do not have to modify their data to conform to standard knowledge graph protocols.
Embedded Implementation of the Hypersphere Neural Network for Energy Consumption Monitoring

The combination of the Internet of Things and Machine Learning has promoted the development of new technological approaches such as Edge Computing and Tiny Machine Learning. The contribution of this paper is the implementation of the Hypersphere Neural Network using the NodeMCU board and the Esp8266 microcontroller for energy consumption monitoring. The energy consumption monitoring consists of recognising the device operating with an IoT device. We use our IoT device for evaluating the performance of the embedded implementation of the Hypersphere Neural Network. The implementation of the Hypersphere Neural Network is carried out from the geometric algebra and conformal geometric algebra viewpoints. The idea behind the design and implementation of the Hypersphere Neural Network is to estimate hyperspheres which produce non-linear decision boundaries and separate the pattern classes. Our approach achieves 99.7 $$\%$$ and 99.4 $$\%$$ of classification success rate for training and validation stages respectively using a simple Hypersphere Neural Network topology.
Automated Flexible Needle Trajectory Planning for Keyhole Neurosurgery Using Reinforcement Learning

Planning a safe trajectory for minimally invasive (keyhole) neurosurgery procedures require avoiding critical anatomical structures such as blood vessels and ventricles while optimizing the needle trajectory parameters such as length and curvature to comply with the needle kinematics. In this paper, we propose a reinforcement learning-based method for obtaining kinematically feasible trajectories for flexible needle insertions. Proposed approach utilizes Bezier curve control points that are generated by a reward-based reinforcement learning framework, called Flexible Needle Path Generation (FNPG). FNPG framework is trained using an environment that consists of (1) critical structures (e.g. ventricles) obtained through atlas based segmentation of MRI-T1 images, (2) blood vessels segmented from MR angiography (MRA) data and, (3) simulated brain tumor with varying size and location. The curvilinear paths obtained through the FNPG framework are compared with the traditional sampling based algorithm RRT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> . The results show that the FNPG approach can produce smoother and shorter trajectories compared to RRT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> while avoiding the critical anatomical structures.
TMN: Temporal-guided Multiattention Network for Action Recognition

2D convolutional neural network, due to its low computational complexity and fast recognition speed, has attracted more and more attention from researchers in the field of video action recognition. Temporal shift and temporal differential, have made tremendous progress, but the lack of crucial spatiotemporal attention mechanism has led to huge performance loss. To address this issue, we propose a Temporal-guided Multiattention Network (TMN), which fully excavate and fuse spatio-temporal attention information for effective video action recognition. Concretely, the multi-attention module squeezes and expands spatio-temporal features to achieve weighting of corresponding regions for video in spatio-temporal dimensions, while the adaptive temporal guidance module imports temporal guiding signal to the spatial attention and re-weight the global temporal attention to accomplish the accurate temporal modeling. Extensive experiments and analyses show that our proposed temporal-guided multiattention network can achieve state-of-the-art promising video action recognition performance on the widely used benchmarks (HMDB51, UCF101 and Something-Something V1).
Online Adaptive Compensation for Model Uncertainty Using Extreme Learning Machine-based Control Barrier Functions

A control barrier functions-based quadratic programming (CBF-QP) method has emerged as a controller synthesis tool to assure safety of autonomous systems owing to the appealing safe forward invariant set. However, the provable safety relies on a precisely described dynamic model, which is not always available in practice. Recent works leverage learning to compensate model uncertainty for a CBF controller. However, these approaches based on reinforcement learning or episodic learning are limited to dealing with time-invariant uncertainty. Also, the reinforcement learning approach learns the uncertainty offline, while episodic learning only updates the controller after a batch of data is available by the end of an episode. Instead, we propose a novel tuning extreme learning machine (tELM)-based CBF controller that can compensate time-variant and time-invariant model uncertainty adaptively in an online manner. We validate our approach's effectiveness in a simulation of an Adaptive Cruise Control (ACC) system.
Robust Metric Boosts Transfer

Transfer metric learning (TML) aims to improve the metric learning in target domains by transferring knowledge from related tasks, where the distance metrics are strong and reliable. Existing TML approaches only focus on how to transfer the source metric knowledge, which is often prone to be over-fitting to the source domain. In this paper, we study how to train a source metric that is appropriate for transfer and then design a general deep TML method for effective metric transfer. In particular, we propose to learn the source metric parameterized by a deep neural network in an adversarial way and then transfer the metric to the target domain by embedding imitation, which allows the inputs of source and target domains to be heterogeneous. Besides, we restrict the size of the target metric network to be small so that the inference is efficient in the target domain. Results in the popular face verification application demonstrate the effectiveness of our method.
Black Hole Algorithm for BigData Anonymization

Due to advancements in technologies, data is increasing tremendously per day and preserving privacy for such datasets is complex. K-anonymity is the well Algorithm to preserve privacy which supports generalization. Although the scope of privacy protection is solved by generalization-based algorithm but fails to solve the information loss issue. Later on, a clustering-based approach was developed to enhance the information quality which also fails to find the optimal solution. Hence a unique approach of Black Hole Algorithm for Big Data to achieve anonymization to support all these limitations is proposed. Experiments results illustrate that data utility has been enhanced related to standard k-anonymity centered MapReduce besides Clustering-centered MapReduce approach.
Multi-property Tensor-Based Learning for Abnormal Event Detection

In this paper, we propose a novel abnormal event detection scheme for video surveillance systems using an unsupervised learning process. Our contribution includes intra and inter property feature encoding to reduce information redundancy within (intra) and across (inter) image features. Intra property encoding is carried out using convolutional auto-encoders. Inter-property encoding is performed using an unsupervised tensor-based learning mode to handle the dimensionality issue arising in cases when different properties are inter-related together. Comprehensive experiments are performed on two benchmarks: Avenue, and ShanghaiTech.
AMS: Adaptive Multiget Scheduling Algorithm for Distributed Key-Value Stores

Distributed key-value stores provide the Multiget API, where many key-value operations are batched together, to meet the parallel requirement of applications. Correspondingly, reducing the latency of Multigets is crucial for the responsiveness of the distributed key-value stores. The latency of a Multiget depends on both which replica server its key-value operations are scheduled to, i.e., the replica selection for each key-value operation, and when these key-value operations are served, i.e., the scheduling of the service sequence at different replica servers. Existing solutions solely focus on either one of them and accordingly lead to suboptimal latency for Multigets. To address these issues, this paper proposes an Adaptive Multiget Scheduling (AMS) algorithm in this paper, and specifically, our AMS re-architectures the framework to remove the conflict between replica selection and service sequence scheduling in the existing solution Rein. Based on the new framework, a sophisticated replica selection method is designed. Furthermore, AMS guides both replica selection and service sequence scheduling by the piggybacked information of replica servers, being adaptive to the heterogeneous time-varying server performance. Consequently, AMS can respectively reduce the median, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$95^{th}$</tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$99^{th}$</tex-math></inline-formula> percentile latencies of Multigets by a factor of 4, 3.1, and 1.86 compared to the default FIFO algorithm and significantly outperforms Rein.
Multiscale Sensor Fusion and Continuous Control with Neural CDEs

Though robot learning is often formulated in terms of discrete-time Markov decision processes (MDPs), physical robots require near-continuous multiscale feedback control. Machines operate on multiple asynchronous sensing modalities, each with different frequencies, e.g., video frames at 30Hz, proprioceptive state at 100Hz, force-torque data at 500Hz, etc. While the classic approach is to batch observations into fixed-time windows then pass them through feed-forward encoders (e.g., with deep networks), we show that there exists a more elegant approach - one that treats policy learning as modeling latent state dynamics in continuous-time. Specifically, we present InFuser, a unified architecture that trains continuous time-policies with Neural Controlled Differential Equations (CDEs). InFuser evolves a single latent state representation over time by (In)tegrating and (Fus)ing multi-sensory observations (arriving at different frequencies), and inferring actions in continuous-time. This enables policies that can react to multi-frequency multi-sensory feedback for truly end-to-end visuomotor control, without discrete-time assumptions. Behavior cloning experiments demonstrate that InFuser learns robust policies for dynamic tasks (e.g., swinging a ball into a cup) notably outperforming several baselines in settings where observations from one sensing modality can arrive at much sparser intervals than others.
Label-Dividing Gated Graph Neural Network for Hierarchical Text Classification

Multi-label hierarchical text classification (MLHTC) is an essential yet challenging task of natural language processing (NLP). Existing methods lack attention to predicting sibling labels. In addition, methods based on graph convolutional networks (GCN) meet the problem of over-smoothing, which further deepens the difficulty of distinguishing sibling labels. In this paper, we propose a label-dividing gated graph neural network (LD-GGNN), which can better distinguish sibling labels and achieve adaptive interaction between text and labels. We optimize gated graph neural network (GGNN) to accurately capture structural features of label hierarchy and deeply explore label dependence. Stronger nonlinear characteristics of GGNN are used to solve the problem of over-smoothing. Furthermore, we propose a dynamic label dividing mechanism (DLDM), which can guide the model to distinguish sibling labels by introducing a dividing bias. Compared with previous works, LD-GGNN achieves significant and consistent improvements on both Micro-F1 and Macro-F1 score on multiple datasets.
A Knowledge-Enhanced Adversarial Model for Cross-lingual Structured Sentiment Analysis

Structured sentiment analysis, which aims to extract the complex semantic structures such as holders, expressions, targets, and polarities, has obtained widespread attention from both industry and academia. Unfortunately, the existing structured sentiment analysis datasets refer to a few languages and are relatively small, limiting neural network models' performance. In this paper, we focus on the cross-lingual structured sentiment analysis task, which aims to transfer the knowledge from the source language to the target one. Notably, we propose a Knowledge-Enhanced Adversarial Model (KEAM) with both implicit distributed and explicit structural knowledge to enhance the cross-lingual transfer. First, we design an adversarial embedding adapter for learning an informative and robust representation by capturing implicit semantic information from diverse multi-lingual embeddings adaptively. Then, we propose a syntax GCN encoder to transfer the explicit semantic information (e.g., universal dependency tree) among multiple languages. We conduct experiments on five datasets and compare KEAM with both the supervised and unsupervised methods. The extensive experimental results show that our KEAM model outperforms all the unsupervised baselines in various metrics.
Advancing the Use of Information Compression Distances in Authorship Attribution

Detecting unreliable information in social media is an open challenge, in part as a result of the difficulty to associate a piece of information to known and trustworthy actors. The identification of the origin of sources can help society deal with unverified, incomplete, or even false information. In this work we tackle the problem of associating a piece of information to a certain politician. The use of inaccurate information is of great relevance in the case of politicians, since it affects social perception and voting behavior. Moreover, misquotation can be weaponized to hinder adversary reputation. We consider the task of applying a compression-based metric to conduct authorship attribution in social media, namely in Twitter. In specific, we leverage the Normalized Compression Distance (NCD) to compare an author’s text with other authors’ texts. We show that this methodology performs well, obtaining 80.3% accuracy in a scenario with 6 different politicians.
Tear Up the Bubble Boom: Lessons Learned From a Deep Learning Research and Development Cluster

With the proliferation of deep learning, there exists a strong need to efficiently operate GPU clusters for deep learning production in giant AI companies, as well as for research and development (R&D) in small-sized research institutes and universities. Existing works have performed thorough trace analysis on large-scale production-level clusters in giant companies, which discloses the characteristics of deep learning production jobs and motivates the design of scheduling frameworks. However, R&D clusters significantly differ from production-level clusters in both job properties and user behaviors, calling for a different scheduling mechanism. In this paper, we present a detailed workload characterization of an R&D cluster, CloudBrain-I, in a research institute, Peng Cheng Laboratory. After analyzing the fine-grained resource utilization, we discover a severe problem for R&D clusters, resource underutilization, which is especially important in R&D clusters while not characterised by existing works. We further investigate two specific underutilization phenomena and conclude several implications and lessons on R&D cluster scheduling. The traces will be open-sourced to motivate further studies in the community.
Continual Stereo Matching of Continuous Driving Scenes with Growing Architecture

The deep stereo models have achieved state-of-the-art performance on driving scenes, but they suffer from severe performance degradation when tested on unseen scenes. Although recent work has narrowed this performance gap through continuous online adaptation, this setup requires continuous gradient updates at inference and can hardly deal with rapidly changing scenes. To address these challenges, we propose to perform continual stereo matching where a model is tasked to 1) continually learn new scenes, 2) overcome forgetting previously learned scenes, and 3) continuously predict disparities at deployment. We achieve this goal by introducing a Reusable Architecture Growth (RAG) framework. RAG leverages task-specific neural unit search and architecture growth for continual learning of new scenes. During growth, it can maintain high reusability by reusing previous neural units while achieving good performance. A module named Scene Router is further introduced to adaptively select the scene-specific architecture path at inference. Experimental results demonstrate that our method achieves compelling performance in various types of challenging driving scenes.
Target-level Sentiment Analysis Based on Image and Text Fusion

Target-level sentiment analysis of social media posts has recently attracted increasing attentions, which task is to predict the sentimental label of target in posts published. Previous target-level sentiment analysis tasks usually only involve text modalities and fail to consider data sources of other modalities, which leads to low accuracy of sentiment analysis tasks. In order to improve the accuracy of sentiment analysis tasks, we propose a new target-level multimodal sentiment model that fuses features of visual and textual. Specifically, first, we use BERT and Bi-LSTM to extract the features of the target words and text, respectively, and then fuse them through a text fusion layer to obtain the final text representation. Next, we use ResNet-152 to extract image features, and then use attention mechanism to match target words and image features to extract target-related visual representations in the image. Finally, we linearly fuse the textual and visual representations using Multimodal fusion layer, and perform the sentiment label classification task after fusion.
Group RandAugment: Video Augmentation for Action Recognition

Data augmentation, as a critical strategy in deep learning, well improves the sample diversity for network training, leading to the obvious improvement of model generalization ability. Besides, automatic data augmentation, while sparking in image tasks, has attracted too little attention in video recognition task. Therefore, this work explores a novel group random augmentation (GRA) that automatically augments video data for recognition. GRA first collects 21 augmentation transformations to enrich the sample diversity, and then divides these transformations into four groups (i.e., pixel-transform, rigidtransform, erasing-transform, environment-transform) to reduce the excessive regularization caused by similar augmentation. To reduce the computational cost of finding the optimal setting, GRA adopts the combination form to select the optimal situation. For frames in the same video clip, GRA uses synchronous GRA. Additionally, the proposed GRA can be integrated into any existing video frameworks. To prove the effectiveness of GRA, we conduct experiments on two commonly used video action recognition benchmarks (Something-Something V1 & V2) and three typical frameworks (TSM, GSM, and TEA), whose results demonstrate GRA can improve performance in all cases without adding additional computational cost.