pwc-archive/papers-with-abstracts huggingface dataset:
HeRo 2.0: A Low-Cost Robot for Swarm Robotics Research
Authors: Paulo Rezeck, Hector Azpurua, Mauricio FS Correa, Luiz Chaimowicz
Date: 2022-02-24 00:00:00

The current state of electronic component miniaturization coupled with the increasing efficiency in hardware and software allow the development of smaller and compact robotic systems. The convenience of using these small, simple, yet capable robots has gathered the research community's attention towards practical applications of swarm robotics. This paper presents the design of a novel platform for swarm robotics applications that is low cost, easy to assemble using off-the-shelf components, and deeply integrated with the most used robotic framework available today: ROS (Robot Operating System). The robotic platform is entirely open, composed of a 3D printed body and open-source software. We describe its architecture, present its main features, and evaluate its functionalities executing experiments using a couple of robots. Results demonstrate that the proposed mobile robot is very effective given its small size and reduced cost, being suitable for swarm robotics research and education.
The rise of the lottery heroes: why zero-shot pruning is hard
Authors: Enzo Tartaglione
Date: 2022-02-24 00:00:00
Tasks: Deep Learning

Recent advances in deep learning optimization showed that just a subset of parameters are really necessary to successfully train a model. Potentially, such a discovery has broad impact from the theory to application; however, it is known that finding these trainable sub-network is a typically costly process. This inhibits practical applications: can the learned sub-graph structures in deep learning models be found at training time? In this work we explore such a possibility, observing and motivating why common approaches typically fail in the extreme scenarios of interest, and proposing an approach which potentially enables training with reduced computational effort. The experiments on either challenging architectures and datasets suggest the algorithmic accessibility over such a computational gain, and in particular a trade-off between accuracy achieved and training complexity deployed emerges.
Standard Deviation-Based Quantization for Deep Neural Networks
Authors: Amir Ardakani, Arash Ardakani, Brett Meyer, James J. Clark, Warren J. Gross
Date: 2022-02-24 00:00:00
Tasks: Quantization

Quantization of deep neural networks is a promising approach that reduces the inference cost, making it feasible to run deep networks on resource-restricted devices. Inspired by existing methods, we propose a new framework to learn the quantization intervals (discrete values) using the knowledge of the network's weight and activation distributions, i.e., standard deviation. Furthermore, we propose a novel base-2 logarithmic quantization scheme to quantize weights to power-of-two discrete values. Our proposed scheme allows us to replace resource-hungry high-precision multipliers with simple shift-add operations. According to our evaluations, our method outperforms existing work on CIFAR10 and ImageNet datasets and even achieves better accuracy performance with 3-bit weights and activations when compared to the full-precision models. Moreover, our scheme simultaneously prunes the network's parameters and allows us to flexibly adjust the pruning ratio during the quantization process.
Directed Graph Auto-Encoders
Authors: Georgios Kollias, Vasileios Kalantzis, Tsuyoshi Idé, Aurélie Lozano, Naoki Abe
Date: 2022-02-25 00:00:00
Tasks: Decoder, Link Prediction

We introduce a new class of auto-encoders for directed graphs, motivated by a direct extension of the Weisfeiler-Leman algorithm to pairs of node labels. The proposed model learns pairs of interpretable latent representations for the nodes of directed graphs, and uses parameterized graph convolutional network (GCN) layers for its encoder and an asymmetric inner product decoder. Parameters in the encoder control the weighting of representations exchanged between neighboring nodes. We demonstrate the ability of the proposed model to learn meaningful latent embeddings and achieve superior performance on the directed link prediction task on several popular network datasets.
MetaVA: Curriculum Meta-learning and Pre-fine-tuning of Deep Neural Networks for Detecting Ventricular Arrhythmias based on ECGs
Authors: Wenrui Zhang, Shijia Geng, Zhaoji Fu, Linlin Zheng, Chenyang Jiang, Shenda Hong
Date: 2022-02-25 00:00:00
Tasks: Diversity, Meta-Learning

Ventricular arrhythmias (VA) are the main causes of sudden cardiac death. Developing machine learning methods for detecting VA based on electrocardiograms (ECGs) can help save people's lives. However, developing such machine learning models for ECGs is challenging because of the following: 1) group-level diversity from different subjects and 2) individual-level diversity from different moments of a single subject. In this study, we aim to solve these problems in the pre-training and fine-tuning stages. For the pre-training stage, we propose a novel model agnostic meta-learning (MAML) with curriculum learning (CL) method to solve group-level diversity. MAML is expected to better transfer the knowledge from a large dataset and use only a few recordings to quickly adapt the model to a new person. CL is supposed to further improve MAML by meta-learning from easy to difficult tasks. For the fine-tuning stage, we propose improved pre-fine-tuning to solve individual-level diversity. We conduct experiments using a combination of three publicly available ECG datasets. The results show that our method outperforms the compared methods in terms of all evaluation metrics. Ablation studies show that MAML and CL could help perform more evenly, and pre-fine-tuning could better fit the model to training data.
Human-Centered Concept Explanations for Neural Networks
Authors: Chih-Kuan Yeh, Been Kim, Pradeep Ravikumar
Date: 2022-02-25 00:00:00

Understanding complex machine learning models such as deep neural networks with explanations is crucial in various applications. Many explanations stem from the model perspective, and may not necessarily effectively communicate why the model is making its predictions at the right level of abstraction. For example, providing importance weights to individual pixels in an image can only express which parts of that particular image are important to the model, but humans may prefer an explanation which explains the prediction by concept-based thinking. In this work, we review the emerging area of concept based explanations. We start by introducing concept explanations including the class of Concept Activation Vectors (CAV) which characterize concepts using vectors in appropriate spaces of neural activations, and discuss different properties of useful concepts, and approaches to measure the usefulness of concept vectors. We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.
Prediction of Depression Severity Based on the Prosodic and Semantic Features with Bidirectional LSTM and Time Distributed CNN
Authors: Kaining Mao, Wei zhang, Deborah Baofeng Wang, Ang Li, Rongqi Jiao, Yanhui Zhu, Bin Wu, Tiansheng Zheng, Lei Qian, Wei Lyu, Minjie Ye, Jie Chen
Date: 2022-02-25 00:00:00
Tasks: Depression Detection, Word Embeddings

Depression is increasingly impacting individuals both physically and psychologically worldwide. It has become a global major public health problem and attracts attention from various research fields. Traditionally, the diagnosis of depression is formulated through semi-structured interviews and supplementary questionnaires, which makes the diagnosis heavily relying on physicians experience and is subject to bias. Mental health monitoring and cloud-based remote diagnosis can be implemented through an automated depression diagnosis system. In this article, we propose an attention-based multimodality speech and text representation for depression prediction. Our model is trained to estimate the depression severity of participants using the Distress Analysis Interview Corpus-Wizard of Oz (DAIC-WOZ) dataset. For the audio modality, we use the collaborative voice analysis repository (COVAREP) features provided by the dataset and employ a Bidirectional Long Short-Term Memory Network (Bi-LSTM) followed by a Time-distributed Convolutional Neural Network (T-CNN). For the text modality, we use global vectors for word representation (GloVe) to perform word embeddings and the embeddings are fed into the Bi-LSTM network. Results show that both audio and text models perform well on the depression severity estimation task, with best sequence level F1 score of 0.9870 and patient-level F1 score of 0.9074 for the audio model over five classes (healthy, mild, moderate, moderately severe, and severe), as well as sequence level F1 score of 0.9709 and patient-level F1 score of 0.9245 for the text model over five classes. Results are similar for the multimodality fused model, with the highest F1 score of 0.9580 on the patient-level depression detection task over five classes. Experiments show statistically significant improvements over previous works.
A residual-based message passing algorithm for constraint satisfaction problems
Authors: Chun-Yan Zhao, Yan-Rong Fu, Jin-Hua Zhao
Date: 2022-02-25 00:00:00

Message passing algorithms, whose iterative nature captures well complicated interactions among interconnected variables in complex systems and extracts information from the fixed point of iterated messages, provide a powerful toolkit in tackling hard computational tasks in optimization, inference, and learning problems. In the context of constraint satisfaction problems (CSPs), when a control parameter (such as constraint density) is tuned, multiple threshold phenomena emerge, signaling fundamental structural transitions in their solution space. Finding solutions around these transition points is exceedingly challenging for algorithm design, where message passing algorithms suffer from a large message fluctuation far from convergence. Here we introduce a residual-based updating step into message passing algorithms, in which messages varying large between consecutive steps are given high priority in the updating process. For the specific example of model RB, a typical prototype of random CSPs with growing domains, we show that our algorithm improves the convergence of message updating and increases the success probability in finding solutions around the satisfiability threshold with a low computational cost. Our approach to message passing algorithms should be of value for exploring their power in developing algorithms to find ground-state solutions and understand the detailed structure of solution space of hard optimization problems.
Bidding Agent Design in the LinkedIn Ad Marketplace
Authors: Yuan Gao, Kaiyu Yang, Yuanlong Chen, Min Liu, Noureddine El Karoui
Date: 2022-02-25 00:00:00

We establish a general optimization framework for the design of automated bidding agent in dynamic online marketplaces. It optimizes solely for the buyer's interest and is agnostic to the auction mechanism imposed by the seller. As a result, the framework allows, for instance, the joint optimization of a group of ads across multiple platforms each running its own auction format. Bidding strategy derived from this framework automatically guarantees the optimality of budget allocation across ad units and platforms. Common constraints such as budget delivery schedule, return on investments and guaranteed results, directly translates to additional parameters in the bidding formula. We share practical learnings of the deployed bidding system in the LinkedIn ad marketplace based on this framework.
Improved Dual Correlation Reduction Network
Authors: Yue Liu, Sihang Zhou, Xinwang Liu, Wenxuan Tu, Xihong Yang
Date: 2022-02-25 00:00:00
Tasks: Clustering, Feature Correlation, Graph Clustering

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different clusters without human annotations, is a fundamental yet challenging task. However, we observed that the existing methods suffer from the representation collapse problem and easily tend to encode samples with different classes into the same latent embedding. Consequently, the discriminative capability of nodes is limited, resulting in sub-optimal clustering performance. To address this problem, we propose a novel deep graph clustering algorithm termed Improved Dual Correlation Reduction Network (IDCRN) through improving the discriminative capability of samples. Specifically, by approximating the cross-view feature correlation matrix to an identity matrix, we reduce the redundancy between different dimensions of features, thus improving the discriminative capability of the latent space explicitly. Meanwhile, the cross-view sample correlation matrix is forced to approximate the designed clustering-refined adjacency matrix to guide the learned latent representation to recover the affinity matrix even across views, thus enhancing the discriminative capability of features implicitly. Moreover, we avoid the collapsed representation caused by the over-smoothing issue in Graph Convolutional Networks (GCNs) through an introduced propagation regularization term, enabling IDCRN to capture the long-range information with the shallow network structure. Extensive experimental results on six benchmarks have demonstrated the effectiveness and the efficiency of IDCRN compared to the existing state-of-the-art deep graph clustering algorithms.
Oscillatory Neural Network as Hetero-Associative Memory for Image Edge Detection
Authors: Madeleine Abernot, Thierry Gil, Aida Todri-Sanial
Date: 2022-02-25 00:00:00
Tasks: Edge Detection

The increasing amount of data to be processed on edge devices, such as cameras, has motivated Artificial Intelligence (AI) integration at the edge. Typical image processing methods performed at the edge, such as feature extraction or edge detection, use convolutional filters that are energy, computation, and memory hungry algorithms. But edge devices and cameras have scarce computational resources, bandwidth, and power and are limited due to privacy constraints to send data over to the cloud. Thus, there is a need to process image data at the edge. Over the years, this need has incited a lot of interest in implementing neuromorphic computing at the edge. Neuromorphic systems aim to emulate the biological neural functions to achieve energy-efficient computing. Recently, Oscillatory Neural Networks (ONN) present a novel brain-inspired computing approach by emulating brain oscillations to perform autoassociative memory types of applications. To speed up image edge detection and reduce its power consumption, we perform an in-depth investigation with ONNs. We propose a novel image processing method by using ONNs as a hetero-associative memory (HAM) for image edge detection. We simulate our ONN-HAM solution using first, a Matlab emulator, and then a fully digital ONN design. We show results on gray scale square evaluation maps, also on black and white and gray scale 28x28 MNIST images and finally on black and white 512x512 standard test images. We compare our solution with standard edge detection filters such as Sobel and Canny. Finally, using the fully digital design simulation results, we report on timing and resource characteristics, and evaluate its feasibility for real-time image processing applications. Our digital ONN-HAM solution can process images with up to 120x120 pixels (166 MHz system frequency) respecting real-time camera constraints. This work is the first to explore ONNs as hetero-associative memory for image processing applications.
6D Rotation Representation For Unconstrained Head Pose Estimation
Authors: Thorsten Hempel, Ahmed A. Abdelrahman, Ayoub Al-Hamadi
Date: 2022-02-25 00:00:00
Tasks: Head Pose Estimation, Pose Estimation, Pose Prediction

In this paper, we present a method for unconstrained end-to-end head pose estimation. We address the problem of ambiguous rotation labels by introducing the rotation matrix formalism for our ground truth data and propose a continuous 6D rotation matrix representation for efficient and robust direct regression. This way, our method can learn the full rotation appearance which is contrary to previous approaches that restrict the pose prediction to a narrow-angle for satisfactory results. In addition, we propose a geodesic distance-based loss to penalize our network with respect to the SO(3) manifold geometry. Experiments on the public AFLW2000 and BIWI datasets demonstrate that our proposed method significantly outperforms other state-of-the-art methods by up to 20\%. We open-source our training and testing code along with our pre-trained models: https://github.com/thohemp/6DRepNet.
Bridging the Gap Between Patient-specific and Patient-independent Seizure Prediction via Knowledge Distillation
Authors: Di wu, Jie Yang, Mohamad Sawan
Date: 2022-02-25 00:00:00
Tasks: Knowledge Distillation, Prediction, Seizure prediction

Objective. Deep neural networks (DNNs) have shown unprecedented success in various brain-machine interface applications such as epileptic seizure prediction. However, existing approaches typically train models in a patient-specific fashion due to the highly personalized characteristics of epileptic signals. Therefore, only a limited number of labeled recordings from each subject can be used for training. As a consequence, current DNN based methods demonstrate poor generalization ability to some extent due to the insufficiency of training data. On the other hand, patient-independent models attempt to utilize more patient data to train a universal model for all patients by pooling patient data together. Despite different techniques applied, results show that patient-independent models perform worse than patient-specific models due to high individual variation across patients. A substantial gap thus exists between patient-specific and patient-independent models. Approach. In this paper, we propose a novel training scheme based on knowledge distillation which makes use of a large amount of data from multiple subjects. It first distills informative features from signals of all available subjects with a pre-trained general model. A patient-specific model can then be obtained with the help of distilled knowledge and additional personalized data. Main results. Four state-of-the-art seizure prediction methods are trained on the Children's Hospital of Boston-MIT sEEG database with our proposed scheme. The resulting accuracy, sensitivity, and false prediction rate show that our proposed training scheme consistently improves the prediction performance of state-of-the-art methods by a large margin. Significance. The proposed training scheme significantly improves the performance of patient-specific seizure predictors and bridges the gap between patient-specific and patient-independent predictors.
Bridging the Urban-Rural Connectivity Gap through Intelligent Space, Air, and Ground Networks
Authors: Fares Fourati, Saeed Hamood Alsamhi, Mohamed-Slim Alouini
Date: 2022-02-25 00:00:00

Connectivity in rural areas is one of the main challenges of communication networks. To overcome this challenge, a variety of solutions for different situations are required. Optimizing the current networking paradigms is therefore mandatory. The high costs of infrastructure and the low revenue of cell sites in rural areas compared with urban areas are especially unattractive for telecommunication operators. Therefore, space, air, and ground networks should all be optimized for achieving connectivity in rural areas. We highlight the latest works on rural connectivity, discuss the solutions for terrestrial networks, and study the potential benefits of nonterrestrial networks. Furthermore, we present an overview of artificial intelligence (AI) techniques for improving space, air, and ground networks, hence improving connectivity in rural areas. AI enables intelligent communications and can integrate space, air, and ground networks for rural connectivity. We discuss the rural connectivity challenges and highlight the latest projects and research and the empowerment of networks using AI. Finally, we discuss the potential positive impacts of providing connectivity to rural communities.
A deep learning approach for direction of arrival estimation using automotive-grade ultrasonic sensors
Authors: Mohamed Shawki Elamir, Heinrich Gotzig, Raoul Zoellner, Patrick Maeder
Date: 2022-02-25 00:00:00
Tasks: Direction of Arrival Estimation

In this paper, a deep learning approach is presented for direction of arrival estimation using automotive-grade ultrasonic sensors which are used for driving assistance systems such as automatic parking. A study and implementation of the state of the art deterministic direction of arrival estimation algorithms is used as a benchmark for the performance of the proposed approach. Analysis of the performance of the proposed algorithms against the existing algorithms is carried out over simulation data as well as data from a measurement campaign done using automotive-grade ultrasonic sensors. Both sets of results clearly show the superiority of the proposed approach under realistic conditions such as noise from the environment as well as eventual errors in measurements. It is demonstrated as well how the proposed approach can overcome some of the known limitations of the existing algorithms such as precision dilution of triangulation and aliasing.
Novel techniques for improving NNetEn entropy calculation for short and noisy time series
Authors: Hanif Heidari, Andrei Velichko, Murugappan Murugappan, Muhammad E. H. Chowdhury
Date: 2022-02-25 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Emotion Recognition, Time Series, Time Series Analysis

Entropy is a fundamental concept in the field of information theory. During measurement, conventional entropy measures are susceptible to length and amplitude changes in time series. A new entropy metric, neural network entropy (NNetEn), has been developed to overcome these limitations. NNetEn entropy is computed using a modified LogNNet neural network classification model. The algorithm contains a reservoir matrix of N=19625 elements that must be filled with the given data. The contribution of this paper is threefold. Firstly, this work investigates different methods of filling the reservoir with time series (signal) elements. The reservoir filling method determines the accuracy of the entropy estimation by convolution of the study time series and LogNNet test data. The present study proposes 6 methods for filling the reservoir for time series. Two of them (Method 3 and Method 6) employ the novel approach of stretching the time series to create intermediate elements that complement it, but do not change its dynamics. The most reliable methods for short time series are Method 3 and Method 5. The second part of the study examines the influence of noise and constant bias on entropy values. Our study examines three different time series data types (chaotic, periodic, and binary) with different dynamic properties, Signal to Noise Ratio (SNR), and offsets. The NNetEn entropy calculation errors are less than 10% when SNR is greater than 30 dB, and entropy decreases with an increase in the bias component. The third part of the article analyzes real-time biosignal EEG data collected from emotion recognition experiments. The NNetEn measures show robustness under low-amplitude noise using various filters. Thus, NNetEn measures entropy effectively when applied to real-world environments with ambient noise, white noise, and 1/f noise.