OpenRaiser/Intern-Atlas huggingface dataset:
Assessing Algorithmic Biases for Musical Version Identification
Venue: arXiv cs.SD 2021
Year: 2021

Version identification (VI) systems now offer accurate and scalable solutions for detecting different renditions of a musical composition, allowing the use of these systems in industrial applications and throughout the wider music ecosystem. Such use can have an important impact on various stakeholders regarding recognition and financial benefits, including how royalties are circulated for digital rights management. In this work, we take a step toward acknowledging this impact and consider VI systems as socio-technical systems rather than isolated technologies. We propose a framework for quantifying performance disparities across 5 systems and 6 relevant side attributes: gender, popularity, country, language, year, and prevalence. We also consider 3 main stakeholders for this particular information retrieval use case: the performing artists of query tracks, those of reference (original) tracks, and the composers. By categorizing the recordings in our dataset using such attributes and stakeholders, we analyze whether the considered VI systems show any implicit biases. We find signs of disparities in identification performance for most of the groups we include in our analyses. Moreover, we also find that learning- and rule-based systems behave differently for some attributes, which suggests an additional dimension to consider along with accuracy and scalability when evaluating VI systems. Lastly, we share our dataset with attribute annotations to encourage VI researchers to take these aspects into account while building new systems.
inbaverSim: An OMNeT++ Model Framework for Content Centric Networking
Venue: arXiv cs.NI 2021
Year: 2021

Today's networks are used primarily to move content. To cater to this requirement Information Centric Networks (ICN) were introduced. One of the main architectures of ICN is Content Centric Networking (CCN) and its derivative, Named Data Networking (NDN). CCN is standardized at the Internet Engineering Task Force (IETF) and is envisaged to replace the current Internet over time. To evaluate large scale deployments of CCN, a model framework called the inbaverSim is developed in OMNeT++. This work presents the architecture of this model framework together with an example evaluation using the model framework. The code is open source and is available at GitHub.
Semi-tensor Product-based TensorDecomposition for Neural Network Compression
Venue: arXiv cs.LG 2021
Year: 2021

The existing tensor networks adopt conventional matrix product for connection. The classical matrix product requires strict dimensionality consistency between factors, which can result in redundancy in data representation. In this paper, the semi-tensor product is used to generalize classical matrix product-based mode product to semi-tensor mode product. As it permits the connection of two factors with different dimensionality, more flexible and compact tensor decompositions can be obtained with smaller sizes of factors. Tucker decomposition, Tensor Train (TT) and Tensor Ring (TR) are common decomposition for low rank compression of deep neural networks. The semi-tensor product is applied to these tensor decompositions to obtained their generalized versions, i.e., semi-tensor Tucker decomposition (STTu), semi-tensor train(STT) and semi-tensor ring (STR). Experimental results show the STTu, STT and STR achieve higher compression factors than the conventional tensor decompositions with the same accuracy but less training times in ResNet and WideResNetcompression. With 2% accuracy degradation, the TT-RN (rank = 14) and the TR-WRN (rank = 16) only obtain 3 times and99t times compression factors while the STT-RN (rank = 14) and the STR-WRN (rank = 16) achieve 9 times and 179 times compression factors, respectively.
Semi-Supervised Text Classification via Self-Pretraining
Venue: arXiv cs.CL 2021
Year: 2021

We present a neural semi-supervised learning model termed Self-Pretraining. Our model is inspired by the classic self-training algorithm. However, as opposed to self-training, Self-Pretraining is threshold-free, it can potentially update its belief about previously labeled documents, and can cope with the semantic drift problem. Self-Pretraining is iterative and consists of two classifiers. In each iteration, one classifier draws a random set of unlabeled documents and labels them. This set is used to initialize the second classifier, to be further trained by the set of labeled documents. The algorithm proceeds to the next iteration and the classifiers' roles are reversed. To improve the flow of information across the iterations and also to cope with the semantic drift problem, Self-Pretraining employs an iterative distillation process, transfers hypotheses across the iterations, utilizes a two-stage training model, uses an efficient learning rate schedule, and employs a pseudo-label transformation heuristic. We have evaluated our model in three publicly available social media datasets. Our experiments show that Self-Pretraining outperforms the existing state-of-the-art semi-supervised classifiers across multiple settings. Our code is available at https://github.com/p-karisani/self_pretraining.
Variance of Twitter Embeddings and Temporal Trends of COVID-19 cases
Venue: arXiv cs.CL 2021
Year: 2021

The severity of the coronavirus pandemic necessitates the need of effective administrative decisions. Over 4 lakh people in India succumbed to COVID-19, with over 3 crore confirmed cases, and still counting. The threat of a plausible third wave continues to haunt millions. In this ever changing dynamic of the virus, predictive modeling methods can serve as an integral tool. The pandemic has further triggered an unprecedented usage of social media. This paper aims to propose a method for harnessing social media, specifically Twitter, to predict the upcoming scenarios related to COVID-19 cases. In this study, we seek to understand how the surges in COVID-19 related tweets can indicate rise in the cases. This prospective analysis can be utilised to aid administrators about timely resource allocation to lessen the severity of the damage. Using word embeddings to capture the semantic meaning of tweets, we identify Significant Dimensions (SDs).Our methodology predicts the rise in cases with a lead time of 15 days and 30 days with R2 scores of 0.80 and 0.62 respectively. Finally, we explain the thematic utility of the SDs.
PIETS: Parallelised Irregularity Encoders for Forecasting with Heterogeneous Time-Series
Venue: arXiv cs.LG 2021
Year: 2021

Heterogeneity and irregularity of multi-source data sets present a significant challenge to time-series analysis. In the literature, the fusion of multi-source time-series has been achieved either by using ensemble learning models which ignore temporal patterns and correlation within features or by defining a fixed-size window to select specific parts of the data sets. On the other hand, many studies have shown major improvement to handle the irregularity of time-series, yet none of these studies has been applied to multi-source data. In this work, we design a novel architecture, PIETS, to model heterogeneous time-series. PIETS has the following characteristics: (1) irregularity encoders for multi-source samples that can leverage all available information and accelerate the convergence of the model; (2) parallelised neural networks to enable flexibility and avoid information overwhelming; and (3) attention mechanism that highlights different information and gives high importance to the most related data. Through extensive experiments on real-world data sets related to COVID-19, we show that the proposed architecture is able to effectively model heterogeneous temporal data and outperforms other state-of-the-art approaches in the prediction task.
Accelerating Inverse Rendering By Using a GPU and Reuse of Light Paths
Venue: arXiv cs.CV 2021
Year: 2021

Inverse rendering seeks to estimate scene characteristics from a set of data images. The dominant approach is based on differential rendering using Monte-Carlo. Algorithms as such usually rely on a forward model and use an iterative gradient method that requires sampling millions of light paths per iteration. This paper presents an efficient framework that speeds up existing inverse rendering algorithms. This is achieved by tailoring the iterative process of inverse rendering specifically to a GPU architecture. For this cause, we introduce two interleaved steps - Path Sorting and Path Recycling. Path Sorting allows the GPU to deal with light paths of the same size. Path Recycling allows the algorithm to use light paths from previous iterations to better utilize the information they encode. Together, these steps significantly speed up gradient optimization. In this paper, we give the theoretical background for Path Recycling. We demonstrate its efficiency for volumetric scattering tomography and reflectometry (surface reflections).
Shaping mmWave Wireless Channel via Multi-Beam Design using Reconfigurable Intelligent Surfaces
Venue: arXiv cs.NI 2021
Year: 2021

Millimeter-wave (mmWave) communications is considered as a key enabler towards the realization of next-generation wireless networks, due to the abundance of available spectrum at mmWave frequencies. However, mmWave suffers from high free-space path-loss and poor scattering resulting in mostly line-of-sight (LoS) channels which result in a lack of coverage. Reconfigurable intelligent surfaces (RIS), as a new paradigm, have the potential to fill the coverage holes by shaping the wireless channel. In this paper, we propose a novel approach for designing RIS with elements arranged in a uniform planar array (UPA) structure. In what we refer to as multi-beamforming, We propose and design RIS such that the reflected beam comprises multiple disjoint lobes. Moreover, the beams have optimized gain within the desired angular coverage with fairly sharp edges avoiding power leakage to other regions. We provide a closed-form low-complexity solution for the multi-beamforming design. We confirm our theoretical results by numerical analysis.
A Unified Discretization Approach to Compute-Forward: From Discrete to Continuous Inputs
Venue: arXiv cs.IT 2021
Year: 2021

Compute-forward is a coding technique that enables receiver(s) in a network to directly decode one or more linear combinations of the transmitted codewords. Initial efforts focused on Gaussian channels and derived achievable rate regions via nested lattice codes and single-user (lattice) decoding as well as sequential (lattice) decoding. Recently, these results have been generalized to discrete memoryless channels via nested linear codes and joint typicality coding, culminating in a simultaneous-decoding rate region for recovering one or more linear combinations from $K$ users. Using a discretization approach, this paper translates this result into a simultaneous-decoding rate region for a wide class of continuous memoryless channels, including the important special case of Gaussian channels. Additionally, this paper derives a single, unified expression for both discrete and continuous rate regions via an algebraic generalization of Rényi's information dimension.
Incremental Layer-wise Self-Supervised Learning for Efficient Speech Domain Adaptation On Device
Venue: arXiv cs.SD 2021
Year: 2021

Streaming end-to-end speech recognition models have been widely applied to mobile devices and show significant improvement in efficiency. These models are typically trained on the server using transcribed speech data. However, the server data distribution can be very different from the data distribution on user devices, which could affect the model performance. There are two main challenges for on device training, limited reliable labels and limited training memory. While self-supervised learning algorithms can mitigate the mismatch between domains using unlabeled data, they are not applicable on mobile devices directly because of the memory constraint. In this paper, we propose an incremental layer-wise self-supervised learning algorithm for efficient speech domain adaptation on mobile devices, in which only one layer is updated at a time. Extensive experimental results demonstrate that the proposed algorithm obtains a Word Error Rate (WER) on the target domain $24.2\%$ better than supervised baseline and costs $89.7\%$ less training memory than the end-to-end self-supervised learning algorithm.
Quantifying the Invisible Labor in Crowd Work
Venue: arXiv cs.HC 2021
Year: 2021

Crowdsourcing markets provide workers with a centralized place to find paid work. What may not be obvious at first glance is that, in addition to the work they do for pay, crowd workers also have to shoulder a variety of unpaid invisible labor in these markets, which ultimately reduces workers' hourly wages. Invisible labor includes finding good tasks, messaging requesters, or managing payments. However, we currently know little about how much time crowd workers actually spend on invisible labor or how much it costs them economically. To ensure a fair and equitable future for crowd work, we need to be certain that workers are being paid fairly for all of the work they do. In this paper, we conduct a field study to quantify the invisible labor in crowd work. We build a plugin to record the amount of time that 100 workers on Amazon Mechanical Turk dedicate to invisible labor while completing 40,903 tasks. If we ignore the time workers spent on invisible labor, workers' median hourly wage was $3.76. But, we estimated that crowd workers in our study spent 33% of their time daily on invisible labor, dropping their median hourly wage to $2.83. We found that the invisible labor differentially impacts workers depending on their skill level and workers' demographics. The invisible labor category that took the most time and that was also the most common revolved around workers having to manage their payments. The second most time-consuming invisible labor category involved hyper-vigilance, where workers vigilantly watched over requesters' profiles for newly posted work or vigilantly searched for labor. We hope that through our paper, the invisible labor in crowdsourcing becomes more visible, and our results help to reveal the larger implications of the continuing invisibility of labor in crowdsourcing.
Q-Net: A Quantitative Susceptibility Mapping-based Deep Neural Network for Differential Diagnosis of Brain Iron Deposition in Hemochromatosis
Venue: arXiv cs.LG 2021
Year: 2021

Brain iron deposition, in particular deep gray matter nuclei, increases with advancing age. Hereditary Hemochromatosis (HH) is the most common inherited disorder of systemic iron excess in Europeans and recent studies claimed high brain iron accumulation in patient with Hemochromatosis. In this study, we focus on Artificial Intelligence (AI)-based differential diagnosis of brain iron deposition in HH via Quantitative Susceptibility Mapping (QSM), which is an established Magnetic Resonance Imaging (MRI) technique to study the distribution of iron in the brain. Our main objective is investigating potentials of AI-driven frameworks to accurately and efficiently differentiate individuals with Hemochromatosis from those of the healthy control group. More specifically, we developed the Q-Net framework, which is a data-driven model that processes information on iron deposition in the brain obtained from multi-echo gradient echo imaging data and anatomical information on T1-Weighted images of the brain. We illustrate that the Q-Net framework can assist in differentiating between someone with HH and Healthy control (HC) of the same age, something that is not possible by just visualizing images. The study is performed based on a unique dataset that was collected from 52 subjects with HH and 47 HC. The Q-Net provides a differential diagnosis accuracy of 83.16% and 80.37% in the scan-level and image-level classification, respectively.
DNN-Opt: An RL Inspired Optimization for Analog Circuit Sizing using Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Analog circuit sizing takes a significant amount of manual effort in a typical design cycle. With rapidly developing technology and tight schedules, bringing automated solutions for sizing has attracted great attention. This paper presents DNN-Opt, a Reinforcement Learning (RL) inspired Deep Neural Network (DNN) based black-box optimization framework for analog circuit sizing. The key contributions of this paper are a novel sample-efficient two-stage deep learning optimization framework leveraging RL actor-critic algorithms, and a recipe to extend it on large industrial circuits using critical device identification. Our method shows 5--30x sample efficiency compared to other black-box optimization methods both on small building blocks and on large industrial circuits with better performance metrics. To the best of our knowledge, this is the first application of DNN-based circuit sizing on industrial scale circuits.
A Step Towards On-Path Security Function Outsourcing
Venue: arXiv cs.CR 2021
Year: 2021

Security function outsourcing has witnessed both research and deployment in the recent years. While most existing services take a straight-forward approach of cloud hosting, on-path transit networks (such as ISPs) are increasingly more interested in offering outsourced security services to end users. Recent proposals (such as SafeBricks and mbTLS) have made it possible to outsource sensitive security applications to untrusted, arbitrary networks, rendering on-path security function outsourcing more promising than ever. However, to provide on-path security function outsourcing, there is one crucial component that is still missing -- a practical end-to-end network protocol. Thus, the discovery and orchestration of multiple capable and willing transit networks for user-requested security functions have only been assumed in many studies without any practical solutions. In this work, we propose Opsec, an end-to-end security-outsourcing protocol that fills this gap and brings us closer to the vision of on-path security function outsourcing. Opsec automatically discovers one or more transit ISPs between a client and a server, and requests user-specified security functions efficiently. When designing Opsec, we prioritize the practicality and applicability of this new end-to-end protocol in the current Internet. Our proof-of-concept implementation of Opsec for web sessions shows that an end user can easily start a new web session with a few clicks of a browser plug-in, to specify a series of security functions of her choice. We show that it is possible to implement such a new end-to-end service model in the current Internet for the majority of the web services without any major changes to the standard protocols (e.g., TCP, TLS, HTTP) and the existing network infrastructure (e.g., ISP's routing primitives).
Discovering care pathways for multi-morbid patients using event graphs
Venue: arXiv cs.DB 2021
Year: 2021

Patients suffering from multiple diseases (multi-morbid patients) often have complex clinical pathways. They are diagnosed and treated by different specialties and undergo other clinical actions related to various diagnoses. Coordination of care for these patients is often challenging, and it would be of great benefit to get better insight into how the clinical pathways develop in reality. Discovering these pathways using traditional process mining techniques and standard event logs may be difficult because the patient is involved in several highly independent clinical processes. Our objective is to explore the potential of analyzing these pathways using an event log representation reflecting the independent clinical processes. Our main research question is: How can we identify valuable insights by using a multi-entity event data representation for clinical pathways of multi-morbid patients? Our method was built on the idea to represent multiple entities in event logs as event graphs. The MIMIC-III data-set was used to evaluate the feasibility of this approach. Several clinical entities were identified and then mapped into an event graph. Finally, multi-entity directly follows graphs were discovered by querying the event graph visualizing them. Our result shows that paths involving multiple entities include traditional process mining concepts not for one clinical process but all involved processes. In addition, the relationship between activities of different clinical processes, which was not recognizable in traditional models, is visible in the event graph representation.
Dynamic Modeling and Simulation of a Four-wheel Skid-Steer Mobile Robot using Linear Graphs
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents the application of the concepts and approaches of linear graph (LG) theory in the modeling and simulation of a 4-wheel skid-steer mobile robotic system. An LG representation of the system is proposed and the accompanying state-space model of the dynamics of a mobile robot system is evaluated using the associated LGtheory MATLAB toolbox, which was developed in our lab. A genetic algorithm (GA)-based parameter estimation method is employed to determine the system parameters, which leads to a very accurate simulation of the model. The developed model is then evaluated and validated by comparing the simulated LG model trajectory with the trajectory of a ROS Gazebo simulated robot and experimental data obtained from the physical robotic system. The obtained results demonstrate that the proposed LG model, combined with the GA parameter estimation process, produces a highly accurate method of modeling and simulating a mobile robotic system.
Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation
Venue: arXiv cs.CV 2021
Year: 2021

Knowledge distillation usually transfers the knowledge from a pre-trained cumbersome teacher network to a compact student network, which follows the classical teacher-teaching-student paradigm. Based on this paradigm, previous methods mostly focus on how to efficiently train a better student network for deployment. Different from the existing practices, in this paper, we propose a novel student-helping-teacher formula, Teacher Evolution via Self-Knowledge Distillation (TESKD), where the target teacher (for deployment) is learned with the help of multiple hierarchical students by sharing the structural backbone. The diverse feedback from multiple students allows the teacher to improve itself through the shared feature representations. The effectiveness of our proposed framework is demonstrated by extensive experiments with various network settings on two standard benchmarks including CIFAR-100 and ImageNet. Notably, when trained together with our proposed method, ResNet-18 achieves 79.15% and 71.14% accuracy on CIFAR-100 and ImageNet, outperforming the baseline results by 4.74% and 1.43%, respectively. The code is available at: https://github.com/zhengli427/TESKD.
Study of Signal Temporal Logic Robustness Metrics for Robotic Tasks Optimization
Venue: arXiv cs.RO 2021
Year: 2021

Signal Temporal Logic (STL) is an efficient technique for describing temporal constraints. It can play a significant role in robotic manipulation, for example, to optimize the robot performance according to task-dependent metrics. In this paper, we evaluate several STL robustness metrics of interest in robotic manipulation tasks and discuss a case study showing the advantages of using STL to define complex constraints. Such constraints can be understood as cost functions in task optimization. We show how STL-based cost functions can be optimized using a variety of off-the-shelf optimizers. We report initial results of this research direction on a simulated planar environment.