OpenRaiser/Intern-Atlas huggingface dataset:
UNIT: Unifying Tensorized Instruction Compilation
Venue: arXiv cs.PL 2021
Year: 2021

Because of the increasing demand for computation in DNN, researchers develope both hardware and software mechanisms to reduce the compute and memory burden. A widely adopted approach is to use mixed precision data types. However, it is hard to leverage mixed precision without hardware support because of the overhead of data casting. Hardware vendors offer tensorized instructions for mixed-precision tensor operations, like Intel VNNI, Tensor Core, and ARM-DOT. These instructions involve a computing idiom that reduces multiple low precision elements into one high precision element. The lack of compilation techniques for this makes it hard to utilize these instructions: Using vendor-provided libraries for computationally-intensive kernels is inflexible and prevents further optimizations, and manually writing hardware intrinsics is error-prone and difficult for programmers. Some prior works address this problem by creating compilers for each instruction. This requires excessive effort when it comes to many tensorized instructions. In this work, we develop a compiler framework to unify the compilation for these instructions -- a unified semantics abstraction eases the integration of new instructions, and reuses the analysis and transformations. Tensorized instructions from different platforms can be compiled via UNIT with moderate effort for favorable performance. Given a tensorized instruction and a tensor operation, UNIT automatically detects the applicability, transforms the loop organization of the operation,and rewrites the loop body to leverage the tensorized instruction. According to our evaluation, UNIT can target various mainstream hardware platforms. The generated end-to-end inference model achieves 1.3x speedup over Intel oneDNN on an x86 CPU, 1.75x speedup over Nvidia cuDNN on an NvidiaGPU, and 1.13x speedup over a carefully tuned TVM solution for ARM DOT on an ARM CPU.
Fidelity and Privacy of Synthetic Medical Data
Venue: arXiv cs.LG 2021
Year: 2021

The digitization of medical records ushered in a new era of big data to clinical science, and with it the possibility that data could be shared, to multiply insights beyond what investigators could abstract from paper records. The need to share individual-level medical data to accelerate innovation in precision medicine continues to grow, and has never been more urgent, as scientists grapple with the COVID-19 pandemic. However, enthusiasm for the use of big data has been tempered by a fully appropriate concern for patient autonomy and privacy. That is, the ability to extract private or confidential information about an individual, in practice, renders it difficult to share data, since significant infrastructure and data governance must be established before data can be shared. Although HIPAA provided de-identification as an approved mechanism for data sharing, linkage attacks were identified as a major vulnerability. A variety of mechanisms have been established to avoid leaking private information, such as field suppression or abstraction, strictly limiting the amount of information that can be shared, or employing mathematical techniques such as differential privacy. Another approach, which we focus on here, is creating synthetic data that mimics the underlying data. For synthetic data to be a useful mechanism in support of medical innovation and a proxy for real-world evidence, one must demonstrate two properties of the synthetic dataset: (1) any analysis on the real data must be matched by analysis of the synthetic data (statistical fidelity) and (2) the synthetic data must preserve privacy, with minimal risk of re-identification (privacy guarantee). In this paper we propose a framework for quantifying the statistical fidelity and privacy preservation properties of synthetic datasets and demonstrate these metrics for synthetic data generated by Syntegra technology.
Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors
Venue: arXiv cs.AR 2021
Year: 2021

Focal-plane Sensor-processors (FPSPs) are a camera technology that enable low power, high frame rate computation, making them suitable for edge computation. Unfortunately, these devices' limited instruction sets and registers make developing complex algorithms difficult. In this work, we present Cain - a compiler that targets SCAMP-5, a general-purpose FPSP - which generates code from multiple convolutional kernels. As an example, given the convolutional kernels for an MNIST digit recognition neural network, Cain produces code that is half as long, when compared to the other available compilers for SCAMP-5.
Maddah-Ali-Niesen Scheme for Multi-access Coded Caching
Venue: arXiv cs.IT 2021
Year: 2021

The well known Maddah-Ali-Niesen (MAN) coded caching scheme for users with dedicated cache is extended for use in multi-access coded cache scheme where the number of users need not be same as the number of caches in the system. The well known MAN scheme is recoverable as a special case of the multi-access system considered. The performance of this scheme is compared with the existing works on multi-access coded caching. To be able to compare the performance of different multi-access schemes with different number of users for the same number of caches, the terminology of per user rate (rate divided by the number of users) introduced in \cite{KNS} is used.
An Efficient Communication Protocol for FPGA IP Protection
Venue: arXiv cs.CR 2021
Year: 2021

We introduce a protection-based IP security scheme to protect soft and firm IP cores which are used on FPGA devices. The scheme is based on Finite State Machin (FSM) obfuscation and exploits Physical Unclonable Function (PUF) for FPGA unique identification (ID) generation which help pay-per-device licensing. We introduce a communication protocol to protect the rights of parties in this market. On standard benchmark circuits, the experimental results show that our scheme is secure, attack-resilient and can be implemented with low area, power and delay overheads.
5G New Radio for Automotive, Rail, and Air Transport
Venue: arXiv cs.NI 2021
Year: 2021

The recent and upcoming releases of the 3rd Generation Partnership Project's 5G New Radio specifications include features that are motivated by providing connectivity services to a broad set of verticals, including the automotive, rail, and air transport industries. Currently, several radio access network features are being further enhanced or newly introduced in NR to improve 5G's capability to provide fast, reliable, and non-limiting connectivity for transport applications. In this article, we review the most important characteristics and requirements of a wide range of services that are driven by the desire to help the transport sector to become more sustainable, economically viable, safe, and secure. These requirements will be supported by the evolving and entirely new features of 5G NR systems, including accurate positioning, reference signal design to enable multi-transmission and reception points, service-specific scheduling configuration, and service quality prediction.
(α, β)-Modules in Graphs
Venue: arXiv cs.DM 2021
Year: 2021

Modular Decomposition focuses on repeatedly identifying a module M (a collection of vertices that shares exactly the same neighbourhood outside of M) and collapsing it into a single vertex. This notion of exactitude of neighbourhood is very strict, especially when dealing with real world graphs. We study new ways to relax this exactitude condition. However, generalizing modular decomposition is far from obvious. Most of the previous proposals lose algebraic properties of modules and thus most of the nice algorithmic consequences. We introduce the notion of an (α, β)-module, a relaxation that allows a bounded number of errors in each node and maintains some of the algebraic structure. It leads to a new combinatorial decomposition with interesting properties. Among the main results in this work, we show that minimal (α, β)-modules can be computed in polynomial time, and that every graph admits an (α,β)-modular decomposition tree, thus generalizing Gallai's Theorem (which corresponds to the case for α = β = 0). Unfortunately we give evidence that computing such a decomposition tree can be difficult.
User-Aware Power Management for Mobile Devices
Venue: arXiv cs.AR 2021
Year: 2021

The power management techniques to extend battery lifespan is becoming increasingly important due to longer user applications' running time in mobile devices. Even when users do not use any applications, battery lifespan decreases continually. It occurs because of service daemons of mobile platform and network-based data synchronization operations. In this paper, we propose a new power management system that recognizes the idle time of the device to reduce the battery consumption of mobile devices.
A co-Design approach to develop a smart cooking appliance. Applying a Domain Specific Language for a community supported appliance
Venue: arXiv cs.HC 2021
Year: 2021

Our environment, whether at work, in public spaces, or at home, is becoming more connected, and increasingly responsive. Meal preparation even when it involves simply heating ready-made food can be perceived as a complex process for people with disabilities. This research aimed to prototype, using a co-Design approach a Community Supported Appliance (CSA) by developing a Domain Specific Language (DSL), precisely created for a semi-automated cooking process. The DSL was shaped and expressed in the idiom of the users and allowed the CSA to support independence for users while performing daily cooking activities.
Distilling Large Language Models into Tiny and Effective Students using pQRNN
Venue: arXiv cs.CL 2021
Year: 2021

Large pre-trained multilingual models like mBERT, XLM-R achieve state of the art results on language understanding tasks. However, they are not well suited for latency critical applications on both servers and edge devices. It's important to reduce the memory and compute resources required by these models. To this end, we propose pQRNN, a projection-based embedding-free neural encoder that is tiny and effective for natural language processing tasks. Without pre-training, pQRNNs significantly outperform LSTM models with pre-trained embeddings despite being 140x smaller. With the same number of parameters, they outperform transformer baselines thereby showcasing their parameter efficiency. Additionally, we show that pQRNNs are effective student architectures for distilling large pre-trained language models. We perform careful ablations which study the effect of pQRNN parameters, data augmentation, and distillation settings. On MTOP, a challenging multilingual semantic parsing dataset, pQRNN students achieve 95.9\% of the performance of an mBERT teacher while being 350x smaller. On mATIS, a popular parsing task, pQRNN students on average are able to get to 97.1\% of the teacher while again being 350x smaller. Our strong results suggest that our approach is great for latency-sensitive applications while being able to leverage large mBERT-like models.
Generative Replay-based Continual Zero-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

Zero-shot learning is a new paradigm to classify objects from classes that are not available at training time. Zero-shot learning (ZSL) methods have attracted considerable attention in recent years because of their ability to classify unseen/novel class examples. Most of the existing approaches on ZSL works when all the samples from seen classes are available to train the model, which does not suit real life. In this paper, we tackle this hindrance by developing a generative replay-based continual ZSL (GRCZSL). The proposed method endows traditional ZSL to learn from streaming data and acquire new knowledge without forgetting the previous tasks' gained experience. We handle catastrophic forgetting in GRCZSL by replaying the synthetic samples of seen classes, which have appeared in the earlier tasks. These synthetic samples are synthesized using the trained conditional variational autoencoder (VAE) over the immediate past task. Moreover, we only require the current and immediate previous VAE at any time for training and testing. The proposed GRZSL method is developed for a single-head setting of continual learning, simulating a real-world problem setting. In this setting, task identity is given during training but unavailable during testing. GRCZSL performance is evaluated on five benchmark datasets for the generalized setup of ZSL with fixed and dynamic (incremental class) settings of continual learning. The existing class setting presented recently in the literature is not suitable for a class-incremental setting. Therefore, this paper proposes a new setting to address this issue. Experimental results show that the proposed method significantly outperforms the baseline and the state-of-the-art method and makes it more suitable for real-world applications.
Hessian-Aware Pruning and Optimal Neural Implant
Venue: arXiv cs.CV 2021
Year: 2021

Pruning is an effective method to reduce the memory footprint and FLOPs associated with neural network models. However, existing structured-pruning methods often result in significant accuracy degradation for moderate pruning levels. To address this problem, we introduce a new Hessian Aware Pruning (HAP) method coupled with a Neural Implant approach that uses second-order sensitivity as a metric for structured pruning. The basic idea is to prune insensitive components and to use a Neural Implant for moderately sensitive components, instead of completely pruning them. For the latter approach, the moderately sensitive components are replaced with with a low rank implant that is smaller and less computationally expensive than the original component. We use the relative Hessian trace to measure sensitivity, as opposed to the magnitude based sensitivity metric commonly used in the literature. We test HAP for both computer vision tasks and natural language tasks, and we achieve new state-of-the-art results. Specifically, HAP achieves less than $0.1\%$/$0.5\%$ degradation on PreResNet29/ResNet50 (CIFAR-10/ImageNet) with more than 70\%/50\% of parameters pruned. Meanwhile, HAP also achieves significantly better performance (up to 0.8\% with 60\% of parameters pruned) as compared to gradient based method for head pruning on transformer-based models. The framework has been open sourced and available online.
Human Interaction Recognition Framework based on Interacting Body Part Attention
Venue: arXiv cs.CV 2021
Year: 2021

Human activity recognition in videos has been widely studied and has recently gained significant advances with deep learning approaches; however, it remains a challenging task. In this paper, we propose a novel framework that simultaneously considers both implicit and explicit representations of human interactions by fusing information of local image where the interaction actively occurred, primitive motion with the posture of individual subject's body parts, and the co-occurrence of overall appearance change. Human interactions change, depending on how the body parts of each human interact with the other. The proposed method captures the subtle difference between different interactions using interacting body part attention. Semantically important body parts that interact with other objects are given more weight during feature representation. The combined feature of interacting body part attention-based individual representation and the co-occurrence descriptor of the full-body appearance change is fed into long short-term memory to model the temporal dynamics over time in a single framework. We validate the effectiveness of the proposed method using four widely used public datasets by outperforming the competing state-of-the-art method.
A novel DL approach to PE malware detection: exploring Glove vectorization, MCC_RCNN and feature fusion
Venue: arXiv cs.CR 2021
Year: 2021

In recent years, malware becomes more threatening. Concerning the increasing malware variants, there comes Machine Learning (ML)-based and Deep Learning (DL)-based approaches for heuristic detection. Nevertheless, the prediction accuracy of both needs to be improved. In response to the above issues in the PE malware domain, we propose the DL-based approaches for detection and use static-based features fed up into models. The contributions are as follows: we recapitulate existing malware detection methods. That is, we propose a vec-torized representation model of the malware instruction layer and semantic layer based on Glove. We implement a neural network model called MCC_RCNN (Malware Detection and Recurrent Convolutional Neural Network), comprising of the combination with CNN and RNN. Moreover, we provide a description of feature fusion in static behavior levels. With the numerical results generated from several comparative experiments towards evaluating the Glove-based vectoriza-tion, MCC_RCNN-based classification methodology and feature fusion stages, our proposed classification methods can obtain a higher prediction accuracy than the other baseline methods.
Visual Question Answering based on Local-Scene-Aware Referring Expression Generation
Venue: arXiv cs.CV 2021
Year: 2021

Visual question answering requires a deep understanding of both images and natural language. However, most methods mainly focus on visual concept; such as the relationships between various objects. The limited use of object categories combined with their relationships or simple question embedding is insufficient for representing complex scenes and explaining decisions. To address this limitation, we propose the use of text expressions generated for images, because such expressions have few structural constraints and can provide richer descriptions of images. The generated expressions can be incorporated with visual features and question embedding to obtain the question-relevant answer. A joint-embedding multi-head attention network is also proposed to model three different information modalities with co-attention. We quantitatively and qualitatively evaluated the proposed method on the VQA v2 dataset and compared it with state-of-the-art methods in terms of answer prediction. The quality of the generated expressions was also evaluated on the RefCOCO, RefCOCO+, and RefCOCOg datasets. Experimental results demonstrate the effectiveness of the proposed method and reveal that it outperformed all of the competing methods in terms of both quantitative and qualitative results.
Nonstationary Stochastic Multiarmed Bandits: UCB Policies and Minimax Regret
Venue: arXiv cs.LG 2021
Year: 2021

We study the nonstationary stochastic Multi-Armed Bandit (MAB) problem in which the distribution of rewards associated with each arm are assumed to be time-varying and the total variation in the expected rewards is subject to a variation budget. The regret of a policy is defined by the difference in the expected cumulative rewards obtained using the policy and using an oracle that selects the arm with the maximum mean reward at each time. We characterize the performance of the proposed policies in terms of the worst-case regret, which is the supremum of the regret over the set of reward distribution sequences satisfying the variation budget. We extend Upper-Confidence Bound (UCB)-based policies with three different approaches, namely, periodic resetting, sliding observation window and discount factor and show that they are order-optimal with respect to the minimax regret, i.e., the minimum worst-case regret achieved by any policy. We also relax the sub-Gaussian assumption on reward distributions and develop robust versions the proposed polices that can handle heavy-tailed reward distributions and maintain their performance guarantees.
Twisted-Pair Superposition Transmission
Venue: arXiv cs.IT 2021
Year: 2021

We propose in this paper a new coding scheme called twisted-pair superposition transmission (TPST). The encoding is to "mix together" a pair of basic codes by superposition, while the decoding can be implemented as a successive cancellation list decoding algorithm. The most significant features of the TPST code are its predictable performance that can be estimated numerically from the basic codes and its flexible construction in the sense that it can be easily adapted to different coding rates. To construct good TPST codes in the finite length regime, we propose two design approaches-rate allocation and partial superposition. By taking tail-biting convolutional codes (TBCC) as basic codes, we show by numerical results that the TPST codes can have near-capacity performance in the short length regime.
Does a Hybrid Neural Network based Feature Selection Model Improve Text Classification?
Venue: arXiv cs.CL 2021
Year: 2021

Text classification is a fundamental problem in the field of natural language processing. Text classification mainly focuses on giving more importance to all the relevant features that help classify the textual data. Apart from these, the text can have redundant or highly correlated features. These features increase the complexity of the classification algorithm. Thus, many dimensionality reduction methods were proposed with the traditional machine learning classifiers. The use of dimensionality reduction methods with machine learning classifiers has achieved good results. In this paper, we propose a hybrid feature selection method for obtaining relevant features by combining various filter-based feature selection methods and fastText classifier. We then present three ways of implementing a feature selection and neural network pipeline. We observed a reduction in training time when feature selection methods are used along with neural networks. We also observed a slight increase in accuracy on some datasets.
Unsupervised Technical Domain Terms Extraction using Term Extractor
Venue: arXiv cs.CL 2021
Year: 2021

Terminology extraction, also known as term extraction, is a subtask of information extraction. The goal of terminology extraction is to extract relevant words or phrases from a given corpus automatically. This paper focuses on the unsupervised automated domain term extraction method that considers chunking, preprocessing, and ranking domain-specific terms using relevance and cohesion functions for ICON 2020 shared task 2: TermTraction.
Chemistry42: An AI-based platform for de novo molecular design
Venue: arXiv cs.AI 2021
Year: 2021

Chemistry42 is a software platform for de novo small molecule design that integrates Artificial Intelligence (AI) techniques with computational and medicinal chemistry methods. Chemistry42 is unique in its ability to generate novel molecular structures with predefined properties validated through in vitro and in vivo studies. Chemistry42 is a core component of Insilico Medicine Pharma.ai drug discovery suite that also includes target discovery and multi-omics data analysis (PandaOmics) and clinical trial outcomes predictions (InClinico).