OpenRaiser/Intern-Atlas huggingface dataset:
Knowledge Transfer by Discriminative Pre-training for Academic Performance Prediction
Venue: arXiv cs.CY 2021
Year: 2021

The needs for precisely estimating a student's academic performance have been emphasized with an increasing amount of attention paid to Intelligent Tutoring System (ITS). However, since labels for academic performance, such as test scores, are collected from outside of ITS, obtaining the labels is costly, leading to label-scarcity problem which brings challenge in taking machine learning approaches for academic performance prediction. To this end, inspired by the recent advancement of pre-training method in natural language processing community, we propose DPA, a transfer learning framework with Discriminative Pre-training tasks for Academic performance prediction. DPA pre-trains two models, a generator and a discriminator, and fine-tunes the discriminator on academic performance prediction. In DPA's pre-training phase, a sequence of interactions where some tokens are masked is provided to the generator which is trained to reconstruct the original sequence. Then, the discriminator takes an interaction sequence where the masked tokens are replaced by the generator's outputs, and is trained to predict the originalities of all tokens in the sequence. Compared to the previous state-of-the-art generative pre-training method, DPA is more sample efficient, leading to fast convergence to lower academic performance prediction error. We conduct extensive experimental studies on a real-world dataset obtained from a multi-platform ITS application and show that DPA outperforms the previous state-of-the-art generative pre-training method with a reduction of 4.05% in mean absolute error and more robust to increased label-scarcity.
Fedlearn-Algo: A flexible open-source privacy-preserving machine learning platform
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we present Fedlearn-Algo, an open-source privacy preserving machine learning platform. We use this platform to demonstrate our research and development results on privacy preserving machine learning algorithms. As the first batch of novel FL algorithm examples, we release vertical federated kernel binary classification model and vertical federated random forest model. They have been tested to be more efficient than existing vertical federated learning models in our practice. Besides the novel FL algorithm examples, we also release a machine communication module. The uniform data transfer interface supports transferring widely used data formats between machines. We will maintain this platform by adding more functional modules and algorithm examples. The code is available at https://github.com/fedlearnAI/fedlearn-algo.
A Systematic Survey of Text Worlds as Embodied Natural Language Environments
Venue: arXiv cs.CL 2021
Year: 2021

Text Worlds are virtual environments for embodied agents that, unlike 2D or 3D environments, are rendered exclusively using textual descriptions. These environments offer an alternative to higher-fidelity 3D environments due to their low barrier to entry, providing the ability to study semantics, compositional inference, and other high-level tasks with rich high-level action spaces while controlling for perceptual input. This systematic survey outlines recent developments in tooling, environments, and agent modeling for Text Worlds, while examining recent trends in knowledge graphs, common sense reasoning, transfer learning of Text World performance to higher-fidelity environments, as well as near-term development targets that, once achieved, make Text Worlds an attractive general research paradigm for natural language processing.
On pure MDS asymmetric entanglement-assisted quantum error-correcting codes
Venue: arXiv cs.IT 2021
Year: 2021

Recently, Galindo et al. introduced the concept of asymmetric entanglement-assisted quantum error-correcting codes (AEAQECCs) from Calderbank-Shor-Steane (CSS) construction. In general, it's difficult to determine the required number of maximally entangled states of an AEAQECC, which is associated with the dimension of the intersection of the two corresponding linear codes. Two linear codes are said to be a linear l-intersection pair if their intersection has dimension l. In this paper, all possible linear l-intersection pairs of MDS codes are given. As an application, we give a complete characterization of pure MDS AEAQECCs for all possible parameters.
A Survey on RISC-V Security: Hardware and Architecture
Venue: arXiv cs.CR 2021
Year: 2021

The Internet of Things (IoT) is an ongoing technological revolution. Embedded processors are the processing engines of smart IoT devices. For decades, these processors were mainly based on the Arm instruction set architecture (ISA). In recent years, the free and open RISC-V ISA standard has attracted the attention of industry and academia and is becoming the mainstream. Data security and user privacy protection are common challenges faced by all IoT devices. In order to deal with foreseeable security threats, the RISC-V community is studying security solutions aimed at achieving a root of trust (RoT) and ensuring that sensitive information on RISC-V devices is not tampered with or leaked. Many RISC-V security research projects are underway, but the academic community has not yet conducted a comprehensive survey of RISC-V security solutions. In order to fill this research gap, this paper presents an in-depth survey on RISC-V security technologies. This paper summarizes the representative security mechanisms of RISC-V hardware and architecture. Based on our survey, we predict the future research and development directions of RISC-V security. We hope that our research can inspire RISC-V researchers and developers.
Neural-Network-Optimized Degree-Specific Weights for LDPC MinSum Decoding
Venue: arXiv cs.IT 2021
Year: 2021

Neural Normalized MinSum (N-NMS) decoding delivers better frame error rate (FER) performance on linear block codes than conventional normalized MinSum (NMS) by assigning dynamic multiplicative weights to each check-to-variable message in each iteration. Previous N-NMS efforts have primarily investigated short-length block codes (N < 1000), because the number of N-NMS parameters to be trained is proportional to the number of edges in the parity check matrix and the number of iterations, which imposes am impractical memory requirement when Pytorch or Tensorflow is used for training. This paper provides efficient approaches to training parameters of N-NMS that support N-NMS for longer block lengths. Specifically, this paper introduces a family of neural 2-dimensional normalized (N-2D-NMS) decoders with with various reduced parameter sets and shows how performance varies with the parameter set selected. The N-2D-NMS decoders share weights with respect to check node and/or variable node degree. Simulation results justify this approach, showing that the trained weights of N-NMS have a strong correlation to the check node degree, variable node degree, and iteration number. Further simulation results on the (3096,1032) protograph-based raptor-like (PBRL) code show that N-2D-NMS decoder can achieve the same FER as N-NMS with significantly fewer parameters required. The N-2D-NMS decoder for a (16200,7200) DVBS-2 standard LDPC code shows a lower error floor than belief propagation. Finally, a hybrid decoding structure combining a feedforward structure with a recurrent structure is proposed in this paper. The hybrid structure shows similar decoding performance to full feedforward structure, but requires significantly fewer parameters.
Causal Inference for Influence Propagation -- Identifiability of the Independent Cascade Model
Venue: arXiv cs.SI 2021
Year: 2021

Independent cascade (IC) model is a widely used influence propagation model for social networks. In this paper, we incorporate the concept and techniques from causal inference to study the identifiability of parameters from observational data in extended IC model with unobserved confounding factors, which models more realistic propagation scenarios but is rarely studied in influence propagation modeling before. We provide the conditions for the identifiability or unidentifiability of parameters for several special structures including the Markovian IC model, semi-Markovian IC model, and IC model with a global unobserved variable. Parameter identifiability is important for other tasks such as influence maximization under the diffusion networks with unobserved confounding factors.
Control Lyapunov Functions for Compliant Hybrid Zero Dynamic Walking
Venue: arXiv cs.RO 2021
Year: 2021

The ability to realize nonlinear controllers with formal guarantees on dynamic robotic systems has the potential to enable more complex robotic behaviors -- yet, realizing these controllers is often practically challenging. To address this challenge, this paper presents the end-to-end realization of dynamic bipedal locomotion on an underactuated bipedal robot via hybrid zero dynamics and control Lyapunov functions. A compliant model of Cassie is represented as a hybrid system to set the stage for a trajectory optimization framework. With the goal of achieving a variety of walking speeds in all directions, a library of compliant walking motions is compiled and then parameterized for efficient use within real-time controllers. Control Lyapunov functions, which have strong theoretic guarantees, are synthesized to leverage the gait library and coupled with inverse dynamics to obtain optimization-based controllers framed as quadratic programs. It is proven that this controller provably achieves stable locomotion; this is coupled with a theoretic analysis demonstrating useful properties of the controller for tuning and implementation. The proposed theoretic framework is practically demonstrated on the Cassie robot, wherein 3D walking is achieved through the use of optimization-based torque control. The experiments highlight robotic walking at different speeds and terrains, illustrating the end-to-end realization of theoretically justified nonlinear controllers on dynamic underactuated robotic systems.
Unity Perception: Generate Synthetic Data for Computer Vision
Venue: arXiv cs.CV 2021
Year: 2021

We introduce the Unity Perception package which aims to simplify and accelerate the process of generating synthetic datasets for computer vision tasks by offering an easy-to-use and highly customizable toolset. This open-source package extends the Unity Editor and engine components to generate perfectly annotated examples for several common computer vision tasks. Additionally, it offers an extensible Randomization framework that lets the user quickly construct and configure randomized simulation parameters in order to introduce variation into the generated datasets. We provide an overview of the provided tools and how they work, and demonstrate the value of the generated synthetic datasets by training a 2D object detection model. The model trained with mostly synthetic data outperforms the model trained using only real data.
Fast Pixel-Matching for Video Object Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Video object segmentation, aiming to segment the foreground objects given the annotation of the first frame, has been attracting increasing attentions. Many state-of-the-art approaches have achieved great performance by relying on online model updating or mask-propagation techniques. However, most online models require high computational cost due to model fine-tuning during inference. Most mask-propagation based models are faster but with relatively low performance due to failure to adapt to object appearance variation. In this paper, we are aiming to design a new model to make a good balance between speed and performance. We propose a model, called NPMCA-net, which directly localizes foreground objects based on mask-propagation and non-local technique by matching pixels in reference and target frames. Since we bring in information of both first and previous frames, our network is robust to large object appearance variation, and can better adapt to occlusions. Extensive experiments show that our approach can achieve a new state-of-the-art performance with a fast speed at the same time (86.5% IoU on DAVIS-2016 and 72.2% IoU on DAVIS-2017, with speed of 0.11s per frame) under the same level comparison. Source code is available at https://github.com/siyueyu/NPMCA-net.
Differentially private training of neural networks with Langevin dynamics for calibrated predictive uncertainty
Venue: arXiv cs.LG 2021
Year: 2021

We show that differentially private stochastic gradient descent (DP-SGD) can yield poorly calibrated, overconfident deep learning models. This represents a serious issue for safety-critical applications, e.g. in medical diagnosis. We highlight and exploit parallels between stochastic gradient Langevin dynamics, a scalable Bayesian inference technique for training deep neural networks, and DP-SGD, in order to train differentially private, Bayesian neural networks with minor adjustments to the original (DP-SGD) algorithm. Our approach provides considerably more reliable uncertainty estimates than DP-SGD, as demonstrated empirically by a reduction in expected calibration error (MNIST $\sim{5}$-fold, Pediatric Pneumonia Dataset $\sim{2}$-fold).
Understanding surrogate explanations: the interplay between complexity, fidelity and coverage
Venue: arXiv cs.LG 2021
Year: 2021

This paper analyses the fundamental ingredients behind surrogate explanations to provide a better understanding of their inner workings. We start our exposition by considering global surrogates, describing the trade-off between complexity of the surrogate and fidelity to the black-box being modelled. We show that transitioning from global to local - reducing coverage - allows for more favourable conditions on the Pareto frontier of fidelity-complexity of a surrogate. We discuss the interplay between complexity, fidelity and coverage, and consider how different user needs can lead to problem formulations where these are either constraints or penalties. We also present experiments that demonstrate how the local surrogate interpretability procedure can be made interactive and lead to better explanations.
Dynamic Modeling of Bucket-Soil Interactions Using Koopman-DFL Lifting Linearization for Model Predictive Contouring Control of Autonomous Excavators
Venue: arXiv cs.RO 2021
Year: 2021

A lifting-linearization method based on the Koopman operator and Dual Faceted Linearization is applied to the control of a robotic excavator. In excavation, a bucket interacts with the surrounding soil in a highly nonlinear and complex manner. Here, we propose to represent the nonlinear bucket-soil dynamics with a set of linear state equations in a higher-dimensional space. The space of independent state variables is augmented by adding variables associated with nonlinear elements involved in the bucket-soil dynamics. These include nonlinear resistive forces and moment acting on the bucket from the soil, and the effective inertia of the bucket that varies as the soil is captured into the bucket. Variables associated with these nonlinear resistive and inertia elements are treated as additional state variables, and their time evolution is represented as another set of linear differential equations. The lifted linear dynamic model is then applied to Model Predictive Contouring Control, where a cost functional is minimized as a convex optimization problem thanks to the linear dynamics in the lifted space. The lifted linear model is tuned based on a data-driven method by using a soil dynamics simulator. Simulation experiments verify the effectiveness of the proposed lifting linearization compared to its counterpart.
Graph-based Deep Generative Modelling for Document Layout Generation
Venue: arXiv cs.CV 2021
Year: 2021

One of the major prerequisites for any deep learning approach is the availability of large-scale training data. When dealing with scanned document images in real world scenarios, the principal information of its content is stored in the layout itself. In this work, we have proposed an automated deep generative model using Graph Neural Networks (GNNs) to generate synthetic data with highly variable and plausible document layouts that can be used to train document interpretation systems, in this case, specially in digital mailroom applications. It is also the first graph-based approach for document layout generation task experimented on administrative document images, in this case, invoices.
Specialists Outperform Generalists in Ensemble Classification
Venue: arXiv cs.LG 2021
Year: 2021

Consider an ensemble of $k$ individual classifiers whose accuracies are known. Upon receiving a test point, each of the classifiers outputs a predicted label and a confidence in its prediction for this particular test point. In this paper, we address the question of whether we can determine the accuracy of the ensemble. Surprisingly, even when classifiers are combined in the statistically optimal way in this setting, the accuracy of the resulting ensemble classifier cannot be computed from the accuracies of the individual classifiers-as would be the case in the standard setting of confidence weighted majority voting. We prove tight upper and lower bounds on the ensemble accuracy. We explicitly construct the individual classifiers that attain the upper and lower bounds: specialists and generalists. Our theoretical results have very practical consequences: (1) If we use ensemble methods and have the choice to construct our individual (independent) classifiers from scratch, then we should aim for specialist classifiers rather than generalists. (2) Our bounds can be used to determine how many classifiers are at least required to achieve a desired ensemble accuracy. Finally, we improve our bounds by considering the mutual information between the true label and the individual classifier's output.
Hoechst Is All You Need: Lymphocyte Classification with Deep Learning
Venue: arXiv cs.CV 2021
Year: 2021

Multiplex immunofluorescence and immunohistochemistry benefit patients by allowing cancer pathologists to identify several proteins expressed on the surface of cells, enabling cell classification, better understanding of the tumour micro-environment, more accurate diagnoses, prognoses, and tailored immunotherapy based on the immune status of individual patients. However, they are expensive and time consuming processes which require complex staining and imaging techniques by expert technicians. Hoechst staining is much cheaper and easier to perform, but is not typically used in this case as it binds to DNA rather than to the proteins targeted by immunofluorescent techniques, and it was not previously thought possible to differentiate cells expressing these proteins based only on DNA morphology. In this work we show otherwise, training a deep convolutional neural network to identify cells expressing three proteins (T lymphocyte markers CD3 and CD8, and the B lymphocyte marker CD20) with greater than 90% precision and recall, from Hoechst 33342 stained tissue only. Our model learns previously unknown morphological features associated with expression of these proteins which can be used to accurately differentiate lymphocyte subtypes for use in key prognostic metrics such as assessment of immune cell infiltration,and thereby predict and improve patient outcomes without the need for costly multiplex immunofluorescence.
Algebra and coalgebra of stream products
Venue: arXiv cs.FL 2021
Year: 2021

We study connections among polynomials, differential equations and streams over a field K, in terms of algebra and coalgebra. We first introduce the class of (F,G)-products on streams, those where the stream derivative of a product can be expressed as a polynomial of the streams themselves and their derivatives. Our first result is that, for every (F,G)-product, there is a canonical way to construct a transition function on polynomials such that the induced unique final coalgebra morphism from polynomials into streams is the (unique) K-algebra homomorphism -- and vice versa. This implies one can reason algebraically on streams, via their polynomial representation. We apply this result to obtain an algebraic-geometric decision algorithm for polynomial stream equivalence, for an underlying generic (F,G)-product. As an example of reasoning on streams, we focus on specific products (convolution, shuffle, Hadamard) and show how to obtain closed forms of algebraic generating functions of combinatorial sequences, as well as solutions of nonlinear ordinary differential equations.
Cross-modal Attention for MRI and Ultrasound Volume Registration
Venue: arXiv cs.CV 2021
Year: 2021

Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registration. Our proposed cross-modal attention block effectively maps each of the features in one volume to all features in the corresponding volume. Our experimental results demonstrate that a CNN network designed with the cross-modal attention block embedded outperforms an advanced CNN network 10 times of its size. We also incorporated visualization techniques to improve the interpretability of our network. The source code of our work is available at https://github.com/DIAL-RPI/Attention-Reg .
Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion
Venue: arXiv cs.LG 2021
Year: 2021

ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a single experience (e.g. watching a VR video) to a single stress level, which again prevents design of dynamic experiences where real-time in-game stress assessment can be utilized. In this paper, we report our findings on a new study on VR stress assessment, where three stress levels are assessed. ECG data was collected from 9 users experiencing a VR roller coaster. The VR experience was then manually labeled in 10-seconds segments to three stress levels by three raters. We then propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window. Experimental results demonstrate that the proposed model outperforms the classical HRV-based ML models (9% increase in accuracy) and baseline deep learning models (2.5% increase in accuracy). We also report results on the benchmark WESAD dataset to show the supremacy of the model.
Seven Basic Expression Recognition Using ResNet-18
Venue: arXiv cs.CV 2021
Year: 2021

We propose to use a ResNet-18 architecture that was pre-trained on the FER+ dataset for tackling the problem of affective behavior analysis in-the-wild (ABAW) for classification of the seven basic expressions, namely, neutral, anger, disgust, fear, happiness, sadness and surprise. As part of the second workshop and competition on affective behavior analysis in-the-wild (ABAW2), a database consisting of 564 videos with around 2.8M frames is provided along with labels for these seven basic expressions. We resampled the dataset to counter class-imbalances by under-sampling the over-represented classes and over-sampling the under-represented classes along with class-wise weights. To avoid overfitting we performed data-augmentation and used L2 regularisation. Our classifier reaches an ABAW2 score of 0.4 and therefore exceeds the baseline results provided by the hosts of the competition.