OpenRaiser/Intern-Atlas huggingface dataset:
Almost Envy-free Repeated Matching in Two-sided Markets
Venue: arXiv cs.GT 2020
Year: 2020

A two-sided market consists of two sets of agents, each of whom have preferences over the other (Airbnb, Upwork, Lyft, Uber, etc.). We propose and analyze a repeated matching problem, where some set of matches occur on each time step, and our goal is to ensure fairness with respect to the cumulative allocations over an infinite time horizon. Our main result is a polynomial-time algorithm for additive, symmetric (v_i(j) = v_j(i)), and binary (v_i(j) \in \{a,1\}) valuations that both (1) guarantees "envy-freeness up to a single match" (EF1) and (2) selects a maximum weight matching on each time step. Thus for this class of valuations, fairness can be achieved without sacrificing economic efficiency. This result holds even for "dynamic valuations", i.e., valuations that change over time. Although symmetry is a strong assumption, we show that this result cannot be extended to asymmetric binary valuations: (1) and (2) together are impossible even when valuations do not change over time, and for dynamic valuations, even (1) alone is impossible. To our knowledge, this is the first analysis of envy-freeness in a repeated matching setting.
TorchDyn: A Neural Differential Equations Library
Venue: arXiv cs.LG 2020
Year: 2020

Continuous-depth learning has recently emerged as a novel perspective on deep learning, improving performance in tasks related to dynamical systems and density estimation. Core to these approaches is the neural differential equation, whose forward passes are the solutions of an initial value problem parametrized by a neural network. Unlocking the full potential of continuous-depth models requires a different set of software tools, due to peculiar differences compared to standard discrete neural networks, e.g inference must be carried out via numerical solvers. We introduce TorchDyn, a PyTorch library dedicated to continuous-depth learning, designed to elevate neural differential equations to be as accessible as regular plug-and-play deep learning primitives. This objective is achieved by identifying and subdividing different variants into common essential components, which can be combined and freely repurposed to obtain complex compositional architectures. TorchDyn further offers step-by-step tutorials and benchmarks designed to guide researchers and contributors.
A Hybrid Simulation-based Duopoly Game Framework for Analysis of Supply Chain and Marketing Activities
Venue: arXiv cs.GT 2020
Year: 2020

A hybrid simulation-based framework involving system dynamics and agent-based simulation is proposed to address duopoly game considering multiple strategic decision variables and rich payoff, which cannot be addressed by traditional approaches involving closed-form equations. While system dynamics models are used to represent integrated production, logistics, and pricing determination activities of duopoly companies, agent-based simulation is used to mimic enhanced consumer purchasing behavior considering advertisement, promotion effect, and acquaintance recommendation in the consumer social network. The payoff function of the duopoly companies is assumed to be the net profit based on the total revenue and various cost items such as raw material, production, transportation, inventory and backorder. A unique procedure is proposed to solve and analyze the proposed simulation-based game, where the procedural components include strategy refinement, data sampling, gaming solving, and performance evaluation. First, design of experiment and estimated conformational value of information techniques are employed for strategy refinement and data sampling, respectively. Game solving then focuses on pure strategy equilibriums, and performance evaluation addresses game stability, equilibrium strictness, and robustness. A hypothetical case scenario involving soft-drink duopoly on Coke and Pepsi is considered to illustrate and demonstrate the proposed approach. Final results include P-values of statistical tests, confidence intervals, and simulation steady state analysis for different pure equilibriums.
An Improved Approach of Intention Discovery with Machine Learning for POMDP-based Dialogue Management
Venue: arXiv cs.AI 2020
Year: 2020

An Embodied Conversational Agent (ECA) is an intelligent agent that works as the front end of software applications to interact with users through verbal/nonverbal expressions and to provide online assistance without the limits of time, location, and language. To help to improve the experience of human-computer interaction, there is an increasing need to empower ECA with not only the realistic look of its human counterparts but also a higher level of intelligence. This thesis first highlights the main topics related to the construction of ECA, including different approaches of dialogue management, and then discusses existing techniques of trend analysis for its application in user classification. As a further refinement and enhancement to prior work on ECA, this thesis research proposes a cohesive framework to integrate emotion-based facial animation with improved intention discovery. In addition, a machine learning technique is introduced to support sentiment analysis for the adjustment of policy design in POMDP-based dialogue management. The proposed research work is going to improve the accuracy of intention discovery while reducing the length of dialogues.
Estimation of Individual Device Contributions for Incentivizing Federated Learning
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning (FL) is an emerging technique used to train a machine-learning model collaboratively using the data and computation resource of the mobile devices without exposing privacy-sensitive user data. Appropriate incentive mechanisms that motivate the data and mobile-device owner to participate in FL is key to building a sustainable platform for FL. However, it is difficult to evaluate the contribution level of the devices/owners to determine appropriate rewards without large computation and communication overhead. This paper proposes a computation-and communication-efficient method of estimating a participating device's contribution level. The proposed method enables such estimation during a single FL training process, there by reducing the need for traffic and computation overhead. The performance evaluations using the MNIST dataset show that the proposed method estimates individual participants' contributions accurately with 46-49% less computation overhead and no communication overhead than a naive estimation method.
Softmax Tempering for Training Neural Machine Translation Models
Venue: arXiv cs.CL 2020
Year: 2020

Neural machine translation (NMT) models are typically trained using a softmax cross-entropy loss where the softmax distribution is compared against smoothed gold labels. In low-resource scenarios, NMT models tend to over-fit because the softmax distribution quickly approaches the gold label distribution. To address this issue, we propose to divide the logits by a temperature coefficient, prior to applying softmax, during training. In our experiments on 11 language pairs in the Asian Language Treebank dataset and the WMT 2019 English-to-German translation task, we observed significant improvements in translation quality by up to 3.9 BLEU points. Furthermore, softmax tempering makes the greedy search to be as good as beam search decoding in terms of translation quality, enabling 1.5 to 3.5 times speed-up. We also study the impact of softmax tempering on multilingual NMT and recurrently stacked NMT, both of which aim to reduce the NMT model size by parameter sharing thereby verifying the utility of temperature in developing compact NMT models. Finally, an analysis of softmax entropies and gradients reveal the impact of our method on the internal behavior of NMT models.
A Model for Reliable Uplink Transmissions in LoRaWAN
Venue: arXiv cs.NI 2020
Year: 2020

Long range wide area networks (LoRaWAN) technology provides a simple solution to enable low-cost services for low power internet-of-things (IoT) networks in various applications. The current evaluation of LoRaWAN networks relies on simulations or early testing, which are typically time consuming and prevent effective exploration of the design space. This paper proposes an analytical model to calculate the delay and energy consumed for reliable Uplink (UL) data delivery in Class A LoRaWAN. The analytical model is evaluated using a real network test-bed as well as simulation experiments based on the ns-3 LoRaWAN module. The resulting comparison confirms that the model accurately estimates the delay and energy consumed in the considered environment. The value of the model is demonstrated via its application to evaluate the impact of the number of end-devices and the maximum number of data frame retransmissions on delay and energy consumed for the confirmed UL data delivery in LoRaWAN networks. The model can be used to optimize different transmission parameters in future LoRaWAN networks.
MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition with Multi-Domain Deep Learning Model
Venue: arXiv cs.CV 2020
Year: 2020

Together with the rapid development of the Internet of Things (IoT), human activity recognition (HAR) using wearable Inertial Measurement Units (IMUs) becomes a promising technology for many research areas. Recently, deep learning-based methods pave a new way of understanding and performing analysis of the complex data in the HAR system. However, the performance of these methods is mostly based on the quality and quantity of the collected data. In this paper, we innovatively propose to build a large database based on virtual IMUs and then address technical issues by introducing a multiple-domain deep learning framework consisting of three technical parts. In the first part, we propose to learn the single-frame human activity from the noisy IMU data with hybrid convolutional neural networks (CNNs) in the semi-supervised form. For the second part, the extracted data features are fused according to the principle of uncertainty-aware consistency, which reduces the uncertainty by weighting the importance of the features. The transfer learning is performed in the last part based on the newly released Archive of Motion Capture as Surface Shapes (AMASS) dataset, containing abundant synthetic human poses, which enhances the variety and diversity of the training dataset and is beneficial for the process of training and feature transfer in the proposed method. The efficiency and effectiveness of the proposed method have been demonstrated in the real deep inertial poser (DIP) dataset. The experimental results show that the proposed methods can surprisingly converge within a few iterations and outperform all competing methods.
ContourCNN: convolutional neural network for contour data classification
Venue: arXiv cs.CV 2020
Year: 2020

This paper proposes a novel Convolutional Neural Network model for contour data analysis (ContourCNN) and shape classification. A contour is a circular sequence of points representing a closed shape. For handling the cyclical property of the contour representation, we employ circular convolution layers. Contours are often represented sparsely. To address information sparsity, we introduce priority pooling layers that select features based on their magnitudes. Priority pooling layers pool features with low magnitudes while leaving the rest unchanged. We evaluated the proposed model using letters and digits shapes extracted from the EMNIST dataset and obtained a high classification accuracy.
Implicit Feature Networks for Texture Completion from Partial 3D Data
Venue: arXiv cs.CV 2020
Year: 2020

Prior work to infer 3D texture use either texture atlases, which require uv-mappings and hence have discontinuities, or colored voxels, which are memory inefficient and limited in resolution. Recent work, predicts RGB color at every XYZ coordinate forming a texture field, but focus on completing texture given a single 2D image. Instead, we focus on 3D texture and geometry completion from partial and incomplete 3D scans. IF-Nets have recently achieved state-of-the-art results on 3D geometry completion using a multi-scale deep feature encoding, but the outputs lack texture. In this work, we generalize IF-Nets to texture completion from partial textured scans of humans and arbitrary objects. Our key insight is that 3D texture completion benefits from incorporating local and global deep features extracted from both the 3D partial texture and completed geometry. Specifically, given the partial 3D texture and the 3D geometry completed with IF-Nets, our model successfully in-paints the missing texture parts in consistence with the completed geometry. Our model won the SHARP ECCV'20 challenge, achieving highest performance on all challenges.
Experimental Evaluation of LoRaWAN in NS-3
Venue: arXiv cs.NI 2020
Year: 2020

Long Range Wide Area Networks (LoRaWAN) is an open medium access control (MAC) layer technology devised for the long range connectivity of massive number of low power network devices. This work gives an overview of the key aspects of LoRaWAN technology and presents results that we achieved via extensive evaluation of Class A LoRaWAN devices in different network settings using the state-of-the-art network simulator (NS-3). At first, we focus on a single device and its mobility. We further undertook evaluations in an extended network scenario with a changing number of devices and traffic intensity. In particular, we evaluate the packet delivery ratio (PDR), uplink (UL) throughput, and sub-band utilization for the confirmed and unconfirmed UL transmissions in different environments. Our results give new insights for future efforts to optimize the LoRaWAN performance for different large scale Internet of Things (IoT) applications with low power end devices.
Heterogeneous Runtime Verification of Safety Critical Cyber Physical Systems
Venue: arXiv cs.SE 2020
Year: 2020

Advanced embedded system technology is one of the key driving forces behind the rapid growth of Cyber-Physical System (CPS) applications. Cyber-Physical Systems are comprised of multiple coordinating and cooperating components, which are often software intensive and interacting with each other to achieve unprecedented tasks. Such complex CPSs have multiple attack surfaces and attack vectors that we have to secure against. Towards this goal, we demonstrate a multilevel runtime safety and security monitor framework where there are monitors across the CPS for detection and isolation of attacks. We implement the runtime monitors on FPGA using a stream-based runtime verification tool called TeSSLa. We demonstrate our monitoring scheme for an Autonomous Emergency Braking (AEB) CPS system.
Efficient Design of Subblock Energy-Constrained Codes and Sliding Window-Constrained Codes
Venue: arXiv cs.IT 2020
Year: 2020

The subblock energy-constrained codes (SECCs) and sliding window-constrained codes (SWCCs) have recently attracted attention due to various applications in communcation systems such as simultaneous energy and information transfer. In a SECC, each codewod is divided into smaller non-overlapping windows, called subblocks, and every subblock is constrained to carry sufficient energy. In a SWCC, the energy constraint is enforced over every window. In this work, we focus on the binary channel, where sufficient energy is achieved theoretically by using relatively high weight codes, and study SECCs and SWCCs under more general constraints, namely bounded SECCs and bounded SWCCs. We propose two methods to construct such codes with low redundancy and linear-time complexity, based on Knuth's balancing technique and sequence replacement technique. For certain codes parameters, our methods incur only one redundant bit. We also impose the minimum distance constraint for error correction capability of the designed codes, which helps to reduce the error propagation during decoding as well.
On the Performance of Generative Adversarial Network (GAN) Variants: A Clinical Data Study
Venue: arXiv cs.NE 2020
Year: 2020

Generative Adversarial Network (GAN) is a useful type of Neural Networks in various types of applications including generative models and feature extraction. Various types of GANs are being researched with different insights, resulting in a diverse family of GANs with a better performance in each generation. This review focuses on various GANs categorized by their common traits.
Improving Robustness and Generality of NLP Models Using Disentangled Representations
Venue: arXiv cs.CL 2020
Year: 2020

Supervised neural networks, which first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$, have achieved remarkable success in a wide range of natural language processing (NLP) tasks. Despite their success, neural models lack for both robustness and generality: small perturbations to inputs can result in absolutely different outputs; the performance of a model trained on one domain drops drastically when tested on another domain. In this paper, we present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. Instead of mapping $x$ to a single representation $z$, the proposed strategy maps $x$ to a set of representations $\{z_1,z_2,...,z_K\}$ while forcing them to be disentangled. These representations are then mapped to different logits $l$s, the ensemble of which is used to make the final prediction $y$. We propose different methods to incorporate this idea into currently widely-used models, including adding an $L$2 regularizer on $z$s or adding Total Correlation (TC) under the framework of variational information bottleneck (VIB). We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.
Assessing the Severity of Health States based on Social Media Posts
Venue: arXiv cs.CL 2020
Year: 2020

The unprecedented growth of Internet users has resulted in an abundance of unstructured information on social media including health forums, where patients request health-related information or opinions from other users. Previous studies have shown that online peer support has limited effectiveness without expert intervention. Therefore, a system capable of assessing the severity of health state from the patients' social media posts can help health professionals (HP) in prioritizing the user's post. In this study, we inspect the efficacy of different aspects of Natural Language Understanding (NLU) to identify the severity of the user's health state in relation to two perspectives(tasks) (a) Medical Condition (i.e., Recover, Exist, Deteriorate, Other) and (b) Medication (i.e., Effective, Ineffective, Serious Adverse Effect, Other) in online health communities. We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. Specifically, our model utilizes the NLU views such as sentiment, emotions, personality, and use of figurative language to extract the contextual information. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
The Complexity of Constrained Min-Max Optimization
Venue: arXiv cs.CC 2020
Year: 2020

Despite its important applications in Machine Learning, min-max optimization of nonconvex-nonconcave objectives remains elusive. Not only are there no known first-order methods converging even to approximate local min-max points, but the computational complexity of identifying them is also poorly understood. In this paper, we provide a characterization of the computational complexity of the problem, as well as of the limitations of first-order methods in constrained min-max optimization problems with nonconvex-nonconcave objectives and linear constraints. As a warm-up, we show that, even when the objective is a Lipschitz and smooth differentiable function, deciding whether a min-max point exists, in fact even deciding whether an approximate min-max point exists, is NP-hard. More importantly, we show that an approximate local min-max point of large enough approximation is guaranteed to exist, but finding one such point is PPAD-complete. The same is true of computing an approximate fixed point of Gradient Descent/Ascent. An important byproduct of our proof is to establish an unconditional hardness result in the Nemirovsky-Yudin model. We show that, given oracle access to some function $f : P \to [-1, 1]$ and its gradient $\nabla f$, where $P \subseteq [0, 1]^d$ is a known convex polytope, every algorithm that finds a $\varepsilon$-approximate local min-max point needs to make a number of queries that is exponential in at least one of $1/\varepsilon$, $L$, $G$, or $d$, where $L$ and $G$ are respectively the smoothness and Lipschitzness of $f$ and $d$ is the dimension. This comes in sharp contrast to minimization problems, where finding approximate local minima in the same setting can be done with Projected Gradient Descent using $O(L/\varepsilon)$ many queries. Our result is the first to show an exponential separation between these two fundamental optimization problems.
An Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution
Venue: arXiv cs.NE 2020
Year: 2020

Weight initialization is critical in being able to successfully train artificial neural networks (ANNs), and even more so for recurrent neural networks (RNNs) which can easily suffer from vanishing and exploding gradients. In neuroevolution, where evolutionary algorithms are applied to neural architecture search, weights typically need to be initialized at three different times: when initial genomes (ANN architectures) are created at the beginning of the search, when offspring genomes are generated by crossover, and when new nodes or edges are created during mutation. This work explores the difference between using Xavier, Kaiming, and uniform random weight initialization methods, as well as novel Lamarckian weight inheritance methods for initializing new weights during crossover and mutation operations. These are examined using the Evolutionary eXploration of Augmenting Memory Models (EXAMM) neuroevolution algorithm, which is capable of evolving RNNs with a variety of modern memory cells (e.g., LSTM, GRU, MGU, UGRNN and Delta-RNN cells) as well recurrent connections with varying time skips through a high performance island based distributed evolutionary algorithm. Results show that with statistical significance, utilizing the Lamarckian strategies outperforms Kaiming, Xavier and uniform random weight initialization, and can speed neuroevolution by requiring less backpropagation epochs to be evaluated for each generated RNN.
DeepDyve: Dynamic Verification for Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Deep neural networks (DNNs) have become one of the enabling technologies in many safety-critical applications, e.g., autonomous driving and medical image analysis. DNN systems, however, suffer from various kinds of threats, such as adversarial example attacks and fault injection attacks. While there are many defense methods proposed against maliciously crafted inputs, solutions against faults presented in the DNN system itself (e.g., parameters and calculations) are far less explored. In this paper, we develop a novel lightweight fault-tolerant solution for DNN-based systems, namely DeepDyve, which employs pre-trained neural networks that are far simpler and smaller than the original DNN for dynamic verification. The key to enabling such lightweight checking is that the smaller neural network only needs to produce approximate results for the initial task without sacrificing fault coverage much. We develop efficient and effective architecture and task exploration techniques to achieve optimized risk/overhead trade-off in DeepDyve. Experimental results show that DeepDyve can reduce 90% of the risks at around 10% overhead.