OpenRaiser/Intern-Atlas huggingface dataset:
Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers
Venue: arXiv cs.LG 2020
Year: 2020

Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-stepping algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration computationally expensive due to numerical stiffness. In this work, we develop a hierarchy of deep neural network time-steppers to approximate the flow map of the dynamical system over a disparate range of time-scales. The resulting model is purely data-driven and leverages features of the multiscale dynamics, enabling numerical integration and forecasting that is both accurate and highly efficient. Moreover, similar ideas can be used to couple neural network-based models with classical numerical time-steppers. Our multiscale hierarchical time-stepping scheme provides important advantages over current time-stepping algorithms, including (i) circumventing numerical stiffness due to disparate time-scales, (ii) improved accuracy in comparison with leading neural-network architectures, (iii) efficiency in long-time simulation/forecasting due to explicit training of slow time-scale dynamics, and (iv) a flexible framework that is parallelizable and may be integrated with standard numerical time-stepping algorithms. The method is demonstrated on a wide range of nonlinear dynamical systems, including the Van der Pol oscillator, the Lorenz system, the Kuramoto-Sivashinsky equation, and fluid flow pass a cylinder; audio and video signals are also explored. On the sequence generation examples, we benchmark our algorithm against state-of-the-art methods, such as LSTM, reservoir computing, and clockwork RNN. Despite the structural simplicity of our method, it outperforms competing methods on numerical integration.
HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection
Venue: arXiv cs.CL 2020
Year: 2020

Sentiment analysis for code-mixed social media text continues to be an under-explored area. This work adds two common approaches: fine-tuning large transformer models and sample efficient methods like ULMFiT. Prior work demonstrates the efficacy of classical ML methods for polarity detection. Fine-tuned general-purpose language representation models, such as those of the BERT family are benchmarked along with classical machine learning and ensemble methods. We show that NB-SVM beats RoBERTa by 6.2% (relative) F1. The best performing model is a majority-vote ensemble which achieves an F1 of 0.707. The leaderboard submission was made under the codalab username nirantk, with F1 of 0.689.
On the Size of Minimal Separators for Treedepth Decomposition
Venue: arXiv cs.DS 2020
Year: 2020

Treedepth decomposition has several practical applications and can be used to speed up many parameterized algorithms. There are several works aiming to design a scalable algorithm to compute exact treedepth decompositions. Those include works based on a set of all minimal separators. In those algorithms, although a number of minimal separators are enumerated, the minimal separators that are used for an optimal solution are empirically very small. Therefore, analyzing the upper bound on the size of minimal separators is an important problem because it has the potential to significantly reduce the computation time. A minimal separator $S$ is called an optimal top separator if $td(G) = |S| + td(G \backslash S)$, where $td(G)$ denotes the treedepth of $G$. Then, we have two theoretical results on the size of optimal top separators. (1) For any $G$, there is an optimal top separator $S$ such that $|S| \le 2tw(G)$, where $tw(G)$ is the treewidth of $G$. (2) For any $c < 2$, there exists a graph $G$ such that any optimal top separator $S$ of $G$ have $|S| > c \cdot tw(G)$, i.e., the first result gives a tight bound on the size of an optimal top separator.
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
Venue: arXiv cs.CV 2020
Year: 2020

We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at https://github.com/RayRuizhiLiao/joint_chestxray.
Parameter-controlled inserting constructions of constant dimension subspace codes
Venue: arXiv cs.IT 2020
Year: 2020

A basic problem in constant dimension subspace coding is to determine the maximal possible size ${\bf A}_q(n,d,k)$ of a set of $k$-dimensional subspaces in ${\bf F}_q^n$ such that the subspace distance satisfies $\operatorname{dis}(U,V)=2k-2\dim(U \cap V) \geq d$ for any two different $k$-dimensional subspaces $U$ and $V$ in this set. In this paper we propose new parameter-controlled inserting constructions of constant dimension subspace codes. These inserting constructions are flexible because they are controlled by parameters. Several new better lower bounds which are better than all previously constructive lower bounds can be derived from our flexible inserting constructions. $141$ new constant dimension subspace codes of distances $4,6,8$ better than previously best known codes are constructed.
Ptolemy: Architecture Support for Robust Deep Learning
Venue: arXiv cs.AR 2020
Year: 2020

Deep learning is vulnerable to adversarial attacks, where carefully-crafted input perturbations could mislead a well-trained Deep Neural Network to produce incorrect results. Today's countermeasures to adversarial attacks either do not have capability to detect adversarial samples at inference time, or introduce prohibitively high overhead to be practical at inference time. We propose Ptolemy, an algorithm-architecture co-designed system that detects adversarial attacks at inference time with low overhead and high accuracy.We exploit the synergies between DNN inference and imperative program execution: an input to a DNN uniquely activates a set of neurons that contribute significantly to the inference output, analogous to the sequence of basic blocks exercised by an input in a conventional program. Critically, we observe that adversarial samples tend to activate distinctive paths from those of benign inputs. Leveraging this insight, we propose an adversarial sample detection framework, which uses canary paths generated from offline profiling to detect adversarial samples at runtime. The Ptolemy compiler along with the co-designed hardware enable efficient execution by exploiting the unique algorithmic characteristics. Extensive evaluations show that Ptolemy achieves higher or similar adversarial example detection accuracy than today's mechanisms with a much lower runtime (as low as 2%) overhead.
Matching Guided Distillation
Venue: arXiv cs.CV 2020
Year: 2020

Feature distillation is an effective way to improve the performance for a smaller student model, which has fewer parameters and lower computation cost compared to the larger teacher model. Unfortunately, there is a common obstacle - the gap in semantic feature structure between the intermediate features of teacher and student. The classic scheme prefers to transform intermediate features by adding the adaptation module, such as naive convolutional, attention-based or more complicated one. However, this introduces two problems: a) The adaptation module brings more parameters into training. b) The adaptation module with random initialization or special transformation isn't friendly for distilling a pre-trained student. In this paper, we present Matching Guided Distillation (MGD) as an efficient and parameter-free manner to solve these problems. The key idea of MGD is to pose matching the teacher channels with students' as an assignment problem. We compare three solutions of the assignment problem to reduce channels from teacher features with partial distillation loss. The overall training takes a coordinate-descent approach between two optimization objects - assignments update and parameters update. Since MGD only contains normalization or pooling operations with negligible computation cost, it is flexible to plug into network with other distillation methods.
On the Ruin of Age of Information in Augmented Reality over Wireless Terahertz (THz) Networks
Venue: arXiv cs.IT 2020
Year: 2020

Guaranteeing fresh and reliable information for augmented reality (AR) services is a key challenge to enable a real-time experience and sustain a high quality of physical experience (QoPE) for the users. In this paper, a terahertz (THz) cellular network is used to exchange rate-hungry AR content. For this network, guaranteeing an instantaneous low peak age of information (PAoI) is necessary to overcome the uncertainty stemming from the THz channel. In particular, a novel economic concept, namely, the risk of ruin is proposed to examine the probability of occurrence of rare, but extremely high PAoI that can jeopardize the operation of the AR service. To assess the severity of these hazards, the cumulative distribution function (CDF) of the PAoI is derived for two different scheduling policies. This CDF is then used to find the probability of maximum severity of ruin PAoI. Furthermore, to provide long term insights about the AR content's age, the average PAoI of the overall system is also derived. Simulation results show that an increase in the number of users will positively impact the PAoI in both the expected and worst-case scenarios. Meanwhile, an increase in the bandwidth reduces the average PAoI but leads to a decline in the severity of ruin performance. The results also show that a system with preemptive last come first served (LCFS) queues of limited size buffers have a better ruin performance (12% increase in the probability of guaranteeing a less severe PAoI while increasing the number of users), whereas first come first served (FCFS) queues of limited buffers lead to a better average PAoI performance (45% lower PAoI as we increase the bandwidth).
A Cost & Performance-Efficient Field-Programmable Pin-Constrained Digital Microfluidic Biochip
Venue: arXiv cs.ET 2020
Year: 2020

Digital microfluidic biochips (DMFBs) constitute modern generation of Lab-on-Chip (LoC) devices aimed at automation, miniaturization and cost-affordability of biochemistry and laboratory procedures. Over the course of past few years there have been various application-specific and general-purpose DMFBs aimed at reduced manufacturing costs; following the same trend this study presents a general-purpose DMFB with highly competitive characteristics compared with the state-of-the-art DMFBs. The proposed DMFB architecture provides lower Layout / PCB fabrication costs thereby reducing the total manufacturing costs. While more cost-affordable the proposed design is competitive with the state-of-the-art DMFB architectures.
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we investigate the problem of lip-syncing a talking face video of an arbitrary identity to match a target speech segment. Current works excel at producing accurate lip movements on a static image or videos of specific people seen during the training phase. However, they fail to accurately morph the lip movements of arbitrary identities in dynamic, unconstrained talking face videos, resulting in significant parts of the video being out-of-sync with the new audio. We identify key reasons pertaining to this and hence resolve them by learning from a powerful lip-sync discriminator. Next, we propose new, rigorous evaluation benchmarks and metrics to accurately measure lip synchronization in unconstrained videos. Extensive quantitative evaluations on our challenging benchmarks show that the lip-sync accuracy of the videos generated by our Wav2Lip model is almost as good as real synced videos. We provide a demo video clearly showing the substantial impact of our Wav2Lip model and evaluation benchmarks on our website: \url{cvit.iiit.ac.in/research/projects/cvit-projects/a-lip-sync-expert-is-all-you-need-for-speech-to-lip-generation-in-the-wild}. The code and models are released at this GitHub repository: \url{github.com/Rudrabha/Wav2Lip}. You can also try out the interactive demo at this link: \url{bhaasha.iiit.ac.in/lipsync}.
Single-Timescale Stochastic Nonconvex-Concave Optimization for Smooth Nonlinear TD Learning
Venue: arXiv cs.LG 2020
Year: 2020

Temporal-Difference (TD) learning with nonlinear smooth function approximation for policy evaluation has achieved great success in modern reinforcement learning. It is shown that such a problem can be reformulated as a stochastic nonconvex-strongly-concave optimization problem, which is challenging as naive stochastic gradient descent-ascent algorithm suffers from slow convergence. Existing approaches for this problem are based on two-timescale or double-loop stochastic gradient algorithms, which may also require sampling large-batch data. However, in practice, a single-timescale single-loop stochastic algorithm is preferred due to its simplicity and also because its step-size is easier to tune. In this paper, we propose two single-timescale single-loop algorithms which require only one data point each step. Our first algorithm implements momentum updates on both primal and dual variables achieving an $O(\varepsilon^{-4})$ sample complexity, which shows the important role of momentum in obtaining a single-timescale algorithm. Our second algorithm improves upon the first one by applying variance reduction on top of momentum, which matches the best known $O(\varepsilon^{-3})$ sample complexity in existing works. Furthermore, our variance-reduction algorithm does not require a large-batch checkpoint. Moreover, our theoretical results for both algorithms are expressed in a tighter form of simultaneous primal and dual side convergence.
Optimization of Quality of Experience for Video Traffic
Venue: arXiv cs.NI 2020
Year: 2020

The rapid shift toward video on-demand and real time information systems has affected mobile as well as wired networks. The research community has placed a strong focus on optimizing the Quality of Experience (QoE) of video traffic, mainly because video is popular among Internet users. Techniques have been proposed in different directions towards improvement of the perception of video users. This paper investigates the performance of a novel cross-layer architecture for optimizing the QoE of video traffic. The proposed architecture is compared to two other architectures; non-adaptive and adaptive. For the former, video traffic is sent without adaptation, whereas for the later video sources adapt their transmission rate. Both are compared in terms of the mean opinion score of video sessions, number of sessions, delay, packet drop ratio, jitter and utilization. The results from extensive simulations show that the proposed architecture outperforms the non-adaptive and adaptive architectures for video traffic.
Handling of uncertainty in medical data using machine learning and probability theory techniques: A review of 30 years (1991-2020)
Venue: arXiv cs.AI 2020
Year: 2020

Understanding data and reaching valid conclusions are of paramount importance in the present era of big data. Machine learning and probability theory methods have widespread application for this purpose in different fields. One critically important yet less explored aspect is how data and model uncertainties are captured and analyzed. Proper quantification of uncertainty provides valuable information for optimal decision making. This paper reviewed related studies conducted in the last 30 years (from 1991 to 2020) in handling uncertainties in medical data using probability theory and machine learning techniques. Medical data is more prone to uncertainty due to the presence of noise in the data. So, it is very important to have clean medical data without any noise to get accurate diagnosis. The sources of noise in the medical data need to be known to address this issue. Based on the medical data obtained by the physician, diagnosis of disease, and treatment plan are prescribed. Hence, the uncertainty is growing in healthcare and there is limited knowledge to address these problems. We have little knowledge about the optimal treatment methods as there are many sources of uncertainty in medical science. Our findings indicate that there are few challenges to be addressed in handling the uncertainty in medical raw data and new models. In this work, we have summarized various methods employed to overcome this problem. Nowadays, application of novel deep learning techniques to deal such uncertainties have significantly increased.
Good Graph to Optimize: Cost-Effective, Budget-Aware Bundle Adjustment in Visual SLAM
Venue: arXiv cs.CV 2020
Year: 2020

The cost-efficiency of visual(-inertial) SLAM (VSLAM) is a critical characteristic of resource-limited applications. While hardware and algorithm advances have been significantly improved the cost-efficiency of VSLAM front-ends, the cost-efficiency of VSLAM back-ends remains a bottleneck. This paper describes a novel, rigorous method to improve the cost-efficiency of local BA in a BA-based VSLAM back-end. An efficient algorithm, called Good Graph, is developed to select size-reduced graphs optimized in local BA with condition preservation. To better suit BA-based VSLAM back-ends, the Good Graph predicts future estimation needs, dynamically assigns an appropriate size budget, and selects a condition-maximized subgraph for BA estimation. Evaluations are conducted on two scenarios: 1) VSLAM as standalone process, and 2) VSLAM as part of closed-loop navigation system. Results from the first scenario show Good Graph improves accuracy and robustness of VSLAM estimation, when computational limits exist. Results from the second scenario, indicate that Good Graph benefits the trajectory tracking performance of VSLAM-based closed-loop navigation systems, which is a primary application of VSLAM.
A Lagrangian Dual-based Theory-guided Deep Neural Network
Venue: arXiv cs.LG 2020
Year: 2020

The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of TgNN. We convert the original loss function into a constrained form with fewer items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable tradeoff between observation data and corresponding constraints, in order to improve prediction accuracy, and conserve time and computational resources adjusted by an ad-hoc procedure. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.
Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

Conditional Maximum Mean Discrepancy (CMMD) can capture the discrepancy between conditional distributions by drawing support from nonlinear kernel functions, thus it has been successfully used for pattern classification. However, CMMD does not work well on complex distributions, especially when the kernel function fails to correctly characterize the difference between intra-class similarity and inter-class similarity. In this paper, a new kernel learning method is proposed to improve the discrimination performance of CMMD. It can be operated with deep network features iteratively and thus denoted as KLN for abbreviation. The CMMD loss and an auto-encoder (AE) are used to learn an injective function. By considering the compound kernel, i.e., the injective function with a characteristic kernel, the effectiveness of CMMD for data category description is enhanced. KLN can simultaneously learn a more expressive kernel and label prediction distribution, thus, it can be used to improve the classification performance in both supervised and semi-supervised learning scenarios. In particular, the kernel-based similarities are iteratively learned on the deep network features, and the algorithm can be implemented in an end-to-end manner. Extensive experiments are conducted on four benchmark datasets, including MNIST, SVHN, CIFAR-10 and CIFAR-100. The results indicate that KLN achieves state-of-the-art classification performance.
Fair Proof-of-Stake using VDF+VRF Consensus
Venue: arXiv cs.CR 2020
Year: 2020

We propose a new Proof-of-Stake consensus protocol constructed with a verifiable random function (VRF) and a verifiable delay function (VDF) that has the following properties: a) all addresses with positive stake can participate; b) is fair because the coin stake is proportional to the distribution of rewards; c) is resistant to several classic blockchain attacks such as Sybil attacks, "Nothing-at-stake" attacks and "Winner-takes-all" attacks. We call it Vixify Consensus.
Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep Reinforcement Learning Approach
Venue: arXiv cs.RO 2020
Year: 2020

Industrial robot manipulators are playing a more significant role in modern manufacturing industries. Though peg-in-hole assembly is a common industrial task which has been extensively researched, safely solving complex high precision assembly in an unstructured environment remains an open problem. Reinforcement Learning (RL) methods have been proven successful in solving manipulation tasks autonomously. However, RL is still not widely adopted on real robotic systems because working with real hardware entails additional challenges, especially when using position-controlled manipulators. The main contribution of this work is a learning-based method to solve peg-in-hole tasks with position uncertainty of the hole. We proposed the use of an off-policy model-free reinforcement learning method and bootstrap the training speed by using several transfer learning techniques (sim2real) and domain randomization. Our proposed learning framework for position-controlled robots was extensively evaluated on contact-rich insertion tasks on a variety of environments.