OpenRaiser/Intern-Atlas huggingface dataset:
Traceability Transformed: Generating more Accurate Links with Pre-Trained BERT Models
Venue: arXiv cs.SE 2021
Year: 2021

Software traceability establishes and leverages associations between diverse development artifacts. Researchers have proposed the use of deep learning trace models to link natural language artifacts, such as requirements and issue descriptions, to source code; however, their effectiveness has been restricted by availability of labeled data and efficiency at runtime. In this study, we propose a novel framework called Trace BERT (T-BERT) to generate trace links between source code and natural language artifacts. To address data sparsity, we leverage a three-step training strategy to enable trace models to transfer knowledge from a closely related Software Engineering challenge, which has a rich dataset, to produce trace links with much higher accuracy than has previously been achieved. We then apply the T-BERT framework to recover links between issues and commits in Open Source Projects. We comparatively evaluated accuracy and efficiency of three BERT architectures. Results show that a Single-BERT architecture generated the most accurate links, while a Siamese-BERT architecture produced comparable results with significantly less execution time. Furthermore, by learning and transferring knowledge, all three models in the framework outperform classical IR trace models. On the three evaluated real-word OSS projects, the best T-BERT stably outperformed the VSM model with average improvements of 60.31% measured using Mean Average Precision (MAP). RNN severely underperformed on these projects due to insufficient training data, while T-BERT overcame this problem by using pretrained language models and transfer learning.
Semiquantitative Group Testing in at Most Two Rounds
Venue: arXiv cs.IT 2021
Year: 2021

Semiquantitative group testing (SQGT) is a pooling method in which the test outcomes represent bounded intervals for the number of defectives. Alternatively, it may be viewed as an adder channel with quantized outputs. SQGT represents a natural choice for Covid-19 group testing as it allows for a straightforward interpretation of the cycle threshold values produced by polymerase chain reactions (PCR). Prior work on SQGT did not address the need for adaptive testing with a small number of rounds as required in practice. We propose conceptually simple methods for 2-round and nonadaptive SQGT that significantly improve upon existing schemes by using ideas on nonbinary measurement matrices based on expander graphs and list-disjunct matrices.
Designing for Contestation: Insights from Administrative Law
Venue: arXiv cs.HC 2021
Year: 2021

A paper presented at the Workshop on Contestability in Algorithmic Systems at CSCW 2019. Challenging algorithmic decisions is important to decision subjects, yet numerous factors can limit a person's ability to contest such decisions. We propose that administrative law systems, which were created to ensure that governments are kept accountable for their actions and decision making in relation to individuals, can provide guidance on how to design contestation systems for algorithmic decision-making.
RMOPP: Robust Multi-Objective Post-Processing for Effective Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Over the last few decades, many architectures have been developed that harness the power of neural networks to detect objects in near real-time. Training such systems requires substantial time across multiple GPUs and massive labeled training datasets. Although the goal of these systems is generalizability, they are often impractical in real-life applications due to flexibility, robustness, or speed issues. This paper proposes RMOPP: A robust multi-objective post-processing algorithm to boost the performance of fast pre-trained object detectors with a negligible impact on their speed. Specifically, RMOPP is a statistically driven, post-processing algorithm that allows for simultaneous optimization of precision and recall. A unique feature of RMOPP is the Pareto frontier that identifies dominant possible post-processed detectors to optimize for both precision and recall. RMOPP explores the full potential of a pre-trained object detector and is deployable for near real-time predictions. We also provide a compelling test case on YOLOv2 using the MS-COCO dataset.
A comparative study of two-dimensional vocal tract acoustic modeling based on Finite-Difference Time-Domain methods
Venue: arXiv cs.SD 2021
Year: 2021

The two-dimensional (2D) numerical approaches for vocal tract (VT) modelling can afford a better balance between the low computational cost and accurate rendering of acoustic wave propagation. However, they require a high spatio-temporal resolution in the numerical scheme for a precise estimation of acoustic formants at the simulation run-time expense. We have recently proposed a new VT acoustic modelling technique, known as the 2.5D Finite-Difference Time-Domain (2.5D FDTD), which extends the existing 2D FDTD approach by adding tube depth to its acoustic wave solver. In this work, first, the simulated acoustic outputs of our new model are shown to be comparable with the 2D FDTD and a realistic 3D FEM VT model at a low spatio-temporal resolution. Next, a radiation model is developed by including a circular baffle around the VT as head geometry. The transfer functions of the radiation model are analyzed using five different vocal tract shapes for vowel sounds /a/, /e/, /i/, /o/ and /u/.
UVTomo-GAN: An adversarial learning based approach for unknown view X-ray tomographic reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

Tomographic reconstruction recovers an unknown image given its projections from different angles. State-of-the-art methods addressing this problem assume the angles associated with the projections are known a-priori. Given this knowledge, the reconstruction process is straightforward as it can be formulated as a convex problem. Here, we tackle a more challenging setting: 1) the projection angles are unknown, 2) they are drawn from an unknown probability distribution. In this set-up our goal is to recover the image and the projection angle distribution using an unsupervised adversarial learning approach. For this purpose, we formulate the problem as a distribution matching between the real projection lines and the generated ones from the estimated image and projection distribution. This is then solved by reaching the equilibrium in a min-max game between a generator and a discriminator. Our novel contribution is to recover the unknown projection distribution and the image simultaneously using adversarial learning. To accommodate this, we use Gumbel-softmax approximation of samples from categorical distribution to approximate the generator's loss as a function of the unknown image and the projection distribution. Our approach can be generalized to different inverse problems. Our simulation results reveal the ability of our method in successfully recovering the image and the projection distribution in various settings.
Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks
Venue: arXiv cs.CL 2021
Year: 2021

Intent detection and slot filling are two fundamental tasks for building a spoken language understanding (SLU) system. Multiple deep learning-based joint models have demonstrated excellent results on the two tasks. In this paper, we propose a new joint model with a wheel-graph attention network (Wheel-GAT) which is able to model interrelated connections directly for intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges. Intent nodes can provide utterance-level semantic information for slot filling, while slot nodes can also provide local keyword information for intent. Experiments show that our model outperforms multiple baselines on two public datasets. Besides, we also demonstrate that using Bidirectional Encoder Representation from Transformer (BERT) model further boosts the performance in the SLU task.
Trustless, privacy-preserving blockchain bridges
Venue: arXiv cs.CR 2021
Year: 2021

In this paper, we present a protocol for facilitating trust-less cross-chain cryptocurrency transfers that preserve privacy of bridge withdrawals. We leverage zero-knowledge primitives that are commonly used to design cryptocurrency mixing protocols to provide similar functionality but across two or more blockchains. To that end, we receive cryptocurrency mixing for free through the bridge operations and de-scribe how to extend these protocols to incentivise bridge transfers using past ideas. We describe how resulting protocols lead to similar vampire style attacks coined in the Uniswap vs. Sushiswap saga but across chains.
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Venue: arXiv cs.SE 2021
Year: 2021

Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.
Graph-Aided Online Multi-Kernel Learning
Venue: arXiv cs.LG 2021
Year: 2021

Multi-kernel learning (MKL) has been widely used in function approximation tasks. The key problem of MKL is to combine kernels in a prescribed dictionary. Inclusion of irrelevant kernels in the dictionary can deteriorate accuracy of MKL, and increase the computational complexity. To improve the accuracy of function approximation and reduce the computational complexity, the present paper studies data-driven selection of kernels from the dictionary that provide satisfactory function approximations. Specifically, based on the similarities among kernels, the novel framework constructs and refines a graph to assist choosing a subset of kernels. In addition, random feature approximation is utilized to enable online implementation for sequentially obtained data. Theoretical analysis shows that our proposed algorithms enjoy tighter sub-linear regret bound compared with state-of-art graph-based online MKL alternatives. Experiments on a number of real datasets also showcase the advantages of our novel graph-aided framework.
Learning Unsupervised Cross-domain Image-to-Image Translation Using a Shared Discriminator
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised image-to-image translation is used to transform images from a source domain to generate images in a target domain without using source-target image pairs. Promising results have been obtained for this problem in an adversarial setting using two independent GANs and attention mechanisms. We propose a new method that uses a single shared discriminator between the two GANs, which improves the overall efficacy. We assess the qualitative and quantitative results on image transfiguration, a cross-domain translation task, in a setting where the target domain shares similar semantics to the source domain. Our results indicate that even without adding attention mechanisms, our method performs at par with attention-based methods and generates images of comparable quality.
The Exact Rate Memory Tradeoff for Small Caches with Coded Placement
Venue: arXiv cs.IT 2021
Year: 2021

The idea of coded caching was introduced by Maddah-Ali and Niesen who demonstrated the advantages of coding in caching problems. To capture the essence of the problem, they introduced the $(N, K)$ canonical cache network in which $K$ users with independent caches of size $M$ request files from a server that has $N$ files. Among other results, the caching scheme and lower bounds proposed by them led to a characterization of the exact rate memory tradeoff when $M\geq \frac{N}{K}(K-1)$. These lower bounds along with the caching scheme proposed by Chen et al. led to a characterization of the exact rate memory tradeoff when $M\leq \frac{1}{K}$. In this paper we focus on small caches where $M\in \left[0,\frac{N}{K}\right]$ and derive new lower bounds. For the case when $\big\lceil\frac{K+1}{2}\big\rceil\leq N \leq K$ and $M\in \big[\frac{1}{K},\frac{N}{K(N-1)}\big]$, our lower bounds demonstrate that the caching scheme introduced by G{ó}mez-Vilardeb{ó} is optimal and thus extend the characterization of the exact rate memory tradeoff. For the case $1\leq N\leq \big\lceil\frac{K+1}{2}\big\rceil$, we show that the new lower bounds improve upon the previously known lower bounds.
Classification of Handwritten Names of Cities and Handwritten Text Recognition using Various Deep Learning Models
Venue: arXiv cs.CV 2021
Year: 2021

This article discusses the problem of handwriting recognition in Kazakh and Russian languages. This area is poorly studied since in the literature there are almost no works in this direction. We have tried to describe various approaches and achievements of recent years in the development of handwritten recognition models in relation to Cyrillic graphics. The first model uses deep convolutional neural networks (CNNs) for feature extraction and a fully connected multilayer perceptron neural network (MLP) for word classification. The second model, called SimpleHTR, uses CNN and recurrent neural network (RNN) layers to extract information from images. We also proposed the Bluechet and Puchserver models to compare the results. Due to the lack of available open datasets in Russian and Kazakh languages, we carried out work to collect data that included handwritten names of countries and cities from 42 different Cyrillic words, written more than 500 times in different handwriting. We also used a handwritten database of Kazakh and Russian languages (HKR). This is a new database of Cyrillic words (not only countries and cities) for the Russian and Kazakh languages, created by the authors of this work.
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained language models (PLMs) like BERT have made great progress in NLP. News articles usually contain rich textual information, and PLMs have the potentials to enhance news text modeling for various intelligent news applications like news recommendation and retrieval. However, most existing PLMs are in huge size with hundreds of millions of parameters. Many online news applications need to serve millions of users with low latency tolerance, which poses huge challenges to incorporating PLMs in these scenarios. Knowledge distillation techniques can compress a large PLM into a much smaller one and meanwhile keeps good performance. However, existing language models are pre-trained and distilled on general corpus like Wikipedia, which has some gaps with the news domain and may be suboptimal for news intelligence. In this paper, we propose NewsBERT, which can distill PLMs for efficient and effective news intelligence. In our approach, we design a teacher-student joint learning and distillation framework to collaboratively learn both teacher and student models, where the student model can learn from the learning experience of the teacher model. In addition, we propose a momentum distillation method by incorporating the gradients of teacher model into the update of student model to better transfer useful knowledge learned by the teacher model. Extensive experiments on two real-world datasets with three tasks show that NewsBERT can effectively improve the model performance in various intelligent news applications with much smaller models.
A simple artificial damping method for total Lagrangian smoothed particle hydrodynamics
Venue: arXiv cs.CE 2021
Year: 2021

In this paper, we present a simple artificial damping method to enhance the robustness of total Lagrangian smoothed particle hydrodynamics (TL-SPH). Specifically, an artificial damping stress based on the Kelvin-Voigt type damper with a scaling factor imitating a von Neumann-Richtmyer type artificial viscosity is introduced in the constitutive equation to alleviate the spurious oscillation in the vicinity of the sharp spatial gradients. After validating the robustness and accuracy of the present method with a set of benchmark tests with very challenging cases, we demonstrate its potentials in the field of bio-mechanics by simulating the deformation of complex stent structures.
Affordance-Based Mobile Robot Navigation Among Movable Obstacles
Venue: arXiv cs.RO 2021
Year: 2021

Avoiding obstacles in the perceived world has been the classical approach to autonomous mobile robot navigation. However, this usually leads to unnatural and inefficient motions that significantly differ from the way humans move in tight and dynamic spaces, as we do not refrain interacting with the environment around us when necessary. Inspired by this observation, we propose a framework for autonomous robot navigation among movable obstacles (NAMO) that is based on the theory of affordances and contact-implicit motion planning. We consider a realistic scenario in which a mobile service robot negotiates unknown obstacles in the environment while navigating to a goal state. An affordance extraction procedure is performed for novel obstacles to detect their movability, and a contact-implicit trajectory optimization method is used to enable the robot to interact with movable obstacles to improve the task performance or to complete an otherwise infeasible task. We demonstrate the performance of the proposed framework by hardware experiments with Toyota's Human Support Robot.
FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Graph neural network (GNN) is widely used for recommendation to model high-order interactions between users and items. Existing GNN-based recommendation methods rely on centralized storage of user-item graphs and centralized model learning. However, user data is privacy-sensitive, and the centralized storage of user-item graphs may arouse privacy concerns and risk. In this paper, we propose a federated framework for privacy-preserving GNN-based recommendation, which can collectively train GNN models from decentralized user data and meanwhile exploit high-order user-item interaction information with privacy well protected. In our method, we locally train GNN model in each user client based on the user-item graph inferred from the local user-item interaction data. Each client uploads the local gradients of GNN to a server for aggregation, which are further sent to user clients for updating local GNN models. Since local gradients may contain private information, we apply local differential privacy techniques to the local gradients to protect user privacy. In addition, in order to protect the items that users have interactions with, we propose to incorporate randomly sampled items as pseudo interacted items for anonymity. To incorporate high-order user-item interactions, we propose a user-item graph expansion method that can find neighboring users with co-interacted items and exchange their embeddings for expanding the local user-item graphs in a privacy-preserving way. Extensive experiments on six benchmark datasets validate that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile effectively protect user privacy.
Massive Self-Assembly in Grid Environments
Venue: arXiv cs.MA 2021
Year: 2021

Self-assembly plays an essential role in many natural processes, involving the formation and evolution of living or non-living structures, and shows potential applications in many emerging domains. In existing research and practice, there still lacks an ideal self-assembly mechanism that manifests efficiency, scalability, and stability at the same time. Inspired by phototaxis observed in nature, we propose a computational approach for massive self-assembly of connected shapes in grid environments. The key component of this approach is an artificial light field superimposed on a grid environment, which is determined by the positions of all agents and at the same time drives all agents to change their positions, forming a dynamic mutual feedback process. This work advances the understanding and potential applications of self-assembly.
"What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning models are now widely deployed in real-world applications. However, the existence of adversarial examples has been long considered a real threat to such models. While numerous defenses aiming to improve the robustness have been proposed, many have been shown ineffective. As these vulnerabilities are still nowhere near being eliminated, we propose an alternative deployment-based defense paradigm that goes beyond the traditional white-box and black-box threat models. Instead of training a single partially-robust model, one could train a set of same-functionality, yet, adversarially-disjoint models with minimal in-between attack transferability. These models could then be randomly and individually deployed, such that accessing one of them minimally affects the others. Our experiments on CIFAR-10 and a wide range of attacks show that we achieve a significantly lower attack transferability across our disjoint models compared to a baseline of ensemble diversity. In addition, compared to an adversarially trained set, we achieve a higher average robust accuracy while maintaining the accuracy of clean examples.
Distributed Learning for Proportional-Fair Resource Allocation in Coexisting WiFi Networks
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we revisit the widely known performance anomaly that results in severe network utility degradation in WiFi networks when nodes use diverse modulation and coding schemes. The proportional-fair allocation was shown to mitigate this anomaly and provide a good throughput to the stations. It can be achieved through the selection of contention window values based on the explicit solution of an optimization problem or, as proposed recently, by following a learning-based approach that uses a centralized gradient descent algorithm. In this paper, we leverage our recent theoretical work on asynchronous distributed optimization and propose a simple algorithm that allows WiFi nodes to independently tune their contention window to achieve proportional fairness. We compare the throughputs and air-time allocation that this algorithm achieves to those of the standard WiFi binary exponential back-off and show the improvements.