OpenRaiser/Intern-Atlas huggingface dataset:
Decentralized and Communication-Free Multi-Robot Navigation through Distributed Games
Venue: arXiv cs.RO 2021
Year: 2021

Effective multi-robot teams require the ability to move to goals in complex environments in order to address real-world applications such as search and rescue. Multi-robot teams should be able to operate in a completely decentralized manner, with individual robot team members being capable of acting without explicit communication between neighbors. In this paper, we propose a novel game theoretic model that enables decentralized and communication-free navigation to a goal position. Robots each play their own distributed game by estimating the behavior of their local teammates in order to identify behaviors that move them in the direction of the goal, while also avoiding obstacles and maintaining team cohesion without collisions. We prove theoretically that generated actions approach a Nash equilibrium, which also corresponds to an optimal strategy identified for each robot. We show through extensive simulations that our approach enables decentralized and communication-free navigation by a multi-robot system to a goal position, and is able to avoid obstacles and collisions, maintain connectivity, and respond robustly to sensor noise.
PCT: Point cloud transformer
Venue: arXiv cs.CV 2021
Year: 2021

The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation and normal estimation tasks.
Game-theoretic Models of Moral and Other-Regarding Agents
Venue: arXiv cs.GT 2021
Year: 2021

We investigate Kantian equilibria in finite normal form games, a class of non-Nashian, morally motivated courses of action that was recently proposed in the economics literature. We highlight a number of problems with such equilibria, including computational intractability, a high price of miscoordination, and expensive/problematic extension to general normal form games. We point out that such a proper generalization will likely involve the concept of program equilibrium. Finally we propose some general, intuitive, computationally tractable, other-regarding equilibria related to Kantian equilibria, as well as a class of courses of action that interpolates between purely self-regarding and Kantian behavior.
NeurST: Neural Speech Translation Toolkit
Venue: arXiv cs.CL 2021
Year: 2021

NeurST is an open-source toolkit for neural speech translation. The toolkit mainly focuses on end-to-end speech translation, which is easy to use, modify, and extend to advanced speech translation research and products. NeurST aims at facilitating the speech translation research for NLP researchers and building reliable benchmarks for this field. It provides step-by-step recipes for feature extraction, data preprocessing, distributed training, and evaluation. In this paper, we will introduce the framework design of NeurST and show experimental results for different benchmark datasets, which can be regarded as reliable baselines for future research. The toolkit is publicly available at https://github.com/bytedance/neurst/ and we will continuously update the performance of NeurST with other counterparts and studies at https://st-benchmark.github.io/.
Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning
Venue: arXiv cs.CL 2021
Year: 2021

Query reformulation aims to alter noisy or ambiguous text sequences into coherent ones closer to natural language questions. This is to prevent errors from propagating in a client-facing pipeline and promote better communication with users. Besides, it is crucial to maintain performance in downstream environments like question answering when rephrased queries are given as input. We show that under the previous framework (AQA), attempts to alter RL algorithms do not bring significant benefits to either reward acquisition or sequence fluency. Instead, we leverage a query-reformulating text-to-text transformer (QRT5) and apply policy-based RL algorithms to further nudge this reformulator and obtain better answers downstream by generating reward-acquiring query trajectories. QRT5 shows better sample efficiency in RL to achieve the same level of QA performance as the previous approach. It can generate reformulations with more readability based on query well-formedness evaluations and can generalize to out-of-sample data. Our framework is demonstrated to be flexible, allowing reward signals to be sourced from different downstream environments such as intent classification.
AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition
Venue: arXiv cs.CV 2021
Year: 2021

The domain diversities including inconsistent annotation and varied image collection conditions inevitably exist among different facial expression recognition (FER) datasets, which pose an evident challenge for adapting the FER model trained on one dataset to another one. Recent works mainly focus on domain-invariant deep feature learning with adversarial learning mechanism, ignoring the sibling facial action unit (AU) detection task which has obtained great progress. Considering AUs objectively determine facial expressions, this paper proposes an AU-guided unsupervised Domain Adaptive FER (AdaFER) framework to relieve the annotation bias between different FER datasets. In AdaFER, we first leverage an advanced model for AU detection on both source and target domain. Then, we compare the AU results to perform AU-guided annotating, i.e., target faces that own the same AUs with source faces would inherit the labels from source domain. Meanwhile, to achieve domain-invariant compact features, we utilize an AU-guided triplet training which randomly collects anchor-positive-negative triplets on both domains with AUs. We conduct extensive experiments on several popular benchmarks and show that AdaFER achieves state-of-the-art results on all these benchmarks.
Crowd-Driven Mapping, Localization and Planning
Venue: arXiv cs.RO 2021
Year: 2021

Navigation in dense crowds is a well-known open problem in robotics with many challenges in mapping, localization, and planning. Traditional solutions consider dense pedestrians as passive/active moving obstacles that are the cause of all troubles: they negatively affect the sensing of static scene landmarks and must be actively avoided for safety. In this paper, we provide a new perspective: the crowd flow locally observed can be treated as a sensory measurement about the surrounding scenario, encoding not only the scene's traversability but also its social navigation preference. We demonstrate that even using the crowd-flow measurement alone without any sensing about static obstacles, our method still accomplishes good results for mapping, localization, and social-aware planning in dense crowds. Videos of the experiments are available at https://sites.google.com/view/crowdmapping.
An Empirical Investigation of Command-Line Customization
Venue: arXiv cs.SE 2021
Year: 2021

The interactive command line, also known as the shell, is a prominent mechanism used extensively by a wide range of software professionals (engineers, system administrators, data scientists, etc.). Shell customizations can therefore provide insight into the tasks they repeatedly perform, how well the standard environment supports those tasks, and ways in which the environment could be productively extended or modified. To characterize the patterns and complexities of command-line customization, we mined the collective knowledge of command-line users by analyzing more than 2.2 million shell alias definitions found on GitHub. Shell aliases allow command-line users to customize their environment by defining arbitrarily complex command substitutions. Using inductive coding methods, we found three types of aliases that each enable a number of customization practices: Shortcuts (for nicknaming commands, abbreviating subcommands, and bookmarking locations), Modifications (for substituting commands, overriding defaults, colorizing output, and elevating privilege), and Scripts (for transforming data and chaining subcommands). We conjecture that identifying common customization practices can point to particular usability issues within command-line programs, and that a deeper understanding of these practices can support researchers and tool developers in designing better user experiences. In addition to our analysis, we provide an extensive reproducibility package in the form of a curated dataset together with well-documented computational notebooks enabling further knowledge discovery and a basis for learning approaches to improve command-line workflows.
ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on
Venue: arXiv cs.CV 2021
Year: 2021

Virtual try-on has garnered interest as a neural rendering benchmark task to evaluate complex object transfer and scene composition. Recent works in virtual clothing try-on feature a plethora of possible architectural and data representation choices. However, they present little clarity on quantifying the isolated visual effect of each choice, nor do they specify the hyperparameter details that are key to experimental reproduction. Our work, ShineOn, approaches the try-on task from a bottom-up approach and aims to shine light on the visual and quantitative effects of each experiment. We build a series of scientific experiments to isolate effective design choices in video synthesis for virtual clothing try-on. Specifically, we investigate the effect of different pose annotations, self-attention layer placement, and activation functions on the quantitative and qualitative performance of video virtual try-on. We find that DensePose annotations not only enhance face details but also decrease memory usage and training time. Next, we find that attention layers improve face and neck quality. Finally, we show that GELU and ReLU activation functions are the most effective in our experiments despite the appeal of newer activations such as Swish and Sine. We will release a well-organized code base, hyperparameters, and model checkpoints to support the reproducibility of our results. We expect our extensive experiments and code to greatly inform future design choices in video virtual try-on. Our code may be accessed at https://github.com/andrewjong/ShineOn-Virtual-Tryon.
Predictor Antenna: A Technique to Boost the Performance of Moving Relays
Venue: arXiv cs.IT 2021
Year: 2021

In future wireless systems, a large number of users may access the networks via moving relays (MRs) installed on top of vehicles. One of the main challenges of MRs is rapid channel variation which may make channel estimation, and its following procedures, difficult. To address these issues, various schemes are designed, among which predictor antenna (PA) is a candidate. The PA setup refers to a system with two (sets of) antennas on top of a vehicle, where the PA(s) positioned in front of the vehicle is(are) utilized to predict the channel state information required for data transmission to the receive antennas (RAs) aligned behind the PA. In this paper, we introduce the concept and the potentials of PA systems. Moreover, summarizing the field trials for PAs and the 3GPP attempts on (moving) relays, we compare the performance of different PA and non-PA methods for vehicle communications in both urban and rural areas with the PA setup backhauled through terrestrial or satellite technology, respectively. As we show, with typical parameter settings and vehicle speeds, a single-antenna PA-assisted setup can boost the backhaul throughput of MRs, compared to state-of-the-art open-loop schemes, by up to 50%.
Enabling Micro-payments on IoT Devices using Bitcoin Lightning Network
Venue: arXiv cs.CR 2021
Year: 2021

Lightning Network (LN) addresses the scalability problem of Bitcoin by leveraging off-chain transactions. Nevertheless, it is not possible to run LN on resource-constrained IoT devices due to its storage, memory, and processing requirements. Therefore, in this paper, we propose an efficient and secure protocol that enables an IoT device to use LN's functions through a gateway LN node. The idea is to involve the IoT device in LN operations with its digital signature by replacing original 2-of-2 multisignature channels with 3-of-3 multisignature channels. Our protocol enforces the LN gateway to request the IoT device's cryptographic signature for all operations on the channel. We evaluated the proposed protocol by implementing it on a Raspberry Pi for a toll payment scenario and demonstrated its feasibility and security.
More on extension-based semantics of argumentation
Venue: arXiv cs.AI 2021
Year: 2021

After a few decades of development, computational argumentation has become one of the active realms in AI. This paper considers extension-based concrete and abstract semantics of argumentation. For concrete ones, based on Grossi and Modgil's recent work, this paper considers some issues on graded extension-based semantics of abstract argumentation framework (AAF, for short). First, an alternative fundamental lemma is given, which generalizes the corresponding result due to Grossi and Modgil by relaxing the constraint on parameters. This lemma provides a new sufficient condition for preserving conflict-freeness and brings a Galois adjunction between admissible sets and complete extensions, which is of vital importance in constructing some special extensions in terms of iterations of the defense function. Applying such a lemma, some flaws in Grossi and Modgil's work are corrected, and the structural property and universal definability of various extension-based semantics are given. Second, an operator so-called reduced meet modulo an ultrafilter is presented, which is a simple but powerful tool in exploring infinite AAFs. The neutrality function and the defense function, which play central roles in Dung's abstract argumentation theory, are shown to be distributive over reduced meets modulo any ultrafilter. A variety of fundamental semantics of AAFs, including conflict-free, admissible, complete and stable semantics, etc, are shown to be closed under this operator. Based on this fact, a number of applications of such operators are considered. In particular, we provide a simple and uniform method to prove the universal definability of a family of range related semantics. Since all graded concrete semantics considered in this paper are generalizations of corresponding non-graded ones, all results about them obtained in this paper also hold in the traditional situation.
AWA: Adversarial Website Adaptation
Venue: arXiv cs.CR 2021
Year: 2021

One of the most important obligations of privacy-enhancing technologies is to bring confidentiality and privacy to users' browsing activities on the Internet. The website fingerprinting attack enables a local passive eavesdropper to predict the target user's browsing activities even she uses anonymous technologies, such as VPNs, IPsec, and Tor. Recently, the growth of deep learning empowers adversaries to conduct the website fingerprinting attack with higher accuracy. In this paper, we propose a new defense against website fingerprinting attack using adversarial deep learning approaches called Adversarial Website Adaptation (AWA). AWA creates a transformer set in each run so that each website has a unique transformer. Each transformer generates adversarial traces to evade the adversary's classifier. AWA has two versions, including Universal AWA (UAWA) and Non-Universal AWA (NUAWA). Unlike NUAWA, there is no need to access the entire trace of a website in order to generate an adversarial trace in UAWA. We accommodate secret random elements in the training phase of transformers in order for AWA to generate various sets of transformers in each run. We run AWA several times and create multiple sets of transformers. If an adversary and a target user select different sets of transformers, the accuracy of adversary's classifier is almost 19.52% and 31.94% with almost 22.28% and 26.28% bandwidth overhead in UAWA and NUAWA, respectively. If a more powerful adversary generates adversarial traces through multiple sets of transformers and trains a classifier on them, the accuracy of adversary's classifier is almost 49.10% and 25.93% with almost 62.52% and 64.33% bandwidth overhead in UAWA and NUAW, respectively.
Analyzing the Performance of Graph Neural Networks with Pipe Parallelism
Venue: arXiv cs.LG 2021
Year: 2021

Many interesting datasets ubiquitous in machine learning and deep learning can be described via graphs. As the scale and complexity of graph-structured datasets increase, such as in expansive social networks, protein folding, chemical interaction networks, and material phase transitions, improving the efficiency of the machine learning techniques applied to these is crucial. In this study, we focus on Graph Neural Networks (GNN) that have found great success in tasks such as node or edge classification and link prediction. However, standard GNN models have scaling limits due to necessary recursive calculations performed through dense graph relationships that lead to memory and runtime bottlenecks. While new approaches for processing larger networks are needed to advance graph techniques, and several have been proposed, we study how GNNs could be parallelized using existing tools and frameworks that are known to be successful in the deep learning community. In particular, we investigate applying pipeline parallelism to GNN models with GPipe, introduced by Google in 2018.
IntersectX: An Efficient Accelerator for Graph Mining
Venue: arXiv cs.AR 2021
Year: 2021

Graph pattern mining applications try to find all embeddings that match specific patterns. Compared to the traditional graph computation, graph mining applications are computation-intensive. The state-of-the-art method, pattern enumeration, constructs the embeddings that match the pattern. The key operation -- intersection -- of two edge lists, poses challenges to conventional architectures and requires substantial execution time. In this paper, we propose IntersectX, a vertically designed accelerator for pattern enumeration with stream instruction set extension and architectural supports based on conventional processor. The stream based ISA can be considered as a natural extension to the traditional instructions that operate on scalar values. We develop the IntersectX architecture composed of specialized mechanisms that efficiently implement the stream ISA extensions, including: (1) Stream Mapping Table (SMT) that records the mapping between stream ID and stream register; (2) the read-only Stream Cache (S-Cache) that enables efficient stream data movements; (3) tracking the dependency between streams with a property of intersection; (4) Stream Value Processing Unit (SVPU) that implements sparse value computations; and (5) the nested intersection translator that generates micro-op sequences for implementing nested intersections. We implement IntersectX ISA and architecture on zSim, and test it with seven popular graph mining applications (triangle/three-chain/tailed-traingle counting, 3-motif mining, 4/5-clique counting, and FSM) on ten real graphs. We develop our own implementation of AutoMine (InHouseAutomine). The results show that IntersectX significantly outperforms InHouseAutomine on CPU, on average 10.7 times and up to 83.9 times ; and GRAMER, a state-of-the-art graph pattern mining accelerator, based on exhaustive check, on average 40.1 times and up to 181.8 times.
Towards Automatic Digital Documentation and Progress Reporting of Mechanical Construction Pipes using Smartphones
Venue: arXiv cs.CV 2021
Year: 2021

This manuscript presents a new framework towards automated digital documentation and progress reporting of mechanical pipes in building construction projects, using smartphones. New methods were proposed to optimize video frame rate to achieve a desired image overlap; define metric scale for 3D reconstruction; extract pipes from point clouds; and classify pipes according to their planned bill of quantity radii. The effectiveness of the proposed methods in both laboratory (six pipes) and construction site (58 pipes) conditions was evaluated. It was observed that the proposed metric scale definition achieved sub-millimeter pipe radius estimation accuracy. Both laboratory and field experiments revealed that increasing the defined image overlap improved point cloud quality, pipe classification quality, and pipe radius/length estimation. Overall, it was found possible to achieve pipe classification F-measure, radius estimation accuracy, and length estimation percent error of 96.4%, 5.4mm, and 5.0%, respectively, on construction sites using at least 95% image overlap.
PEFP: Efficient k-hop Constrained s-t Simple Path Enumeration on FPGA
Venue: arXiv cs.DB 2021
Year: 2021

Graph plays a vital role in representing entities and their relationships in a variety of fields, such as e-commerce networks, social networks and biological networks. Given two vertices s and t, one of the fundamental problems in graph databases is to investigate the relationships between s and t. A well-studied problem in such area is k-hop constrained s-t simple path enumeration. Nevertheless, all existing algorithms targeting this problem follow the DFS-based paradigm, which cannot scale up well. Moreover, using hardware devices like FPGA to accelerate graph computation has become popular. Motivated by this, in this paper, we propose the first FPGA-based algorithm PEFP to solve the problem of k-hop constrained s-t simple path enumeration efficiently. On the host side, we propose a preprocessing algorithm Pre-BFS to reduce the graph size and search space. On the FPGA side in PEFP, we propose a novel DFS-based batching technique to save on-chip memory efficiently. In addition, we also propose caching techniques to cache necessary data in BRAM, which overcome the latency bottleneck brought by the read/write operations from/to FPGA DRAM. Finally, we propose a data separation technique to enable dataflow optimization for the path verification module; hence the sub-stages in that module can be executed in parallel. Comprehensive experiments show that PEFP outperforms the state-of-the-art algorithm JOIN by more than 1 order of magnitude by average, and up to 2 orders of magnitude in terms of preprocessing time, query processing time and total time, respectively.
Narrative Incoherence Detection
Venue: arXiv cs.CL 2021
Year: 2021

We propose the task of narrative incoherence detection as a new arena for inter-sentential semantic understanding: Given a multi-sentence narrative, decide whether there exist any semantic discrepancies in the narrative flow. Specifically, we focus on the missing sentence and discordant sentence detection. Despite its simple setup, this task is challenging as the model needs to understand and analyze a multi-sentence narrative, and predict incoherence at the sentence level. As an initial step towards this task, we implement several baselines either directly analyzing the raw text (\textit{token-level}) or analyzing learned sentence representations (\textit{sentence-level}). We observe that while token-level modeling has better performance when the input contains fewer sentences, sentence-level modeling performs better on longer narratives and possesses an advantage in efficiency and flexibility. Pre-training on large-scale data and auxiliary sentence prediction training objective further boost the detection performance of the sentence-level model.